The aim of this paper is to find roots of the private language argument in older Wittgenstein‘s texts. Scrutiny of them reveals that the more general form of the argument, so called solitary language argument, precedes the traditional sense-data argument from the first Wittgenstein‘s consideration of the problem in The Brown Book with a continuation in lectures on sense data and private experience. In consequence, this undermines exegetical objections to the solitary language argument.