This article explores material-semiotic movements of literacy and the collective potentials of an estranged relationship to language in early childhood bilingual education. It engages three different situations from a bilingual, Spanish-Swedish early childhood institution in Sweden with notions of a minor literature, as it appears in Deleuze and Guattari's writings. This engagement puts forward the material presences of language as well as the creativity of what could be called an (il)literate expertise: balancing and experimenting on the cutting edge between things and propositions. This involves an intensive, deterritorializing treatment of language, a "becoming minor" with language in passionate, sensual, experimenting ways, rather than reproducing a standard, already known version of language. The discussion highlights the inventive processes of biliteracy as well as the need to differentiate the notion of expertise in early childhood bilingual education.