To reveal the formation process of speech processing with early hearing experiences, we tracked the development of functional connectivity in the auditory and language-related cortical areas of 84 (36 female) congenitally deafened toddlers using repeated functional near-infrared spectroscopy for up to 36 months post cochlear implantation (CI). Upon hearing restoration, the CI children lacked the modular organization of the mature speech-processing network and demonstrated a higher degree of immaturity in temporo-parietal than temporo-frontal connections. The speech-processing network appeared to form rapidly with early CI experiences, with two-thirds of the developing connections following nonlinear trajectories reflecting possibly more than one synaptogenesis-pruning cycle. A few key features of the mature speech-processing network emerged within the first year of CI hearing, including left-hemispheric advantage, differentiation of the dorsal and ventral processing streams, and functional state (speech listening vs. resting) specific patterns of connectivity development. The developmental changes were predictable of future auditory and verbal communication skills of the CI children, with prominent contribution from temporo-parietal connections in the dorsal stream, suggesting a mediating role of speech-processing network formation with early hearing experiences in speech acquisition.