Speech Recognition for Early Learning

Why children's speech is hard

Current automatic speech recognition (ASR) models transcribe adult speech well but struggle with children's speech.

Word error rate on child speech vs. adult benchmarks

Models trained on adult speech don't work for kids because child speech is fundamentally different. Kids have distinct vocal characteristics, different speech patterns, and make frequent speech shortcuts and pronunciation errors, all of which make understanding them difficult.

This poor performance is a significant barrier to educational tools that support early learning. Addressing it requires assembling more representative data and developing better ASR models built for children's speech.

Models

Open, retrained speech recognition models for word- and phone-level transcription of children's speech.

The ASR models developed in this project were created in two phases:

• The competition surfaced the best modeling approaches. A global community with a wide variety of backgrounds explored the solution space to find the most effective open approaches, each evaluated against the same test set.
• Post-challenge retraining focused on the best final models for release. The winning approaches were streamlined to be easier to run, retrained on more data, and packaged with clear documentation and evaluation of results.

The result is two open-weight models on Hugging Face, one for each kind of transcription. Both are released under the BigCode OpenRAIL-M license, which allows commercial and noncommercial use subject to responsible-use restrictions, and each has a model card covering its training data, intended uses, evaluation, and known limitations:

• pasketti-word 🍝 Word-level transcription: which words the child said
• pasketti-phonetic 🍝 Phonetic (IPA) transcription: exactly how the child said it

Both final models build on the challenge's top models:

• pasketti-word is a full fine-tune of Qwen3-ASR-1.7B, trained on 372 hours of child speech from more than 3,600 children in clips of up to 20 seconds.
• pasketti-phonetic streamlines the challenge's large ensembles into a single WavLM Large model, trained on 116 hours of phonetically transcribed speech from about 1,700 children. A second, word-level output used only during training let it learn from an additional 371 hours of word-transcribed speech.

While the competition models were trained only on competition data, the retrained versions add a larger and more representative dataset, including sensitive data from realistic education populations that could not be shared with solvers, and data deliberately gathered from groups where current models perform worst, such as children with atypical speech development, non-white students, and students from low-income backgrounds. Each model was then tested on held-out data and checked for differences in performance by age, race, speech pathology, sex, setting, and source corpus.

Choosing the right model

The two models capture different outputs and therefore serve different purposes:

How the final models perform

Word model: Word error rate

Phonetic model: Phone error rate

Both models improved on the leading open alternatives across every age and demographic group in their evaluation sets, demonstrating the importance of diverse training data.

• pasketti-word has a word error rate (WER) of 17%, about half that of KidWhisper (32%) and OpenAI Whisper (34%).
• pasketti-word also narrows the gap between white and non-white children, and improves on KidWhisper by 29% in classroom settings, which are especially difficult for models (and humans) because of background talk and noise.
• pasketti-phonetic has a phone error rate (PER) of 33%, compared with 59% for PhoneticXeus, cutting errors by more than a third in every age group.
• Children ages 3–4 remain the hardest to transcribe, but they are also where some of the biggest gains came compared with existing models.

Word model: By setting

Word model: By race and corpus

Performance by age is shown at the top of this page. Full evaluation results for both models, including breakdowns by age, race, speech pathology, sex, and source corpus, are in the Pasketti model cards on Hugging Face.

As is, these models are usable, but should be tested and evaluated in the specific context and population they would be used in, given that performance will vary by age, dialect, setting, and task. They will be especially useful in contexts like literacy assessments, where a known target transcript is available to compare against, and can serve as strong starting points for fine-tuning to specific tasks, grade levels, or student populations. Where exact transcripts matter, outputs should be reviewed by a person.

For intended uses, out-of-scope uses, and known limitations, see the Pasketti model cards.

Data

The largest dataset of its kind for developing open ASR models for children's speech: 14 datasets identified, sourced, and augmented with new annotations to build the models above.

Data has been the biggest constraint to better ASR for children. Models that work well for adults learned from vast amounts of adult speech, and no comparable, representative dataset existed for kids. To assemble such a dataset, DrivenData reviewed the landscape of existing child-speech resources, then identified and sourced audio from public, research, and enterprise partners. Our transcription team added over 250 hours of new orthographic annotation and harmonized it with existing corpora into a common schema.

The result spans a wide range of children and settings: ages from preschoolers to young adults in clinical research data, typically developing kids alongside children with diagnosed or suspected speech and language differences, and dialects including African American English and General American English. The table below is a directory to each dataset — what it is, how much of it each model used, where to find the audio and the transcripts, and information on licensing outside the challenge.

Child speech data is sensitive and difficult to share. We are grateful for the work of the data providers who thoughtfully collected and provided the data used in this project under appropriate consents, and to the speakers represented whose voices have helped to advance this work.

On Top of Pasketti Competition

An open online competition catalyzed development against the challenge data and elevated the best-performing approaches.

Children's speech recognition is a hard problem where the best approaches are not evident at the outset. An open challenge brings a large expert community to the problem, tests hundreds of models quickly and cost-effectively, and lets the best-performing solutions rise to the top of the leaderboard.

The On Top of Pasketti: Children's Speech Recognition Challenge featured two tracks: the Word Track, scored on word error rate, and the Phonetic Track, scored on IPA character error rate. There was also a bonus prize for solutions that performed best on noisy classroom audio. The challenge was set up to keep evaluation data private and test solutions by bringing the models to the data.

Results

The challenge website drew more than 6,500 visitors from 179 countries. Over the course of the challenge, 828 participants submitted over 2,000 solutions for evaluation. Within the first weeks, the best submissions had passed the benchmarks, and they kept improving until the final days.

Word Track: 1,542 submissions evaluated

Phonetic Track: 684 submissions evaluated

Final Word Track winners improved more than 50% over both OpenAI Whisper and KidWhisper. Phonetic Track winners improved 49% over a baseline that transcribes words with Parakeet and looks up their pronunciations in the CMU Pronouncing Dictionary. On noisy classroom audio, the Word Track bonus winner improved 39% over KidWhisper.

$120,000 in prizes went to eight winners across the two tracks and the noisy classroom bonus:

Full write-ups of every winning approach are in the winners blog post, and all the code is in the winners' repository.

Resources

A consolidated list of shared models, data, and resources connected with this project.

Note on future opportunities

The open models, data, and resources from this project provide a foundation that also enables further development. Several opportunities identified from stakeholder interviews could make them easier to adopt, more trustworthy, and more useful in applied settings:

While none of these is needed to use the models today, each would build directly on this work to support better ASR tooling for early learners. If you'd like to explore these opportunities, discuss other ideas, or share how you're using these models and data in your work, we'd love to hear from you.