Voice is one of the most natural ways for kids to learn, explore, and show what they know. Yet today's automatic speech recognition (ASR) technology hardly understands them.
Models trained on adult speech don't work for kids because child speech is fundamentally different. Children are still learning to produce speech sounds and their motor skills are still developing. They have distinct vocal characteristics, different speech patterns, and make frequent speech shortcuts and pronunciation errors, all of which make understanding them difficult. Poor speech recognition is a significant barrier to educational tools that support early learning, such as screening for speech pathologies, supporting teacher development based on classroom interactions, and making tutoring more widely available.
Before designing the On Top of Pasketti: Children's Speech Recognition Challenge, we at DrivenData needed a baseline. How big is the gap between adult and child speech recognition, and where is it largest? In mid-2025, we benchmarked two open-source ASR models on four children's speech datasets.
In this post, we share what we found when we tested off-the-shelf ASR models on children's speech. Our goal was to measure the size of the gap, not to crown the best model.
Our work confirmed that Whisper performs significantly worse on child speech than on adult speech, based on results from four benchmark datasets. Accuracy declines most sharply for younger children and children who speak African American English. Our findings highlight the need for more diverse and representative training data to unblock the creation of ASR models better tailored to child speech.
In this post:
Looking for a model that works on children's speech? We've since released open-weight word and phonetic models specifically for kids. Check out the Pasketti collection on Hugging Face 🤗 and the project summary.
How we benchmarked¶
Models¶
We tested two open-source models:
- Whisper (
openai/whisper-medium) is a state-of-the-art ASR model released by OpenAI in 2022. - KidWhisper (
aadel4/kid-whisper-medium-en-myst_cslu) is a version of Whisper fine-tuned on children's speech. Its training data includes the My Science Tutor (MyST) dataset, which is also one of our benchmark datasets.
We ran both models out of the box using Hugging Face's transformers package, with no additional tuning.
Data¶
We evaluated on four datasets of children's speech. Together, these cover a wide range of kids and settings.
| Dataset | Description | Children | Utterances | Ages |
|---|---|---|---|---|
| JIBO Kids Corpus | Collected through a social robot. Includes letter and digit identification tasks and longer conversational tasks | 110 | 8,505 | Pre-K to grade 1 |
| Ellis Weismer Corpus | Lab-based adult-child interactions with late talkers and typically developing children | 131 | 50,524 | 2.5 to 5.5 years |
| Cameron | Storytelling task with speakers of multiple dialects including African American English | 40 | 11,364 | 3 to 5 years |
| My Science Tutor | Students in spoken dialog with a virtual science tutor | 1,371 | 13,180 | Grades 3 to 5 |
Metric¶
We measured performance with word error rate (WER), the most common metric for speech recognition. WER counts how many edits it would take to turn the model's prediction into the correct transcript:
$$\text{WER} = \frac{\text{substitutions} + \text{deletions} + \text{insertions}}{\text{number of words in the correct transcript}}$$
WER corresponds roughly to what percent of predicted words are incorrect. WER is an error metric, so lower is better.
Key findings¶
Models trained on adult speech don't translate well to child speech. The younger the child, the worse the performance. As of our benchmarking in mid-2025, state-of-the-art adult speech recognition models achieved WER of 6–7% on adult data, with real-world performance for off-the-shelf offerings being closer to 15–30%. We found that WER on child speech ranged from 13–17% for older kids in grades 3–5 to 70–76% for toddlers and preschoolers. That means that for young children, roughly three in four words are incorrect.
Models perform worse on children speaking African American English (AAE). KidWhisper's WER was 66%, 17 percentage points higher than for non-AAE peers (18 points higher for Whisper). By comparison, accented speech for adults is around 10–20% WER. Only the Cameron dataset included dialect for each child. There were 20 children who spoke AAE, and 20 who spoke non-AAE.
Fine-tuning requires large, representative data to make a big difference. KidWhisper is a version of Whisper fine-tuned on children's speech from the My Science Tutor (MyST) dataset. KidWhisper only slightly outperforms Whisper, with the most notable gains on the MyST dataset used in its fine-tuning. Fine-tuning on targeted data does improve performance on children, but the gain is limited by the size of the fine-tuning dataset and its diversity. Good real-world models need more training data that captures a broader diversity of child speech across ages, dialects, and speaking styles.
Performance was slightly worse on children with a speech delay compared to those with typical speech development. Gender showed no consistent relationship with WER at young ages.
Lessons for evaluating ASR models¶
If you want to run your own evaluation on children's speech, here are a few things we learned along the way.
Text normalization choice has a big impact on WER. Before calculating WER, we normalized both the correct transcripts and the predictions with Whisper's text normalizer (WhisperTokenizer.normalize). Normalization handles things like how numbers are written, non-word interjections like "um", and contractions (e.g., "there's" becomes "there is"). When we left punctuation in, KidWhisper's WER on the Ellis Weismer dataset jumped from 70% to 85%.
Watch out for models getting stuck in a loop. Occasionally a model gets stuck repeating itself and produces extremely long predictions. In one case, a child said "there", and Whisper predicted "and the water" repeated 160 times. That's a WER of roughly 48,000% for a single utterance. WER is calculated across the whole dataset. It is the total number of insertions, substitutions, and deletions over the total number of actual words. These extreme cases with a ton of insertions can dominate the overall WER.
To prevent these edge cases from dominating WER, we dropped the worst-performing 1% of utterances in each dataset. We recommend running models with loop protection rather than dropping utterances — that's what we did in our later work preparing to post our new model on Hugging Face.
The "correct" transcript isn't always correct. Children's speech is hard for human transcribers, too. Some of what shows up as model error is actually error in the ground truth labels.
What came next¶
These results shaped how we designed the On Top of Pasketti challenge. We went looking for data that would fill the gaps above, including the youngest children, more dialects, children with speech pathologies, and noisy classrooms. We ended up with around 520 hours of children's speech across 14 datasets, and the resulting models achieved roughly half the errors of leading open alternatives. You can see the results and learn how to access that data in the project summary, and read about how the winners did in the winners blog post.
After the competition, we retrained the winning approaches on broader data and released them as open-weight models:
- pasketti-word transcribes the words a child says
- pasketti-phonetic transcribes exactly how words were spoken and pronounced
Note that the results in this post and in the Pasketti model cards come from different test sets, so the numbers aren't directly comparable. The models above were rerun on the same evaluation set as the Pasketti models to enable direct comparison.
