case studies

Building research infrastructure for conversational AI

DrivenData partnered with BetterUp Labs to design and build the data infrastructure, machine learning pipelines, and research tools behind the CANDOR Corpus—enabling cutting-edge insights into human conversation.

The organization

BetterUp Labs is the research and development arm of BetterUp, a platform that combines coaching, AI, and behavioral science to drive personal and professional growth. With expertise spanning business, academia, and behavioral science, the Labs team generates research-driven insights using the world’s largest real-time dataset on workplace coaching and performance. Its pioneering work shapes BetterUp’s coaching products and methodologies.

The challenge

To advance the science of coaching, BetterUp Labs set out to explore the deep dynamics of human conversation. This required designing and executing the collection and analysis of large-scale multimodal (video, audio, text) data from the ground up. The team needed robust technical infrastructure and advanced machine learning methods to transform this raw data into meaningful scientific insights.

The approach

BetterUp Labs partnered with DrivenData on a multi-year R&D initiative to make this vision a reality. Our work focused on three core areas:

  • Building the end-to-end data pipeline

    We engineered a robust system to handle the full lifecycle of more than 1,000 hours of newly collected multimodal conversational data. This pipeline supported raw data ingestion, validation, cross-modal synchronization, and complex feature extraction—creating the CANDOR Corpus, a novel and large-scale research asset.

  • Applying advanced AI/ML for multimodal analysis

    We developed and validated models to extract deep insights across modalities:

    • Visual: Gaze direction, smiles, head gestures
    • Acoustic: Vocal prosody and paralinguistic features
    • Textual and Semantic: Sentiment analysis, emotion recognition, semantic embeddings
    • Conversational Dynamics: Turn-taking behaviors and transcription enhancement

  • Delivering the research toolkit

    We created a comprehensive Python toolkit that gave researchers streamlined access to every layer of the data—from raw media and transcripts to structured multimodal features and metadata. The research toolkit enabled ML-powered feature generation and accelerated exploration of the CANDOR Corpus.

    Throughout the project, we collaborated closely with BetterUp Labs on experimental design and analytical approaches to ensure the tools directly supported their scientific goals.

Topics in the CANDOR corpus track major societal shifts in 2020, with COVID-19 rapidly emerging as a dominant subject alongside enduring themes like family, and spikes in discussion around the presidential election and policing reflecting national attention.

The results

  • The CANDOR corpus and pipeline

    We delivered a unique, 1,000+ hour dataset of real-world conversations between Americans, collected in 2020 and enriched with hundreds of behavioral measures. By automating validation and processing, the pipeline ensured consistent data quality and made it easier for researchers to build on the corpus. The dataset has since supported a range of downstream applications, including the interactive storytelling project Hello Stranger, bringing insights from real conversations to broader audiences.

  • The research toolkit and algorithms

    The toolkit provided seamless access to multimodal data and machine learning algorithms for deep feature extraction, enabling sophisticated analysis of conversational behavior.

  • Landmark scientific contributions

    Our technical work led directly to the CANDOR corpus and its publication in Science Advances, a leading, high-impact journal from the American Association for the Advancement of Science (AAAS). The resulting dataset and tools continue to drive ongoing research into the science of conversation and behavior.

  • Foundation for future innovation

    This collaboration established a lasting technical foundation—including data, tools, and validated methods—that continues to support BetterUp Labs' cutting-edge research and product innovation in conversational AI.

"DrivenData is consistently excellent—world-class caliber. It's so rare to find the mix of technical skill, critical thinking, business savvy, and just-plain-fun-to-work-with vibes that hallmark their entire team. Whenever we have a gnarly problem to solve at the intersection of AI, analytics, and real-world application, my first thought is, 'Call DrivenData!' True data science superheroes."

— Andrew Reece, Chief AI Scientist, BetterUp

Our real-world impact

All projects
Partners: Max Planck Institute for Evolutionary Anthropology, Arcus Foundation, WILDLABS

Automating wildlife identification for research and conservation

Detected wildlife in images and videos—automatically and at scale—by building the winning algorithm from a DrivenData competition into an open source python package and a web application running models in the cloud.

Partner: CodePath

Data engineering from the ground up

Built data infrastructure to ingest, clean, integrate, and organize data across CodePath, created interactive dashboards for accurate monitoring of program trends, and provided trusted data expertise to identify and hire talent to carry the work forward.

Partner: The National Center for State Courts

Building a private LLM sandbox for NCSC

We worked with the National Center for State Courts to build an LLM chat sandbox for private usage. This sandbox allows users to experiment with LLM tools in a way that is safe, secure, and cost-effective, with specific use cases and prompts relevant to their work.

Partners: The World Bank, The Conflict and Environment Observatory

Identifying crop types using satellite imagery in Yemen

Used satellite imagery to identify crop extent, crop types and climate risks to agriculture in Yemen, informing World Bank development programs in the country after years of civil war.

Partners: Private sector, social sector

Building applied solutions with LLMs

Built solutions using LLMs for multiple real-world applications, across tasks including semantic search, summarization, named entity recognition, and multimodal analysis. Work has spanned research on state-of-the-art models tuned for specific use cases to production ready retrieval-augmented AI applications.

Partners: Bureau of Ocean Energy Management, NOAA Fisheries, Wild Me

Protecting endangered beluga whales with computer vision

Designed and administered a computer vision challenge that produced state-of-the-art machine learning models to identify and match individual endangered beluga whales from photo surveys.

Partner: EverFree

A production application to support survivors of human trafficking

Built the Freedom Lifemap platform, a digital tool designed to support survivors of human trafficking on their journey toward reintegration and independence

Partner: ReadNet

Crowdsourcing solutions for AI assisted early literacy screening

Ran a machine learning challenge to develop automatic scoring methods for audio clips from literacy screener exercises. Automated scoring can help teachers quickly and reliably identify children in need of early literacy intervention.

Partner: Science for America

Making higher education data more accessible

Created an open source Python library and interactive data visualization platform for analyzing U.S. higher education data and illuminating trends and disparities in STEM education.

Partner: Candid

Linking nonprofit grants to organizations with machine learning

Built Orgmatch, a scalable and explainable entity resolution system to add value to information processed by a leading nonprofit data hub.

Partner: IDEO.org

Illuminating mobile money experiences in Tanzania

Analyzed millions of mobile money records to uncover patterns in behavior, and then combined these insights with human-centered design to shape new approaches to delivering mobile money to low-income populations in Tanzania.

Partners: Insecurity Insight, Physicians for Human Rights

Tracking attacks on health care in Ukraine

Built a real-time, interactive map to visualize attacks on the Ukrainian health care system since the Russian invasion began in February of 2022. The map will support partner efforts to provide aid, hold aggressors accountable in court, and increase public awareness.

Partner: Wellcome

Addressing algorithmic bias in medical research

Conducted a literature review to understand the current state of bias identification & mitigation in mental health research, and synthesized recommended best practices from the field of machine learning.

Partner: National Oceanic and Atmospheric Administration (NOAA)

Forecasting geomagnetic storms

Designed and ran a global challenge that produced open-source models now powering public, real-time predictions of geomagnetic storm activity to help mitigate space weather risks.

Partner: CABI Plantwise

Mining chat messages with plant doctors using language models

Automated recognition of agricultural entities (such as crops, pests, diseases, and chemicals) in WhatsApp and Telegram messages among plant doctors, enabling new ways to surface emerging trends and improve science-based guidance for smallholder farmers.

Partner: NASA

Monitoring water quality from satellite imagery

Created an open-source package to detect harmful algal blooms using machine learning and satellite imagery. Included running a machine-learning competition, conducting end user interviews, and engineering a robust, deployable pipeline.

Partners: Candid, Black Wealth Data Center

Facilitating LLM opportunity workshops

It can be difficult to tell how generative AI can produce the most impact for organizations seeking to incorporate it into their work. We've facilitated human-centered design workshops with our partners to surface how generative AI can accelerate progress toward their goals.

Partner: Data science company foundation

Matching students with schools where they are likely to succeed

Used machine learning to match students with higher education programs where they are more likely to get in and graduate based on their unique profile, with a focus on backgrounds traditionally less likely to attend college or apply to more competitive programs.

Partner: University of Maryland

Processing multimodal tutoring data

Built well-engineered data pipelines to extract machine learning features from audio, video and transcript data collected from online tutoring sessions, enabling a team at the University of Maryland to study how relationship-building affects student outcomes.

Partner: Gates Foundation

Advancing speech recognition for student voices

Ran a global machine learning competition to build open automatic speech recognition (ASR) systems that actually understand children. Current ASR technology was trained on adult speech and often fails on kids' voices — a critical barrier to effective early learning tools. Over 2,100 submissions were evaluated across two tracks, and winning models achieved more than 50% improvement over previous leading child ASR models.

Work with us to build a better world

Learn more about how our team is bringing the transformative power of data science and AI to organizations tackling the world's biggest challenges.

Contact us Learn more about us