The organization¶
The National Center for State Courts (NCSC) is a non-profit organization that provides research, training, and technical assistance to state courts. Through their work NCSC improves how courts operate and how they serve the public, bringing together best practices and technology to help courts operate more efficiently and effectively.
The challenge¶
NCSC wanted to build a private LLM sandbox that would enable state court staff to begin experimenting with LLM tools in a way that was provider agnostic and guaranteed ensured robust data privacy. In addition to controlling the data and models that are available, NCSC wanted to provide a library of specific, court-targeted use cases as starting points for chats. This custom interface would allow experimentation with the latest state-of-the-art LLMs in a way that surfaced the most relevant applications to their user base.
The approach¶
DrivenData developed and deployed a custom chat interface for NCSC. Users can create accounts, select from a list of state-of-the-art LLMs, and chat with the model. In addtion to normal chat flows, we also provided a document upload functionality that would enable users to chat with documents they uploaded for use cases like summarizing proceedings or finding specific information in a law or regulation.
The application was built using typescript, react, and next.js. The backend uses a range of Microsoft Azure services, including Azure OpenAI, Azure App Service, and Azure Cosmos DB. LLMs were provided via the the Azure OpenAI API, which provides enterprise-grade data privacy and compliance guarantees. The system allows simple monitoring of usage, costs, and user management to NCSC while keeping operating costs low. New models and prompts for the use-case library are easily added to expand what the sandbox can offer to users.
The results¶
NCSC makes the deployed production system available to users and uses it as part of their training and research. The system enables state court staff to experiment with LLMs in a way that is safe, secure, and cost-effective. Ultimately, this can help courts understand the capabilities of LLMs and how they can be used to improve their operations.
As NCSC put it:
DrivenData has been a great partner in our work. Our goal was to set up an AI sandbox. The DrivenData team helped us to define our specific needs, prioritize them, create tools to meet those needs, and set up data collection to evaluate the performance. Their communication was timely and professional across all areas. They completed the work on time and within budget. I would be delighted to work with them again.
Diane Robinson, Principal Court Research Associate, National Center for State Courts
Learn more about the National Center for State Courts and their work at ncsc.org.
