Open source projects

Since starting DrivenData, we’ve seen a lot of data science in the wild. As the field develops, it’s becoming increasingly important to organize data science work so that it’s easy to reproduce and build upon. Cookiecutter Data Science is a widely used project template that keeps data scientists organized and on track.

Have you wished for a consistent and easy interface in Python to access files in cloud storage like S3 and Azure? cloudpathlib is an extensible Python library that provides pathlib.Path-style classes for dealing with files in various cloud storage services, with seamless local caching.

Looking for an easy, clean way to visualize your data model? erdantic is a simple tool for drawing entity relationship diagrams (ERDs) that show how data model classes are connected. Generate ERDs from models defined with multiple supported frameworks, such as Pydantic and dataclasses.

nbautoexport automatically exports Jupyter notebooks to various file formats (.py, .html, and more) upon save while using Jupyter. One great use case is to automatically have script versions of your notebooks to facilitate code review commenting.

Love pathlib.Path? Love pandas? pandas_path makes it easy to use pathlib methods on pandas Series. Just one import adds a .path accessor to any pandas Series or Index so that you can use all of the methods on a Path object.

When there's a lot at stake, checklists make sure big questions don't slip through the cracks and tough conversations happen even (especially) in fast-moving environments. The goal of deon is to push that conversation forward and provide concrete, actionable reminders to the developers that have influence over how data science gets done.

Zamba is an open-source Python package that uses machine learning and computer vision to help automate time-intensive image and video processing tasks for wildlife monitoring. Zamba includes multiple state-of-the-art, pretrained machine learning models for species and blank detection in different geographies. It can also be used to train custom models on new species and geographies based on user-provided labeled data.

CyFi is a command line tool that uses satellite imagery and machine learning to detect dangerous concentrations of cyanobacteria in small, inland water bodies. The goal of CyFi is to help water quality managers better allocate resources for in situ sampling and make more informed decisions around public health warnings for critical resources like lakes and reservoirs.

The Integrated Postsecondary Education Data System (IPEDS) offers a wealth of comprehensive data on U.S. higher education institutions, but this data is spread across multiple files and formats, making longitudinal analyses and cross-institution comparisons challenging. scipeds is an open-source Python library that simplifies the analysis of IPEDS data.

DrivenData runs machine learning competitions to help non-profits, NGOs, governments, and other social impact organizations use data science in service of humanity. To enable data scientists and mission-driven organizations to learn from the work done in these competitions, we open source the code submitted by winners for others to learn from, use, and adapt.