How It's Made

7 steps to get started with large-scale labeling

Omar Alonso

Omar Alonso

Mar 12, 2021

How Instacart built a crowdsourced data labeling process (and how you can too!)

Organizations that develop technologies rooted in information retrieval, machine learning, recommender systems, and natural language processing depend on labels for modeling and experimentation. Humans provide these labels in the context of a specific task, and the data collected is used to construct training sets and evaluate the performance of different algorithms.

1. Assess the lay of the land

2. Identify your use cases

3. Understand your data

4. Design your Human Intelligent Task (HIT)

5. Determine your guidelines

6. Communicate your task

7. Maintain high quality

Ready for Takeoff!

Acknowledgments & Further Reading

Learn more about Design at Instacart on our Design blog.

Omar Alonso

Omar Alonso

Omar Alonso is a member of the Instacart team. To read more of Omar Alonso's posts, you can browse the company blog or search by keyword using the search bar at the top of the page.

Most Recent in How It's Made

How Instacart Uses Embeddings to Improve Search Relevance

How It's Made

How Instacart Uses Embeddings to Improve Search Relevance

Authors: Xiao Xiao, Taesik Na, Tejaswi Tenneti, Guanghua Shu As a fast-growing e-commerce platform and the leading grocery technology company, Instacart has an ever-expanding catalog of over 1 billion products from 900+ retailers, spanning a…...

Sep 14, 2022
Lessons Learned: The Journey to Real-Time Machine Learning at Instacart

How It's Made

Lessons Learned: The Journey to Real-Time Machine Learning at Instacart

Authors: Guanghua Shu and Sahil Khanna Figure 1: How ML models support shopping journey at Instacart Instacart incorporates machine learning extensively to improve the quality of experience for all actors in our “four-sided marketplace” — customers who place orders on…...

Sep 7, 2022
How Instacart Uses Machine Learning-Driven Autocomplete to Help People Fill Their Carts

How It's Made

How Instacart Uses Machine Learning-Driven Autocomplete to Help People Fill Their Carts

Authors: Esther Vasiete Tejaswi Tenneti For an eCommerce platform like Instacart, Search is an important tool for making shopping feel effortless by helping people find the products they want, even when they are not sure what they’re…...

Aug 30, 2022