How It's Made

7 steps to get started with large-scale labeling

Omar Alonso

Omar Alonso

Mar 12, 2021

How Instacart built a crowdsourced data labeling process (and how you can too!)

Organizations that develop technologies rooted in information retrieval, machine learning, recommender systems, and natural language processing depend on labels for modeling and experimentation. Humans provide these labels in the context of a specific task, and the data collected is used to construct training sets and evaluate the performance of different algorithms.

1. Assess the lay of the land

2. Identify your use cases

3. Understand your data

4. Design your Human Intelligent Task (HIT)

5. Determine your guidelines

6. Communicate your task

7. Maintain high quality

Ready for Takeoff!

Acknowledgments & Further Reading

Learn more about Design at Instacart on our Design blog.

You may also like...

Designing Digital Experiences That Augment the Analog World

How It's Made

Designing Digital Experiences That Augment the Analog World

Shoppers are the backbone of Instacart’s business. Every day, we’re energized by being able to help them serve customers more effectively and support their learning and development to build a long-term relationship. In their offline…...

Mar 16, 2021
Nailing the Handoff

How It's Made

Nailing the Handoff

Exploring Certified Delivery’s checkout and delivery flows While Instacart’s bread and butter has always been and will continue to be grocery, many grocers and specialty retailers have a wide variety of items in their catalogs…...

Nov 17, 2020
Announcing Coil 1.0

How It's Made

Announcing Coil 1.0

I’m very excited to announce the release of Coil 1.0. Coil is a Kotlin-first image loading library for Android built on top of Kotlin Coroutines. It simplifies loading images from the Internet (or any other…...

Oct 22, 2020