The Economics Team at Instacart

Tilman Drerup, Levi Boxell, and Robert Fletcher

Tech firms are increasingly choosing to hire graduates from PhD programs in economics [1, 2, 3]. In this blog post, we present the economics team at Instacart and our take on the role of an economist in tech. Our “Econ Team” is fairly unique — both in positioning as well as skill set — and presents an iteration of more classical roles for economists. At Instacart, economists work at the intersection of machine learning engineering and economics, blending the merits of both functions in a single hybrid role that delivers end-to-end solutions to some of our most challenging problems.

In the rest of this post, we discuss how we have structured and positioned the team as well the problems we work on. We also talk about the skills we look for in prospective members for the team. In future posts, we will dig deeper into a few specific projects that highlight how we operate.

Economists in Tech

The roles tech economists play vary substantially between and even within companies. While economists are often integrated into broader product teams, some companies assemble dedicated economics teams to work on specific areas or problem spaces. There is no single type of tech economist but rather a whole class of them, mirroring the versatility of the profession itself.

At Instacart, our Econ Team is situated in a somewhat different role than in most other tech firms. The Econ Team is a dedicated unit of primarily academically trained economists that sits within our broader machine learning organization. Members of the Econ Team operate as full-fledged machine learning engineers, focusing on problems at the intersection between economics and machine learning while also doing the leg work of productionizing their models. Instead of being assigned to a single class of problems or product area, we are a horizontal team and form tight and lasting partnerships with product partners across the company.

Below, we provide some of the rationale that led us down this path and some guidance for economists aspiring to a similar role in tech.

The Econ Team at Instacart

About three years ago, Instacart embarked on building its own Econ team with the mission to answer some of the challenging economic questions that our marketplace has to offer. While a lot of these problems require deep economic expertise, a large number of them also require the estimation and deployment of large-scale machine learning models and optimization algorithms.

Take as an example the first generation of Instacart’s optimized bidding feature [4], an algorithm in service of our advertising partners. Optimized bidding in ad auctions is a multi-faceted problem that our team is ideally positioned to tackle. It requires not only a deep understanding of the economic principles that underlie our ads marketplace, including concepts like ads incrementality, auction formats, and auction pressure but also the ability to develop and implement a solution within a high-volume and latency-sensitive environment. The solution we built in close collaboration with our Ads team offers flexible, effective bidding options and is constantly delivering strong results for our advertising partners.

As another example, we also deployed Instacart’s first contextual bandit algorithms for incentive targeting. This project leveraged our econ knowledge in estimating heterogeneous treatment effects and optimizing business metric tradeoffs. At the same time, it required traditional MLE skills to make our models, both batch inference and real-time inference models, accessible to upsell placement resolvers within a production environment. (Stay tuned for a future blog post on this topic!)

Instacart’s Econ team is well-positioned to tackle such problems for several reasons.

First and foremost, our team has focused on hiring economists with strong engineering backgrounds. The majority of the team’s members combine a PhD in economics with exceptional technical skills, often in the form of graduate education in computer science or research projects leveraging machine learning tools. By integrating the economist and the MLE in the same role, we remove a layer of friction in the development of end-to-end machine learning systems. As soon as we have identified the most promising paths forward, this enables us to get a 0–1 solution into production quickly and effectively, including the associated data pipelines, inference endpoints, and monitoring.

Second, economists are by training instilled with a business framework that is applicable in many contexts. We can partner with product teams and apply our framework to understand a given problem along with its potential solutions. This enables us to push forward on new initiatives in areas where a dedicated team of specialists is not yet warranted. On the other hand, for product surfaces like search, ranking, and ads where dedicated MLE teams do exist at Instacart, we take a more supportive role and provide input on specialized topics in these domains where our economic background may provide value.

Last of all, as a horizontal team, an entrepreneurial attitude of seeking out new projects and product partnerships is highly valuable. In many ways, this mirrors the skills obtained in Econ PhD programs where students are expected to seek out dissertation projects and manage these projects from start-to-finish. It can, however, take time to build trust and a collaborative relationship with new product teams, and so patience is also required.

While working at the intersection of economist and machine learning engineer is a unique aspect of our team, similar to other companies, we also work on conceptual questions at the heart of Instacart’s business that leverage our expertise in areas like causal inference. For example, economists on our team have used surrogate models to measure long-run outcomes, used natural experiments to inform experiment design (see a future post on treatments inspired by regression discontinuities), estimated the causal effects of non-randomized treatments (such as Instacart+ membership), and constructed important customer-level segments widely used as machine learning model inputs and for understanding heterogeneity in experimental results throughout the company.

Recommendations for Aspiring Tech Economists

We believe that an increasing number of economists will pursue either hybrid (like ours) or traditional MLE roles. Newer cohorts of economics PhDs are embracing technology and companies are taking notice. However, for aspiring economists seeking a similar role to the one we have outlined, it may not be immediately clear which skills to prioritize.

Graduate school can provide much of the toolkit needed to be successful in the type of role we have created. In fact, many of the frequently used tools often just go by a slightly different name in industry and academia (e.g., uplift modeling vs. heterogeneous treatment effect estimation). Recently, the situation has only gotten better with emerging PhD curricula that complement the standard economics syllabus with dedicated machine learning courses like Stanford’s “Machine Learning and Causal Inference” [5] as well as free online resources like “Causal Inference for the Brave and True” [6].

There are certain methods that we think an economist aiming to work in tech should be deeply familiar with. Causal inference and methods of experimentation in particular permeate almost all of our team’s projects. Acronyms like ATE, HTE, IPW, or OPE show up remarkably often in our day-to-day work. And while economists may in many tech firms not be expected to be experts in all subfields of machine learning, a basic understanding of common algorithms (e.g., XGBoost, contextual bandits) and principles (e.g., model evaluation, hyperparameter tuning) is a necessary condition for the role we sketched.

Economists also should familiarize themselves with the things that economists work on in tech companies. Topics like Ads, Incentives, Pricing, or Marketplace Design are all areas where economic insights provide unique value. This summary [7] by Susan Athey and Michael Luca is an excellent resource to learn more about the many varieties of exciting topics that economists get to work on. Company blogs, like the Instacart Tech Blog, are another one.

Proficiency in coding is indispensable for economists entering the tech industry. Python has emerged as the industry standard and is considered essential. It is also not enough to write functional code, since clean, understandable code–dare we say beautiful code–is imperative in such settings. Unit testing, IDEs, linting, and version control all enable efficient and collaborative coding in this environment.

Lastly, economists in our role cannot be limited to the jupyter notebook, but need to understand the full suite of tools that glue their modeling work into production systems. Tools such as Airflow, jinja, Docker, and protobufs enable us to productionize our models in a way that is accessible to product surfaces.

Glimpse at Our Work

We hope this post was helpful by providing an opinionated perspective onto the world of economists in tech. In future blog posts, we will provide a deeper look at some of the projects we are working on. Follow tech-at-instacart to be notified as they are published.

Stay tuned!

Instacart

Instacart is the leading grocery technology company in North America, partnering with more than 1,400 national, regional, and local retail banners to deliver from more than 80,000 stores across more than 14,000 cities in North America. To read more Instacart posts, you can browse the company blog or search by keyword using the search bar at the top of the page.