Student Events

Featuring Will Dobbie, professor of economics at Princeton University Human Decisions and Machine Predictions in Multistage Systems We study how algorithms can be used to improve and understand decisions in multistage systems, where cases pass through successive stages of human review before a final decision is made. We develop a framework for evaluating humans and algorithms in such multistage systems, allowing us to compare humans and algorithms stage by stage, recover the system-wide effect of deploying an algorithm, and identify how humans add value to decision-making process. We find three main results when applying our framework to administrative data from Michigan's child protective services (CPS) system, where human screeners determine if an allegation warrants an investigation before human investigators visit the family to decide whether to open a case. First, conventional single-stage comparisons show that human screeners significantly underperform an equally selective algorithm but human investigators perform roughly as well as the algorithm. Second, these conventional single-stage comparisons significantly overstate the gains of deploying algorithms throughout the system because of a double-counting problem, where a case left unaddressed at both stages is credited to the algorithm twice even though maltreatment can be prevented only once. Third, there is meaningful heterogeneity in human performance at the investigation stage, with the highest performing investigators gathering and acting on information collected during the investigation itself that is unavailable to the algorithm.

Helen Burkhardt Title: Abstract

Hugo L'Huilier, Columbia University

Vrinda Kapoor Title: Abstract

Matthew Notowidigdo Title Abstract:

John Horton Title Abstract:

Katarina Borovockova, of Federal Reserve Bank of Richmond Title Abstract: