Technology

A new common sense has emerged regarding the perils of predictive algorithms. As the groundbreaking work of scholars like Safiya Noble, Cathy O’Neil, Virginia Eubanks, and Ruha Benjamin has shown, big data tools—from crime predictors in policing to risk predictors in finance—increasingly govern our lives in ways unaccountable and often unknown to the public. They replicate bias, entrench inequalities, and distort institutional aims. They devalue much of what makes us human: our capacities to exercise discretion, act spontaneously, and reason in ways that can’t be quantified. And far from being objective or neutral, technical decisions made in system design embed the values, aims, and interests of mostly white, mostly male technologists working in mostly profit-driven enterprises. Simply put, these tools are dangerous; in O’Neil’s words, they are “weapons of math destruction.”

These arguments offer an essential corrective to the algorithmic solutionism peddled by Big Tech—the breathless enthusiasm that promises, in the words of Silicon Valley venture capitalist Marc Andreessen, to “make everything we care about better.” But they have also helped to reinforce a profound skepticism of this technology as such. Are the political implications of algorithmic tools really so different from those of our decision-making systems of yore? If human systems already entrench inequality, replicate bias, and lack democratic legitimacy, might data-based algorithms offer some promise in addition to peril? If so, how should we approach the collective challenge of building better institutions, both human and machine?