Not Smart Enough To Make Employees Smarter

What does Goldman Sachs' "bubble bursting" Generative AI report mean for education?

Late in June, Goldman Sachs made public a redacted version of a report created by its Global Macro Research division. Titled “Gen AI: Too Much Spend, Too Little Benefit?” the report directly and repeatedly asks the pressing question of whether Generative AI is fueling a speculative bubble. As education is among the sectors where speculative investment (and with it aggressive marketing and publicity) has been targeted since at least late 2022, I read this report with its implications for education in mind, which I’ll share momentarily.

But a couple quick disclaimers are important. Goldman Sachs is, of course, not a disinterested observer. The investment bank’s decision to publish and promote this report, generating headlines aplenty, leads me to cynically presume the bank has some net short positions which might benefit from, for instance, sharply-declining AI ETF prices, as has been the case recently.

Moreover, the narrative promoted on the title page of the report, and reproduced in most of the reporting about it, is more bearish than its overall contents. Even the most cynical expert consulted in the report explicitly advocates for continuing to invest in some of the companies which have benefited most from the recent boom, like NVIDIA and Alphabet.

The report is, however, worth engaging on two levels. First, it includes lots of proprietary data analysis which Goldman doesn’t frequently share with the general public. And, second, there is a rich terrain of subtexts, ideological presumptions, and unexamined biases for a critical finance scholar who is certainly not among the readers imagined by those who prepared the report.

Most surprising (and mildly encouraging) to me was the virtually universal conclusion that the short-term automation of labor with AI is a dead end. Not only are examples of effective automation hard to come by, those few that can be found are exorbitantly expensive. Jim Covello, Goldman’s Head of Global Equity Research, says, for example, “We’ve found that AI can update historical data in our company models more quickly than doing so manually, but at six times the cost.”

There will be some rare instances where employers will be willing to spend exponentially more for marginal improvements to efficiency, but most simply won’t. In the short-term at least, even entry-level coders and call center workers, whose jobs Goldman thinks might be effectively automated by Generative AI applications (based on limited analysis), remain safe because, in aggregate, they need lower wages and less water.

Even the experts who Goldman presents as bullish on AI over the long run don’t regard labor automation as part of their ideal scenario, because the massive investments required to make it efficient would need to be justified by a durable competitive advantage, but the conventional wisdom about labor innovation is that it’s easy to adopt and imitate.

Goldman’s assessment does not bode well for the EdTech entrepreneurial fantasy (and its inverse, the professorial obsolescence paranoia) which has been circulating furiously this past year. Education workers are perennially underpaid to do often idiosyncratic jobs requiring dynamic combinations of intellectual, emotional, and physical labor, the quality of which is being evaluated in real time by a population who is neither silent nor patient when it is not done to their satisfaction. That many of our faculty are not only markedly better educators than any Generative AI application on the horizon, but can work longer hours for lower wages without overheating, is maybe not exactly a flex, but it is a bad omen for the hundreds of ventures marketed on that premise.

As yet, Generative AI does not do anything well enough to scale. Educators who choose to read the whole document will get a few lolz from Goldman bankers describing what we already know about the AI applications students are experimenting with. “People generally substantially overestimate what the technology is capable of today,” Covello says, “In our experience, even basic summarization tasks often yield illegible and nonsensical results.” Later, he says, AI “is much more likely to enable employees to find information faster than enable them to find better information.”

Those pesky things we traditionally associate with education, nobody thinks Generative AI is good at any of them yet, and many doubt that it ever will be. “Including twice as much data from Reddit into the next version of GPT may improve its ability to predict the next word in an informal conversation,” Daron Acemoglu observes, but “the quality of the data matters, and it’s not clear where more high-quality data will come from and whether it will be easily and cheaply available to AI models.” Covello says, “I don’t think the technology is, or will likely be, smart enough to make employees smarter.”

Even Goldman’s AI bulls, Kash Rangan and Eric Sheridan, admit that there is no “killer application” on the immediate horizon that will justify the existing trillion-dollar investment in Generative AI. They espouse an “infrastructure first, platforms next, and applications last” philosophy, confident that the scalable consumer product will come in due time, and claim that this is a common pattern with tech enterprises. Covello disagrees. He claims that the marketable applications for personal computers, the internet, and smartphones were understood early on, and could motivate the investment and engineering through growing pains. “The roadmap on what other technologies would eventually be able to do existed at their inception. No comparable roadmap exists today,” he claims, “Eighteen months after the introduction of generative AI to the world, not one truly transformative - let along cost-effective - application has been found.”

Consumers have already taken notice. Adoption of AI products has slowed, even in the sectors, like education, with the largest marketing spends.

The problem Goldman diagnoses, across the entirety of the report, is one of triaging technological development, spiraling costs, and the time horizon of investment. Acemoglu puts the best case scenario for Generative AI to become “truly transformative” (a phrase he also uses) at ten years. Covello acknowledges that as long as corporations, especially hyperscalers like Alphabet and Amazon, remain profitable under economic conditions which have recently been very agreeable for them, they will continue to throw billions after the alluring story of a techno-utopian future built on Generative AI. But the appeal will gradually fade with time, and will evaporate the instant economic conditions turn even temporarily against them.

Even a minor economic downturn will be a dot-con-level bloodbath for AI start-ups. Nobody denies how exceptionally expensive Generative AI currently is, and the Goldman report outlines in grotesque detail how much energy it requires and how much infrastructure will have to be built in order to accommodate that energy, even if AI products become dramatically more efficient, as well as how hard it will continue be to scale up semiconductor production.

Importantly, the Goldman report does not endorse a single AI start-up or application. The short-run endorsements it does make are for the chip manufacturers, cloud storage hyperscalers, and private utilities, all of whom don’t depend exclusively on AI for their long-term business models.

Assuming investors are willing at this juncture to follow AI’s evangelists, the size and time horizon of the investing required will incline them to cut and run if and when problems emerge. And that’s exactly what Goldman Sachs is advising its clients to be prepared to do in the final pages of its report.

For the schools and instructors who choose to integrate AI tools, the same is true. They should be prepared for the possibility those tools cease to function almost overnight. There is no backstop. The moment the market gets scared, 60% of AI start-ups may disappear.

We won’t have to wait a decade. Even one of the bulls, Sheridan, says he expects investors will get spooked “if scaled consumer applications don’t emerge over the next 6-18 months.”

Literally nothing in the Goldman report would suggest there’s even the slightest chance that’s enough time.