Ajay Agrawal is a professor at the University of Toronto’s Rotman School of Management, Research Associate at the National Bureau of Economic Research in Cambridge, MA, and Faculty Affiliate at the Vector Institute for Artificial Intelligence in Toronto. At Rotman, he founded the Creative Destruction Lab, a global startup program that has helped participating seed-stage, science-based companies generate over CA$28 billion in equity value. He is also co-founder of entrepreneur incubator NEXT Canada, autonomous robot developer Kindred, and Sanctuary AI, which is focused on giving robots safe, human-like capabilities.

It’s become almost a cliché to say that artificial intelligence will revolutionize society.

But from an economic perspective, artificial intelligence (AI) is no different from previous disruptive technologies in one key respect, attendees at CFA Society Toronto’s 2023 Annual Investment Dinner were told.

“Strip away all the hype, and you’re left with the question, ‘What does this reduce the cost of?’” asked this year’s keynote speaker, Ajay Agrawal, Founder of Creative Destruction Lab and Professor at the University of Toronto’s Rotman School of Management.

In a fireside chat with CFA Society Toronto Board member Heather Cooke, CFA, Agrawal noted that semiconductors lowered the time (and cost) of arithmetic, with profound implications for finance and science. The Internet lowered the cost of distributing digital goods and services. AI reduces the cost of high-fidelity (hi-fi) prediction.

The investment community is most focused on AI’s ability to use vast amounts of information to make reliable predictions more quickly and cheaply than humans can. Other applications also make use of this more efficient (and cheaper) hi-fi prediction, including:

• Medical research and diagnostics
• Creative content
• Best-next-response guidance in call centre or robo-advisor applications

Agrawal, co-author of Prediction Machines: The Simple Economics of Artificial Intelligence, told an audience of over 700 attendees to replace the phrase, “artificial intelligence,” with “cheap prediction machine.” That reduces the glare of the AI hype and allows standard economic analysis to take over. How will the drop in the cost of hi-fi prediction affect the cost of substitutes and complements? If the price of butter falls, so does the price of margarine, a substitute. If the price of golf clubs rises, so does the price of complements like golf balls Investors should seek to identify what kinds of goods and services may behave like “margarine” or “golf balls” when it comes to hi-fi prediction.

The focus so far has been substitutes. Banks are replacing previous fraud prediction analytics tools with AI. Investment managers are using ChatGPT-like large language models (LLMs) to enhance or even replace human analysis of large data sets.

To self-driving cars—and beyond!

Fraud prediction and data analysis are narrow, traditional predictive applications. AI is also applied in functions not traditionally viewed as predictive, like driving a car. AI observes and learns to predict how human drivers respond to a wide range of incoming data so that it can eventually make that decision as quickly as—and likely better than—a human.

The ability of LLMs to incorporate both structured—or numeric—and unstructured data in predictions is central to AI’s economic potential. “It allows us to automate all the things that previously moved slowly because they required language and people,” Agrawal said.

Most people so far have thought about AI in terms of “point solutions,” narrow applications in a broader, unchanged system that just deliver small productivity lifts, he continued. Thinking outside that box has historically proven fundamental to genuinely systemic change.

For example, the first proposed electricity use in 19th-century factories was replacing gas lamps with electric ones, a “point solution.” However, using small electric motors to power machinery instead of cumbersome power shafts and belts unleashed systemic change that revolutionized factory production.

Similarly, putting navigational AI in the hands of taxi drivers is a point solution with little marginal benefit to drivers who already know the alternative routes and shortcuts. By creating an app that uses GPS to connect drivers and passengers (and by navigating a regulatory framework that wasn’t built for such a scenario), Uber used the same technology to create systemic change.

Industries primed for an AI revamp

When it comes to AI, industries and transactional environments where the pre-hi-fi prediction infrastructure is relatively high cost are most susceptible to radical change, Agrawal said. Airports, with their shops and restaurants, exemplify a low-fidelity (lo-fi) prediction phenomenon. Passengers spend a lot of time at airports because of the unpredictability of all the processes that lead to getting on the plane. He said there are many other businesses with baked-in processes for dealing with lo-fi prediction.

“Just put on your economist sunglasses to protect yourself from the glare of the hype and look for the choke points and the scarcity in the value chain,” advised Agrawal, who expressed surprise that the investment community appears to have been caught off guard by generative AI despite the years of work that had been going on in the field.

Regulation of AI is another area where the direction seems unclear. But, as AI is a technology with a probabilistic product, Agrawal says it makes sense to look at how regulators deal with other probabilistic industries—like pharmaceuticals, for example. There, even after rafts of tests on mice, monkeys, and people, it’s not possible to predict who will have a bad response to a drug. Regulators can only assess the benefits and costs and the distribution of expected outcomes. They could approach AI the same way, Agrawal said.

“They’ll have to be comfortable saying, ‘Okay, we don’t understand how this 80-billion-parameter large language model works,’ but still go ahead and approve it if testing indicates that the benefits outweigh the costs.”

Human judgment remains key

Agrawal took issue with the popular narrative about how AI will affect the role of humans.

Firstly, while LLMs represent a huge stride in the power of prediction, he said there’s been “zero progress”—contrary to what Hollywood would have us believe—on machines exhibiting judgment. They can’t decide how to value one outcome over another.

An LLM can write a poem or novel (or annual report), but it needs the human to know what content is desired and to make tweaks to the prompts fed into the tool to get the tone right, said Agrawal, and added that his kids recently used ChatGPT to make a Christmas video.

“AI may be devaluing prediction, but there’s still a significant premium on judgment,” Agrawal said.

Secondly, the notion that AI-based automation will hurt lower-skilled workers wrongly takes its cue from the computer revolution of the past 50 years. That phenomenon did widen the wage gap by boosting the productivity of high-skilled workers relative to low-skilled ones. But early studies in call centres and consulting indicate that AI helpers or co-pilots are helping low-skilled workers more than high-skilled ones and narrowing the pay gap.

Agrawal commented briefly on some risks flagged with AI. Widespread concerns about biased data hearken back to early missteps, like AI-based recruiting tools making sexist decisions and early chatbots being taught to make racist comments. Today, you can ask an LLM upfront whether, based on the given data, it would make a biased decision—it will answer factually, and you can correct for that. The greater risk may be that a human decision maker would deny their existing biases, he said.

When it comes to deep fakes, like Canada’s recent “Spamouflage” incident in which federal politicians received bot-produced disinformation and offensive comments about Prime Minister Justin Trudeau and Conservative Party Leader Pierre Poilievre on their social media feeds, Agrawal called it “an arms race.”

“Bad actors are creating [deep fakes], and others are finding ways to detect them,” he said. “It’s hard to tell whether there will be a long-term equilibrium there.”

The key is the “emergent” nature of the technology, he said. Once scaled up, LLMs have produced results that weren’t anticipated and that they weren’t trained for.

“It’s hard to have a strong view on some of these possible outcomes because the technology is so strong and because emergence holds such potential for surprise. Nobody really knows what it’s going to do. We don’t know the limits.”