Can AI Beat the Market? Wall Street Is Desperate to Try

Voleon Group boasts some of the best minds in data science. The hedge fund’s co-founder and chief executive officer is a former nuclear physicist and web pioneer. Its head office is in Berkeley, California, strategically placed near a world-class machine-learning center where it has access to the latest advances in artificial intelligence.

The firm, with some $5 billion in assets under management, is one of a growing number of funds dedicated to creating the ultimate money machine—AI that can teach itself to beat the market. With the technology threatening to disrupt Hollywood, health care and countless other sectors, sparking this year’s defiant stock rally in the process, the fund’s boss says it’s inevitable someone will reach that goal. “In one domain after another, this approach just proves to be more fruitful,” says Michael Kharitonov, a veteran of the CERN nuclear research lab in Geneva. “Finance has its own unique challenges, but over time they can be overcome.”

So far that hasn’t happened. The irony of investors’ piling into AI is that the technology has for years struggled to crack the actual business of investing. Machines get bamboozled by noisy markets and can be caught off guard by fickle trends, and finance—surprisingly—sometimes lacks the oceans of data that underpin the technology in other domains.

A Eurekahedge index of 12 funds using AI has trailed its broader hedge fund index by about 14 percentage points over the past five years. According to Plexus Investments, an asset manager that tracks the returns of boutique AI funds, only 45% outperform the benchmarks they measure themselves against.

While that may seem disappointing to anyone watching AI upend other fields, it’s comparable to the average performance of human stockpickers. “Today, AI and machine learning can already compete with traditional fund managers,” says Andreas Vogel, a senior analyst at Plexus. In other words, you don’t need to hire expensive portfolio managers when a computer can give you similar results.

AI advocates say they’re not striving for market-thumping returns, just a slight edge—which on Wall Street can mint billions. Call it Moneyball for the markets, where they’re looking to consistently get on base, not swing for the fences. “In finance you can be very successful by just being a little bit better than 50%,” Kharitonov says.

Hopes largely rest on machine learning, the subfield of AI where computers are trained on massive amounts of data to perform particular tasks. Generative AI—the power behind ChatGPT—is one strand of the discipline. Practitioners use machine learning to do everything from perusing social media to gauge sentiment around a stock to monitoring market patterns as a means of deciding when best to execute a trade.

The promise has made Jason Hsu a convert. Hsu was an old-school quant, shorthand for “quantitative investor,” money managers who use computers to crunch piles of numbers and then pick securities. Like most quants, he believed in a handful of simple investing rules developed over decades by academics studying the behavior of markets and picking stocks based on characteristics such as their valuation or size. In 2002, Hsu co-founded Research Affiliates, which now manages some $130 billion in assets.

With the backing of the firm, Hsu founded a company in 2016 called Rayliant Global Advisors, and soon he underwent an AI epiphany. His team showed him the hypothetical results of an investing strategy designed by machines, and now most of the firm’s funds are run by an algorithm called eXtreme Gradient Boosting. The manager of about $17 billion parses some 200 signals, using a form of decision tree to identify often complex relationships and make buy or sell decisions. “It took us a while to convince ourselves,” Hsu says. “Then we finally got to a place where we said, ‘We see the value. We get the benefits.’”

By Silicon Valley standards, Rayliant’s methods are almost old-fashioned. Yet they’re a big leap for a money manager who started out picking stocks based on just six criteria, and a good illustration of the way AI promises to revolutionize investing: not loud or dramatic, just a series of tweaks, techniques and enhancements bearing impenetrable names like “support vector machines” or “long short-term memory.”

Many of the applications of AI in investing look like this—taking traditional quant thinking and supercharging it. So where an old approach might’ve used an algorithm that says to “buy stocks with the lowest price-to-book ratio in the market,” AI could figure out that doing so works only in certain industries and when earnings growth is also positive.

That’s a huge oversimplification, and there are big hurdles, not least that market trends and investor behavior might hold for months or even years but do a U-turn in an instant, making whatever the machine has learned suddenly irrelevant. That’s why, when the pandemic struck out of the blue, Voleon was among the many funds that faltered. Rayliant was just deploying its new AI strategies when the old value stocks it used to favor surged post-Covid. “Out of the gate it was horrible,” Hsu says.

Finance doesn’t always have enough data on hand to make effective use of AI, particularly for firms such as Rayliant that have a longer-term horizon for their investments. Traditional quant strategies often track a stock’s price on a monthly or even quarterly basis to eliminate the noise seen in daily or minute-by-minute data sets. But that means they’ll have fewer than 2,000 data points even for stocks in companies that have been around for a century, which will limit how AI can be applied.

While almost all quants experiment with machine learning, Voleon is one of just a handful of firms using a cutting-edge technique called deep learning, which mimics how the human brain works, creating networks with countless connections that can spot complex but subtle patterns in massive data sets. It’s how ChatGPT figured out how to read, Siri learned to listen and cars are teaching themselves to drive. “Markets can be completely random occasionally, and during those times nothing will work—not AI, not machine learning,” says Renee Yao, a former trader at hedge fund Millennium Management who founded Neo Ivy Capital, a New York firm that uses deep learning. “However, during the time when it’s not completely random, I think AI works better” than other investing styles.

Voleon’s longest-running fund has averaged an annual return of about 9.5% since inception, says a person familiar with the matter, who declined to be identified as the information is private. Neo Ivy, which oversees $200 million, has posted about 7% annually, the person says.

Rather than leaving machines to simply adapt to whatever they learn, most money managers using AI try to combine new techniques with established theory. AQR Capital Management says it’s increasingly using AI to find signals from text and sharpen its long-established strategies. Robeco has added machine learning to funds where it can and has introduced a suite of “next-gen” strategies for where it can’t. Vanguard this year added AI to its quant stock strategies to make them more adaptive to markets.

At Man Group, the world’s largest listed hedge fund, the adoption of machine learning has been bumpy. The London-based firm hired a Ph.D. for that purpose in 2009, but the technology didn’t make it into client portfolios until 2014. Since then, Man has slowly broadened its use, with AI now figuring out where to route buy and sell orders, turning text into trading signals, writing code and summarizing economic insights.

Machine learning can find patterns based on economic models, says Stefan Zohren, principal quant in Man’s trading division. “But it can also find many other ones that are potentially not so intuitive,” he says, “which is an advantage because obviously fewer people might have found them,” though it won’t always be clear how reliable the new patterns are.

That hints at one of the last and biggest hurdles to AI adoption: explainability. It turns out human investors generally like to know what’s happening with their money. If an AI strategy underperforms and the fund manager can’t explain why—because the machine’s thinking is unknown—it doesn’t go down well. “It’s natural,” Kharitonov says. “Nobody’s asking ChatGPT to explain why it uses certain words.”