Money Research Collective’s editorial team solely created this content. Opinions are their own, but compensation and in-depth research determine where and how companies may appear. Many featured companies advertise with us. How we make money.
AI Tools Are Getting Better, but They Still Struggle With Money Advice
By Pete Grieve MONEY RESEARCH COLLECTIVE
In one website’s test, ChatGPT Search only correctly answered financial questions 65% of the time.
As ChatGPT and its competitors improve their models, a growing number of people are relying on AI tools to answer everyday financial questions.
But is that a good idea?
When mainstream AI tools took the world by storm two years ago, experts cautioned that ChatGPT and other services were not capable of providing trustworthy financial advice, citing the risk of hallucinations and their limited access to real-time information. And despite improvements to AI chatbots in the time since, recent research indicates artificial intelligence tools still are not advanced enough to reliably answer many personal finance questions.
The team at Investing in the Web, a site that reviews financial services, conducted a test in November that looked at how well ChatGPT Search answered 100 questions about money. According to their scoring of the responses, ChatGPT was correct 65% of the time, “incomplete and/or misleading” 29% of the time and wrong 6% of the time.
The questions included:
- How much should I save for my retirement? (ChatGPT got this “correct,” providing key factors that determine how much an individual should save and explaining several savings frameworks that have been endorsed by financial experts.)
- What is the retirement age in my country? (ChatGPT gave an incomplete answer, failing to mention that the rules were changing in the user’s country of Sweden.)
- How to decide when to buy or sell an investment? (The response was graded “incorrect” because ChatGPT did not discourage the user from “timing the market,” arguably the most important point to make here.)
ChatGPT performed worse on questions involving shifting news and information, while performing better in areas of evergreen financial advice. Taxes and financial aid were particularly challenging subjects for ChatGPT.
In a report, Investing in the Web noted that ChatGPT appeared to source some of its answers from less-reliable personal blogs, which introduced errors into its outputs. Even though researchers found that AI wasn’t dependable for over a third of financial questions, the report concluded that AI may be used as “a good starting point when researching a topic.”
Financial advice, math and AI
Some chatbots appear to have guardrails limiting responses to sensitive questions, including those about money.
For example, Google’s Gemini responds to a request for stock picks by telling the user to consult a financial professional, saying that it is unable to recommend investments. Similarly, if you ask medical questions, you’ll be told to go to a doctor. Political prompts yield a message about restrictions relating to elections.
But ChatGPT — owned by OpenAI — will produce responses to many financial questions, and surveys have shown that a significant swath of Americans are turning to AI to manage their personal finances. This is potentially concerning, in part because AI tools often struggle with basic math.
In forums, internet users who’ve experimented with plugging financial questions into ChatGPT have pointed out that because ChatGPT is a large language model (LLM), asking it to deal with numbers and solve math problems often leads to problems.
A team of Apple researchers dug into this weakness of chatbots in an October paper that concluded “current LLMs are not capable of genuine logical reasoning; instead, they attempt to replicate the reasoning steps observed in their training data.”
Although they have many other uses, LLMs are not necessarily reliable or advanced when it comes to actually doing math beyond the grade-school level. Instead, their attempts to solve math problems “resemble sophisticated pattern matching more than true logical reasoning,” according to Apple.
The bottom line: AI tools may be able to answer some of your financial questions, but you should always tread carefully when deciding what to do with your hard-earned money or how to plan for your future. No matter where the advice you’re getting comes from — even if it’s a human — make sure to double check the main points with other sources.
It's never too late to start investing
Earn an uncapped 1% match when you transfer or rollover a brokerage account, IRA or 401(k)*
- Invest in stocks, bonds, options, crypto and more
- Earn rebates on your options trades, including with our free active trading API
- Turn any idea into an investable index. Just enter a prompt, and put our AI to work
- Invest for retirement with a matching 1% bonus on your annual IRA contributions
- Earn 3.3% APY** on your cash with no fees.
Easy and automated investing, starting at $3 a month
-
Online banking that automatically saves and invests with no hidden fees
-
Earn bonus investments from thousands of top brands
- Expert-led, bite-sized courses to increase your financial knowledge
Get Up To $1,000 When You Open and Fund a New Brokerage Account With a Qualifying Deposit*
- No account minimums or maintenance fees*
- Invest in stocks, etfs, bonds, options and more
- Get investing insights from Morgan Stanley
- Available desktop trading platforms with customization options
Get up to $1,000 in stock when you open & fund a new Active Invest account1
- Buy & sell stocks with no commissions*
- Buy fractional shares starting at $5*
- Access to alternative investments, including commodities
A library of expert stock recommendations
- Two expert-selected stock picks each month
- Provides ETF allocation insights to balance your portfolio
- Access monthly rankings, 5‑year quant scores, detailed stock reports, and strategic “entry style” guidance
More from Money:
5 Best Credit Repair Companies of February 2025
Small Change: What Happened When Other Countries Got Rid of Their Pennies
I Let AI and My 5-Year-Old Pick My Stocks. Who Did Better?
Pete Grieve is a New York-based reporter who covers personal finance news. At Money, Pete covers trending stories that affect Americans’ wallets on topics including car buying, insurance, housing, credit cards, retirement and taxes. He studied political science and photography at the University of Chicago, where he was editor-in-chief of The Chicago Maroon. Pete began his career as a professional journalist in 2019. Prior to joining Money, he was a health reporter for Spectrum News in Ohio, where he wrote digital stories and appeared on TV to provide coverage to a statewide audience. He has also written for the San Francisco Chronicle, the Chicago Sun-Times and CNN Politics. Pete received extensive journalism training through Report for America, a nonprofit organization that places reporters in newsrooms to cover underreported issues and communities, and he attended the annual Investigative Reporters and Editors conference in 2021. Pete has discussed his reporting in interviews with outlets including the Columbia Journalism Review and WBEZ (Chicago's NPR station). He’s been a panelist at the Chicago Headline Club’s FOIA Fest and he received the Institute on Political Journalism’s $2,500 Award for Excellence in Collegiate Reporting in 2017. An essay he wrote for Grey City magazine was published in a 2020 book, Remembering J. Z. Smith: A Career and its Consequence.