Large language models are no longer just generating text. They’re intelligent agents capable of reasoning, and with those advanced capabilities, their value in financial workflows rises.
IT leaders now have a wide range of LLM model architectures and prompt strategies to choose from, each designed to support data interpretation, document analysis and decision-making. From general-purpose models parsing earnings calls to agentic systems managing multistep audits, here is an overview of how these artificial intelligence tools are reshaping finance.
RELATED: How tokenization, masking and data quality empower LLMs.
General-Purpose LLMs: All-Rounders for Financial Text Tasks
General-purpose models such as GPT-4, Anthropic’s Claude and Google Gemini are often embedded in finance-related operations where broad language understanding and contextual summarization are essential.
“These models are extremely valuable for knowledge base types of question-and-answer applications and for summarizing text,” says Michele Rosen, IDC research manager for open generative AI and LLMs.
These general models are ideal for departments such as marketing, HR or internal IT support. Rosen adds that such models are also useful for summarizing regulatory filings and conducting sentiment analysis of earnings calls or news articles.
“Right now, there is no algorithmic way to determine what that outcome is going to be,” she says. “You might choose a general-purpose model that meets 80% of your needs and decide those are the types of projects you’re going to go after.”
But that flexibility can have its trade-offs. “You’re never going to get the level of accuracy or understanding from those kinds of models that you would from a model that has either been post-trained on financial services information or trained with enhanced financial content,” Rosen says.
Click the banner below to harness the power of generative AI in your organization.
Domain-Specific LLMs: Precision for Financial Complexity
Purpose-built LLMs such as BloombergGPT, FinGPT and FinanceBench are particularly popular for their specialized training in finance.
“The more knowledge that the model has about that particular domain, the more likely it is that it will produce the kinds of results you’re looking for,” Rosen says. “That, I think, is the real differentiator.”
These models handle tough, industry-specific work such as reading financial statements, analyzing earnings calls and making sense of dense regulatory documents.
Shrinath Sanjay Thube, a software developer at IBM, agrees that domain-specific models “outperform general-purpose models on niche, complex tasks, such as extracting insights from Form 10-K financial statements or interpreting highly specialized industry jargon.”
That said, these models are not magic. Sometimes they lack updates and can hallucinate. For that reason, Rosen recommends that IT leaders allow for post-training or fine-tuning on proprietary data sets.
“There’s a strong desire to customize when projects are adopted,” Rosen says.
RAG Models: Real-Time Access to Internal Knowledge
Retrieval-augmented generation can help financial teams ground their AI-generated responses in verifiable, up-to-date documents. RAG models blend generative reasoning with dynamic information retrieval from internal databases, filings or feeds.
“RAG is a powerful way to keep answers factual, up to date and grounded in your own source material,” Thube says. “You can use a foundation model as is, and it will retrieve and reference the latest documents in real time.”
READ MORE: Want to learn how your peers are using AI? Read the CDW AI Research Report.
This setup is particularly helpful for compliance and investor relations. “Finance teams can confidently answer investor or compliance queries, knowing every response is based on up-to-date, secure and auditable information,” he says.
There’s also growing traction for RAG in enterprise settings. “Rather than just relying on the model’s ability to predict the best response, you direct it to rely on a compacted database of content that you have selected,” Rosen says. “If someone is asking about a policy, the model doesn’t guess, it looks in the RAG database for the official response.”
RAG systems often work in tandem with orchestration layers such as Model Context Protocol, which Thube notes can “organize and structure that data for the model to process effectively.”
This pairing allows for better automation and accuracy in finance workflows.
Agentic LLMs: Automating Multistep Financial Processes
Agentic LLMs such as AutoGPT and LangChain-based agents introduce autonomy to AI operations, handling detailed multistep processes such as audits, portfolio management and regulatory simulations.
“These models manage complex, multistage tasks and free up human experts for higher-level review,” Thube says. “For example, the LLM can gather records, check them against compliance rules, flag issues and draft a summary without any manual intervention.”
Rosen stresses that agentic systems represent a significant leap in sophistication, but they also require greater governance.
DIG DEEPER: Why data compliance is crucial in financial services.
“You need to have agents talking to other agents,” she says. “We’re seeing vendors offer prebuilt agents that are easier to deploy, but building your own requires a level of maturity and risk tolerance many organizations are not yet ready for.”
Financial teams must also weigh risk when rolling out a new agent. Focus on use cases that bring clear value without overexposing secure data.
“There are higher-impact, lower-risk use cases, like simulating scenarios, where it’s okay to let an agentic system run multistep processes as long as a human reviews the results,” she says. “The risk is when you apply automation to real-world, real-time scenarios like trading.”
“My favorite use cases are things that would be useful to your organization but are infeasible for your workforce to do, like evaluating documents, analyzing requirements or producing new business intelligence,” Rosen says.