Investor’s Last Exam: Can AI Agents Do Equity Investment Without Seeing the Future?

Investor’s Last Exam asks whether AI can research stocks without looking ahead

Equity research is not simply a finance question-answering exercise. A professional analyst must identify relevant evidence, reconcile accounting definitions, build assumptions, forecast future results, assess valuation and risk, and turn the whole chain into a defensible investment decision. The central challenge is that the forecast must be made using only information available at the time.

In Investor’s Last Exam: Can AI Agents Do Equity Investment Without Seeing the Future?, the authors introduce Investor’s Last Exam, or ILE, a benchmark designed to test that complete process. Their headline finding is uncomfortable for developers of financial agents: strong research and reasoning scores do not reliably produce accurate forecasts or successful investment decisions.

Why ordinary financial benchmarks are not enough

Earlier benchmarks have examined pieces of financial work, including document question answering, numerical reasoning, search, research reports, and forecasting. The authors argue that these tests leave a major vulnerability: a date filter on a search query does not guarantee that the retrieved page contains only information available before the cutoff. Web pages can be revised, metadata can be incomplete, and retrospective reports can expose outcomes that were not knowable at decision time.

ILE therefore freezes the information environment at the level of individual source versions. The benchmark database contains 3.43 million document versions across nine source categories, supplemented by structured market data. Each version has an availability time, and the same cutoff is enforced for search, document reading, downloads, tables, cached results, and local computation. Later disclosures and private scoring materials remain outside the agent’s workspace.

This matters because forecasting an already-revealed result is fundamentally different from forecasting an uncertain future. In one audit described by the authors, a dated web query nevertheless returned material reporting Tesla’s realized second-quarter deliveries. In the ILE environment, the corresponding post-cutoff version would be unavailable.

A benchmark built around the analyst’s workflow

The authors organize 86 assignments around professional deliverables rather than generic topics. The tasks cover Discover & Frame, Research & Evidence, Forecast & Model, Evaluate & Decide, Monitor & Update, Integrated Research, and Market Trading. Assignments test issues such as accounting reconciliation, forecasting from operating drivers, peer read-throughs, valuation, thesis updates, and price or return predictions.

A distinctive feature is the financial dependency graph. Expert-led task construction links each scored requirement to source evidence, calculations, assumptions, and the final judgment. This allows evaluators to ask not only whether an answer is correct, but where its reasoning failed. The scoring dimensions include result accuracy, evidence, financial logic, calculation, reconciliation, decision consistency, and uncertainty.

Numerical requirements receive deterministic checks, while semantic criteria are judged by GPT-6 Astra in the main evaluation. The authors also report sensitivity to the choice of judging model, an important qualification when interpreting automated scores.

The main results: research quality is not forecasting ability

Across eleven systems, GPT-6 Astra leads the answer-based overall ranking with 68.1 out of 100, followed by Claude Opus 5.5 at 66.9. However, Astra scores 78.9 in Research & Evidence and 77.6 in Evaluate & Decide, compared with only 52.5 in Forecast & Model. Its score for Integrated Research is 66.3, showing how weaknesses can reappear when separate capabilities must work together.

The rankings change when the benchmark emphasizes realized forecast outcomes. DeepSeek V4.1 Flash leads that measure at 40.3, ahead of Astra at 37.7 and Opus at 37.6. The authors also report a small retrospective trading simulation in which Hunyuan 4 Preview produces an 8.50% return, followed by DeepSeek V4.1 Flash at 7.98% and Astra at 7.18%, compared with 1.18% for an always-long strategy. Kimi K3 achieves a higher simulated return than Opus despite a lower overall research score.

These results do not show that one model is generally the best investor. They demonstrate that different evaluation targets produce different orderings. Research quality, forecast accuracy, and trading returns measure related but distinct capabilities, and a single aggregate score can conceal those differences.

Where financial agents break down

The case studies point to failures beyond retrieval and arithmetic. In an American Electric Power assignment, all eleven systems recover the reported inputs and reconcile two cash-flow identities. Yet only five compute both funding shortfalls, three distinguish a net deferred-fuel adjustment from gross fuel collections, and five reach the supported funding conclusion. Higher operating cash flow coexists with a wider pre-financing funding gap, a relationship that requires interpretation rather than mechanical calculation.

Other cases expose problems with forecast assumptions, reporting scope, and model consistency. A coherent revenue bridge can still rely on assumptions that miss the realized outcome. A correct position-sizing formula does not justify the downside estimate behind it. Overlapping measures, such as a company-wide segment breakdown and a separate cloud-revenue figure, can also create double counting if their definitions are not reconciled.

Execution is another practical weakness. Under strict evaluation, which requires an accepted submission, some systems lose substantial performance because they omit or mishandle the required final deliverable. The authors therefore separate analytical quality from submission reliability.

Limitations and implications

ILE reduces look-ahead leakage from external tools, but it cannot fully determine whether a frontier model has memorized historical outcomes during pretraining. The authors propose counterfactual and synthetic financial scenarios as one possible way to isolate that problem. They also caution that realized market outcomes contain noise: a sound analysis can miss a target because of events outside the model’s information set.

For researchers, the practical lesson is to evaluate financial agents as evidence-to-judgment systems, not as chatbots with finance knowledge. Tests should freeze data versions, verify source provenance, score assumptions and reconciliation, and separate forecast outcomes from the quality of supporting analysis. For businesses, the findings suggest that an AI research assistant may already help organize evidence or structure decisions, while still requiring human review of definitions, assumptions, uncertainty, and final recommendations.

The authors’ broader claim is demonstrated by the benchmark results: better retrieval and polished reasoning are not sufficient for dependable investment research. The harder problem is calibrating assumptions and preserving consistency from evidence through forecast to decision.


Sources

AI-assisted research summary. Check the original paper for complete methods, results, and limitations.

Avatar photo

Tech News

Jeff is a passionate blog writer who shares clear, practical insights on technology, digital trends and AI industries. With a focus on simplicity and real-world experience, his writing helps readers understand complex topics in an accessible way. Through his blog, Jeff aims to inform, educate, and inspire curiosity, always valuing clarity, reliability, and continuous learning.

↑ ↓ navigate Enter open Esc close