From the filing
PDF page 224
No printed folio visible · Image transcription
Read the page text
[Embedded graphic 1 — OCR text] © Jane Street Customer since 2023 Frontier intelligence a trading firm can trust Results Claude Fable 5.1 performance over Fable 5 via Jane Street’s internal agentic coding eval +32% ~ Time to migrate large 1 week codebase with Claude Fable, down from months Background Jane Street is a global trading firm whose edge depends on the quality and speed of its quantitative research. It builds the models, strategies, and systems that price and tradea wide range of financial instruments. For years, its developers struggled to build Al agents that could complete research tasks on their own across large datasets. Early attempts were frustrating and inconsistent. Impact That changed as models became more capable and reliable. When Jane Street adopted Claude 3.7 Sonnet, it saw a step- change in how its agents performed on long, multi-step tasks. Agents could carry a task from start to finish without stopping partway through or needing rework. With each successive model generation, AI adoption accelerated across the firm, and teams built AI into their daily work in research, engineering, and design. Claude is now deployed nearly wall-to-wall across the organization. The payoff is higher research throughput. Researchers now supervise teams of agents that analyze code, monitor training runs, and process results. Experiments that once needed constant supervision now run on their own over a weekend. One researcher estimates completing about 50% more analysis than they did a year ago; the effect is more studies, tested faster. Claude Fable enables agents to take on longer, multi-step tasks. A researcher also used it to refactor a large Python application that he built, organizing the sprawling code into aclean structure. He then migrated the codebase to a more robust language in about a week. Similar work would normally have taken a team months to complete. For a firm “We've always had more ideas than we've had the bandwidth to execute on. Agents like Claude have dramatically improved our velocity, multiplying the impact each person can have.” Craig Falls, Head of Quantitative Research at Jane Street Capital whose edge depends on research speed, Fable has cut the time it takes to run experiments and test new financial tools. For Jane Street, what sets Fable apart is “research taste”— the judgment to pursue reasonable strategies and produce thoughtful work. Its agents check their own work, take on more complex projects, and stay focused on long-term goals. As their results become more reliable, Jane Street is bringing agents deeper into its research. Use has spread well beyond the research teams. Designers now often write production-ready code for features and hand it directly to developers to merge. Jane Street has watched model capability climb quickly on its internal evaluations, which grade an agent’s work on quality, accuracy, and trustworthiness against work its own engineers have already done. Models that once passed just 1% of those tests eventually outgrew them, so Jane Street built a harder agentic coding evaluation. On it, Claude Fable 5.1 improved on Fable 5’s score by 32%. “In internal benchmarks, Claude Fable 5.1 solves more of our coding problems than Fable 5 or Opus 5, and achieves state of the art on trading intuition,” said Craig Falls, Head of Quantitative Research at Jane Street. “While prior models became hard to follow the longer they worked, Fable 5.1 remains readable over long, multi-step tasks.”
This page includes text transcribed from an image. Check the original for tables, images, and exact formatting.