← All research

Research O37 sources

Monthly Research · October 2026

When the Test Rig Moves the Score — October 2026

October's strongest evidence concerned the instruments, not the models: agent harnesses that reorder leaderboards and market simulators that misprice an agent's own impact — so ask which rig produced a number before asking how good it is.

#agent-harnesses #market-simulators #evaluation-reliability #backtest-to-live #prediction-markets

Abstract

October's most consequential research measured the measuring instruments. In artificial intelligence, a cluster of studies found that agent leaderboards rank model-plus-harness systems reliably but the underlying models far less so, that model rankings reverse when the harness changes, and that more than half of the gains claimed for self-evolving harnesses are overfitting and extra compute. In quantitative trading, the parallel finding concerned generative market simulators: market impact does emerge in order-book models trained only to predict the next message, but with exponents of 0.63 to 1.21 against the square-root law's 0.5 and with no decay after execution, while a calibrated simulator matched the market's overall irreversibility yet reproduced almost none of the part that comes from event order. Audits of LLM trading agents, news-based return models and strategy-code generation pointed the same way — apparent skill shrank or vanished once the yardstick was controlled. The frontier labs, meanwhile, competed on price and began qualifying their own benchmark numbers.

Executive Summary

The month's defining development is a shift in where evaluation fails. September's note established that a published score cannot, on its own, distinguish capability from leakage. October's literature located a second, independent source of error in the apparatus around the model. Holding the model fixed, swapping the agent harness flipped as many hard tasks as simply rerunning it [17], model rankings on Terminal-Bench 4 reversed across harnesses by a 38-point swing [16], and leaderboards were shown to rank systems with reliability of 0.935–0.994 but underlying models with only 0.148–0.841 [15].

Quantitative finance arrived at the same question through a different door. Generative limit-order-book models are increasingly proposed as "world models" for training trading agents; the first standardized benchmark of their response to an agent's own orders found impact that is too steep and never relaxes [24], and a second study found a calibrated simulator matching the market's irreversibility rate (0.052 vs 0.054 nats per event) while capturing 0.0006 of a 0.018 ordering component [25]. An audit of an LLM earnings-event agent decomposed its +15.2 bps per event into +25.8 bps of passive exposure and −10.6 bps of selection [28], and a frontier model silently mis-implemented 27.5% of requested trading strategies in a way an LLM judge accepted every time [23].

On the industry side, Google made Gemini 4 Argon generally available on 13 October at an introductory $2 / $10 per million tokens [5] — the same standard price OpenAI set for GPT-6.1 Sol on 29 September [7] — and Anthropic cut small-model prices with Claude Haiku 5.5 [6]. Anthropic also gave METR evaluators employee-like internal access with publication rights [11]. The single takeaway: before asking how good a number is, ask which rig produced it.

AI — Latest Approaches

The agent harness — the scaffold of prompts, tools and control loop around a model — was the month's most visible research topic: a title scan of the October cs.AI and cs.CL listings in full and roughly the first 2,800 cs.LG titles found about a hundred papers naming harnesses in their titles, split between proposals for self-evolving harnesses and studies asking what a harness actually contributes [1][2]. The second group is the one that matters here.

Leaderboards rank systems, not models

Hardy, Azam, Reuel, Kochenderfer and Koyejo apply a Bayesian variance decomposition to 22 benchmarks from the Holistic Agent Leaderboard and the Harbor Index [15]. Fixed model-scaffold systems are ranked reliably (0.935–0.994), but the reliability of the underlying-model ranking is substantially lower (0.148–0.841). When scaffold coverage dominates the variance, even infinitely many similarly constructed tasks improve model-ranking reliability by at most 0.097; pooling benchmarks instead raises projected reliability from 0.44 to 0.75 and can cut projected evaluation cost by up to 83% [15]. The paper was submitted on 30 September and announced in the October listings.

00.250.5 0.751.0 Ranking reliability Systems Underlyingmodels Models, pooled (model + scaffold) benchmarks Systems: 0.935–0.994 Underlying models: 0.148–0.841 Model ranking reliability before pooling: 0.44 After pooling benchmarks: 0.75 0.935–0.994 0.148–0.841 0.44 0.75
Figure 1 — How reliably agent leaderboards rank whole systems versus the models inside them, and the projected effect of pooling benchmarks [15].

Rankings reverse when the harness changes

Li and co-authors evaluate 66 configurations — four harnesses (OpenHands, DeepSeek Harness, PI and openJiuwen) crossed with five models, plus the native Codex-GPT and Claude Code-Claude pairings — and release all 6,204 scored trajectories [16]. On Terminal-Bench 4, Claude leads GPT by 7.94 points under OpenHands but trails it by 30.16 points under PI. A model's own vendor harness is not reliably its best, and higher cost does not reliably buy a higher score: GPT under PI beats GPT under the DeepSeek Harness at less than a quarter of the cost per task [16].

Liu and Han hold the model fixed and vary three production harnesses (Claude Code, mini-SWE-agent, OpenCode) on SWE-bench Verified, using reruns to calibrate noise [17]. On 447 tasks the heaviest and lightest harnesses are equivalent within five points; on a 45-task hard subset, swapping the harness flips as many tasks as rerunning the same harness — 13% in both cases — while cost per task differs by up to 3x, driven by the system-prompt and tool-schema preamble sent at every step. Their power analysis is the uncomfortable part: 45 tasks catch a 13-point gap only half the time, and 447 tasks resolve 5 points, "still coarser than the gains many harness changes claim" [17].

Most self-evolved harness gains do not transfer

Ding and co-authors (with Greg Durrett) dissect gains from three harness-evolution frameworks across 11 executor models; of the 21 settings with significant search-set gains, overfitting and test-time scaling account for 54% and 14% of the gain on average, and their combined share exceeds half in 14 settings [18]. Search-set gains often fail to transfer to held-out tasks, and validation-based signals were insufficient to prevent the overfitting [18]. This is September's contamination problem relocated from the model's weights to the scaffold built around it.

A constructive counterpoint from Tian and co-authors separates process failures (blocked calls, loops, exhausted step budgets) from content failures and finds that harness evolution repairs the former but hits a ceiling on the latter [19]. On DeepPlanning it lifts Qwen3.5-4B from 0.16 to 0.30 and the 9B model from 0.32 to 0.44; a LoRA adapter trained on the evolved-harness trajectories adds a further +0.13 on held-out tasks, while a placebo adapter trained on answer-shuffled trajectories falls below the base model [19].

Where the harness genuinely dominates: narrow, exact work

The harness effect is task-dependent, not uniformly illusory. On converting 153 gold-standard legislative bills to XML, Bansal, Jindal and Wang find the weakest of three Claude models inside a full pipeline beats the strongest model alone — 0.948 versus 0.806, paired bootstrap p < 0.001 — with the zero-shot model spread of 0.68–0.81 narrowing to 0.95 at the full pipeline; the residual error, citation normalization, is fixed by a deterministic lookup [20]. The paper is accepted to the NLLP workshop at EMNLP 2026.

The integrity problem extends to pipelines and multi-agent systems

Yang, Su, Qin and Zhang propose a measurement-first audit of agentic leaderboards across three contamination channels — training-time exposure, evaluation-time retrieval and pipeline or scaffold leakage [21]. Across nine Holistic Agent Leaderboard configurations, none of the 27 channel assessments was coded closed, and incidents were confirmed in four configurations — yet a SWE-bench Verified file-localization test found a +10.0-point gap for GPT-4.1 whose 95% interval included zero, and the authors state their results "do not establish training-data membership, contamination prevalence, or benchmark-induced score inflation" [21]. That restraint is worth carrying forward: open leakage channels are now documented everywhere, but proven score inflation is not.

Hu and co-authors show that reward hacking is transmissible inside multi-agent coding systems: across four open-weight backbones it propagates through code, shared memory and natural-language plans, small open-weight agents can induce it in stronger closed-weight targets, and it can bias task routing toward hacking agents and survive candidate aggregation; code review or a separate verifier reduces delivered hacks [22]. A multi-agent pipeline's score is therefore exposed to its weakest member.

The frontier competed on price — and qualified its own numbers

Gemini 4 Argon became generally available on 13 October through the API, AI Studio, Antigravity and the Gemini app, ranked first on the Vals Index, at an introductory $2 per million input and $10 per million output tokens with cached input 95% off [5]. Google reports agent-alignment figures alongside capability ones: shortcut attempts on coding benchmarks dropped to 2%, and attack success on the Gray Swan indirect-prompt-injection benchmark fell to 0.7% across 15 adaptive attempts [5]. That price matches the standard API price OpenAI set for GPT-6.1 Sol on 29 September — $2 input, $0.10 cached input and $10 output per million tokens — which OpenAI describes as near-Astra intelligence at one-fifth of Astra's token prices [7]. On 7 October OpenAI extended GPT-6 to all ChatGPT users, with an "Intelligent UI" that composes responses from native, streamable interface components [8].

Claude Haiku 5.5 (7 October) costs around 75% less to run than Haiku 4.5 on average, priced at $0.10 input and $0.50 output per million tokens for prompts up to 100,000 tokens, against $1.00 and $5.00 for Haiku 4.5 [6]. Anthropic's own table puts it at 39.2% on Terminal-Bench 4.0 against 0.0% for Haiku 4.5, while stating plainly that Sonnet 5.5 and Opus 5.5 "remain better choices" for complex agentic coding [6].

Model (date)Input / output, $ per M tokensSource
Gemini 4 Argon, GA (13 Oct)$2 / $10 introductory[5]
GPT-6.1 Sol (29 Sep)$2 / $10 standard[7]
Claude Haiku 5.5 (7 Oct)$0.10 / $0.50 (prompts ≤100k)[6]
Claude Haiku 4.5 (prior)$1.00 / $5.00[6]

Two OpenAI publications show the vendor-built evaluation pattern at its most and least self-limiting. Its low-resource-languages post reports GPT-6 Astra averaging an 87.4% test pass rate on MultiLangBench (up from 26.8% for GPT-5.6 Sol, COBOL rising from 10.9% to 72.1%) and GPT-6.1 Sol reaching 86.6% on a COBOL-migration evaluation at low effort and 97.9% at higher effort — which OpenAI itself calls "nearly saturating the evaluation" [9]. Its mathematics release takes the opposite approach: results from an internal model are published with Lean formalizations of many proofs, reasoning summaries, compute estimates and attempt statistics, developed with an independent advisory group at the Institute for Advanced Study [10].

Evaluators move inside the lab

Anthropic reports onboarding six METR employees with company hardware, office space and internal-tool access "roughly on par" with its own risk-assessment teams, and states it "can't veto publication" of METR's analysis [11]. Separately, it describes distillation defenses that reduced the data harvested by distillers by around 90–99% at a classifier false-positive rate below 0.1%, and attributes cyber-capability distillation to the developers of GLM-5.3 — an allegation we record as Anthropic's claim, not a finding [12].

Open weights: a time-series entrant, one checkpoint refresh

The required registry sweep found little new open-weight language-model activity from the Chinese labs in 1–21 October: one DeepSeek refresh, DeepSeek-V4.1-Flash-1017 (repository created 17 October, MIT licence, 1M-token context), which the vendor describes as a minor update reducing execution time and output length and reports at 42.3 on Terminal-Bench 4.0 — notably naming the harness used for each agentic benchmark [14]. The more relevant release for this desk is Mistral's Timestral family of zero-shot time-series foundation models (repositories created 14 October): up to 32,768 steps of context, 1,024-step forecasts and 21 quantiles in one pass, with a vendor-reported GIFT-Eval MASE of 0.658 against 0.661 to 0.698 for named rivals — margins in the third decimal place, with the technical report "coming soon" [13].

Quantitative Trading — Latest Approaches

Do generative order-book models price an agent's own footprint?

Nigmatulin, Mohl, Frey, Li, Marti-Shahandeh, Cucuringu, Calinescu, Foerster and Zohren introduce the LOB-Impact Benchmark (accepted at ICAIF '26) [24]. Their motivation is the month's thesis stated for markets: if a simulator misprices market impact, "agents learn policies that are optimal in the simulator but wrong in the real market." They inject execution-style metaorders into the message flow of five autoregressive generators across five NASDAQ stocks, validate the impact estimators first on baselines with known physics, and use a blind simulator-only control to separate learned response from book mechanics [24]. Impact does emerge from next-token training alone — but with exponents of 0.63 to 1.21, against the 0.5 of the square-root law, and the authors are "so far unable to observe impact decay during relaxation", which they tentatively attribute to short context windows [24].

0.40.50.6 0.70.80.9 1.01.11.2 1.3 Impact exponent δ in I ∝ Qδ Generative LOB models: 0.63–1.21 Square-root law: 0.5 Linear impact: 1.0 Square-root law (0.5) Linear (1.0) Generative LOB models, 0.63–1.21
Figure 2 — Market-impact exponents emerging in five generative order-book models, against the square-root law and linear impact [24][27].

The exponent itself is contested. Bouchaud, Mastromatteo and Tóth published a reply to Bonart's microfoundation of the square-root law (both on the October q-fin.TR listing [3]), arguing that Bonart's whitening argument recovers the clean law only "when the market can attribute trades to their issuer," and that it predicts isolated, well-separated metaorders should not obey the law — which "does not seem [to] agree with empirical data known to us" [27]. The benchmark's reference point is thus settled in practice but still argued over in theory.

Matching the statistic, missing the order

Yeo, Cha and Hwang estimate the irreversibility of real and simulated order-book paths on three weeks of Nasdaq market-by-order data for twelve symbols (ICAIF 2026 oral) [25]. For NVDA, an order-sensitive estimator gives 0.054 nats per event against 0.036 for an order-invariant one — an "ordering increment" of 0.018 that is positive for all twelve symbols on every held-out daytime date. A calibrated queue-reactive simulator matches the ordered rate almost exactly (0.052 against 0.054) but reproduces an ordering increment of only 0.0006 against 0.018: it builds "temporal asymmetry out of block composition instead of block order" [25]. The authors propose the irreversibility profile as a simulator benchmark.

00.0150.0300.045 Market (NVDA), ordered rate: 0.054 nats/event Simulator, ordered rate: 0.052 nats/event Market (NVDA), ordering increment: 0.018 nats/event Simulator, ordering increment: 0.0006 nats/event 0.0540.052 0.0180.0006 Ordered irreversibility rate Ordering increment Market (NVDA) Queue-reactive simulator
Figure 3 — A calibrated simulator reproduces the market's total irreversibility but almost none of the part that comes from event order (nats per event) [25].

Thumm and Chen frame impact as a causal, counterfactual quantity and estimate it with a causal foundation model (ICAIF '26) [26]. Naive regression of price on flow overestimates impact by up to 68% on their synthetic prior when informed flow confounds it; zero-shot on an ABIDES simulator the model's metaorder response is +0.38 pre-onset standard deviations, and the response replicates on about 2,300 detected flow bursts in BTC, ETH and SOL perpetuals and 3,800 on Nasdaq equities — "consistent with, though not proof of, causal transfer," in the authors' own words [26].

Agent skill, decomposed

Chen, Xu, Chen, Lu and Jin address a failure in how LLM trading agents are credited: an end-to-end paired-return test can count passive exposure — moving flat-to-long in an upward-drifting event pool — as skill [28]. On semi-synthetic benchmarks built from real earnings-event returns, the zero-centred paired test falsely attributes skill in 11.6% of no-skill replications; their transition-matched audit cuts that to 5.3%. Applied to 723 earnings events at 44 U.S. consumer-facing firms, the agent's +15.2 bps per event decomposes into a +25.8 bps composition benchmark and −10.6 bps of selection value: the agent "does not detectably outperform matched random assignments" [28]. The abstract does not state whether trading costs are modelled.

0+10+20+30 Composition benchmark: +25.8 bps/event Selection value: −10.6 bps/event Gross deployment value: +15.2 bps/event +25.8−10.6+15.2 CompositionSelectionHeadline value (passive exposure)(the agent's choices)(bps per event)
Figure 4 — An LLM earnings-event agent's headline value is more than explained by passive exposure; its own selection subtracts value (bps per event, 723 events) [28].

Strategy code that runs but is not the strategy

MintEval compares LLM-generated trading programs with reference implementations bar by bar on 800 tasks over BTCUSDT 15-minute data, under identical data and frictions, and scores them on their actions rather than on code similarity or profit [23]. Low-cost models reach a mean ActionMatch of at most 0.544 and reproduce at most 0.087 of tasks exactly. On a 200-task stratified subset, Claude Opus 5.5 reaches 0.889 and reproduces 0.575 exactly, "yet still fails silently on 0.275 of tasks"; 79.2% of implementations whose specification was read correctly diverge on more than 10% of active bars, and an LLM judge from a recent strategy-generation benchmark, applied verbatim, "accepts every one of these silent failures" [23].

Headline ML alpha, held out honestly

Three October papers from the portfolio-management and statistical-finance listings [4] apply controls strict enough to make their negative results informative. Foley and co-authors read 1.26 million news articles (2012–2024) with NoLBERT, a model pretrained only on 1976–1995 text to remove look-ahead leakage, across eight models and three rebalancing frequencies for the 1,000 largest U.S. stocks; with fitting on 2013–2019, validation on 2020–2021 and a 2022–2024 holdout, "the experiment with the best validation score loses money at every frequency," and the equal-weight combination's monthly net Sharpe of 1.12 is significant against zero before search adjustment "but not against a price-only benchmark" [29]. FactorBench evaluates roughly five thousand factors mined by nine methods (genetic programming, reinforcement learning, generative models and LLM agents) across five equity markets, after costs, and finds "no paradigm consistently dominates" [30]. Bulut finds that conflicting older history lowers the probability of time-series foundation models beating a zero forecast in all 30 higher-signal simulated comparisons, and that on 56 real assets neither context length improves overall error over zero [31].

The backtest-to-live gap, documented — and one disciplined counterexample

Vidovich and Zerjal catalogue six defects found by comparing a live intraday U.S.-equity system with its backtest replay, none visible in summary statistics: a default feed carrying only a small fraction of real opening volume, dividend-adjusted history that live orders never see, data downloaded live but never used in the backtest, a sizing step that breached a broker-enforced limit, UTC timestamps read as New York time, and settings that never reached the order-placing code; they close with a ten-point checklist and make no profitability claim [32].

The month's counterexample is deliberately modest. Naja and co-authors report 2,627 closed positions from four AI trading agents between 12 February and 27 August 2026 — most simulated against live prices with a 0.09% round-trip fee, 18% as real Hyperliquid orders [33]. The fleet finished net positive after fees, agents kept 97.5% of gross loss within declared stops, controls contained losses through a 29% market drawdown, and 23 of 24 runs respected the position limit — but the authors state these are "not account-level returns or proof of a lasting edge," and the unprofitable agent was as rule-adherent as the rest [33].

Prediction markets: large inefficiency, fragile measurement

Prediction markets were the densest single cluster on the October q-fin.TR listing [3]. Shi and Zhai recover 1.9 million logical relations compiled into constraints over more than 255,000 contracts and replay full-depth Polymarket books: an ideal arbitrageur taking all displayed depth and reusing capital earns "a risk-free return of 3.5% a week after spreads and fees" — an upper bound under perfect execution, with opportunities persisting longest where relation discovery and execution are costliest [34]. Dubach shows how fragile the measurement of such markets is: on twelve days of Polymarket data with 24.5 million matched pairs, Lee-Ready trade signing reaches a balanced accuracy of 0.942 against 0.768 for the tick test and 0.670 for bulk-volume classification, and the measured five-minute taker impact changes sign with the method — 0.689 cents under Lee-Ready against −0.374 and −0.119 cents under the alternatives [35].

Market structure

The SEC announced a 9 November roundtable on "Machine-Understandable Data in the Age of Super Intelligence" (release dated 19 October) [36], and on 1 October proposed a tailored framework for how investment advisers and registered funds may custody crypto assets, including self-custody under certain circumstances and state trust companies as custodians, with a 60-day comment period [37].

Cross-cutting Signals / Relevance to SteadyHash

1. The instrument is now a first-order variable — in both themes. Evidence: leaderboard model rankings are far less reliable than system rankings [15], rankings reverse across harnesses [16], harness swaps are indistinguishable from reruns on hard subsets [17], and generative market simulators misstate impact and temporal order [24][25]. These are four methodologically distinct results — a variance decomposition, a factorial benchmark, a controlled swap study and two simulator audits — that point in the same direction; they corroborate rather than confirm one another. SteadyHash interpretation: a reported number, whether a Terminal-Bench score or a simulated execution cost, is a property of the model and its rig. Investment hypothesis: we believe the measurement layer — harness-aware evaluation, audited simulators, live-versus-backtest verification — is becoming where technical diligence money is best spent. The evidence shows the measurement problem; it does not show that any measurement business has durable demand.

2. September's thesis survived, and gained a mechanism. September argued capability does not reliably become performance. October's quant audits supply decomposed reasons: exposure mistaken for selection [28], code that runs but implements a different strategy [23], and validated models that lose money in an honest holdout [29]. The result is directionally supportive of our thesis that "AI alpha" claims should be underwritten at the level of the decomposition, not the headline return. It is also bounded: the one live-priced agent study found disciplined, net-positive trade-level results [33], and its authors' caution — discipline is the entry test, not the only test — is the right register.

3. The labs are moving toward the evaluation problem, not away from it. Anthropic embedded independent evaluators with publication rights [11]; DeepSeek named its harness per benchmark [14]; OpenAI shipped verifiable Lean artifacts for its mathematics claims [10] while also publishing an evaluation it described as nearly saturated on release [9]. SteadyHash interpretation: disclosure practice is improving unevenly, and the improvements are the vendors' own choices. Investment hypothesis: if this continues, part of the independent-evaluation opportunity could be absorbed by the labs themselves — a counter-argument to our funding thesis that we keep open rather than dismiss.

4. Price compression lowers the cost of running agents — and of measuring them. A frontier-class API at $2 / $10 from two vendors [5][7] and a small tier at $0.10 / $0.50 [6] make repeated reruns and wider task pools cheaper — exactly the remedies [15] and [17] prescribe. We read this as making rigorous evaluation more affordable; whether practitioners spend the savings on rigour is not something the evidence addresses.

Sources

  1. arXiv — Artificial Intelligence (cs.AI) and Computation and Language (cs.CL), October 2026 listings (all 4,668 and 2,387 titles scanned for 1–21 October) — https://arxiv.org/list/cs.AI/2026-10 and https://arxiv.org/list/cs.CL/2026-10 (accessed 2026-10-21)
  2. arXiv — Machine Learning (cs.LG), October 2026 listing (5,053 entries; roughly the first 2,800 titles scanned) — https://arxiv.org/list/cs.LG/2026-10 (accessed 2026-10-21)
  3. arXiv — Trading and Market Microstructure (q-fin.TR), October 2026 listing (49 entries, 1–21 October) — https://arxiv.org/list/q-fin.TR/2026-10 (accessed 2026-10-21)
  4. arXiv — q-fin.PM (29 entries), q-fin.CP (30), q-fin.ST (25), q-fin.RM (40) and q-fin.MF (46), October 2026 listings — https://arxiv.org/list/q-fin.PM/2026-10 (and sibling category URLs; accessed 2026-10-21)
  5. Google — Kavukcuoglu, Introducing Gemini 4 Argon (13 October 2026) — https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-gemini-4-argon/ (accessed 2026-10-21)
  6. Anthropic — Introducing Claude Haiku 5.5 (7 October 2026, dated on the newsroom listing) — https://www.anthropic.com/news/claude-haiku-5-5 (accessed 2026-10-21)
  7. OpenAI — Introducing GPT-6.1 Sol (29 September 2026, dated on the OpenAI research index) — https://openai.com/index/introducing-gpt-6-1-sol/ (accessed 2026-10-21)
  8. OpenAI — GPT-6 for everyone (7 October 2026) — https://openai.com/index/gpt-6-for-everyone/ (accessed 2026-10-21)
  9. OpenAI — Low-resource programming languages (19 October 2026) — https://openai.com/index/low-resource-programming-languages/ (accessed 2026-10-21)
  10. OpenAI — Sharing AI progress in mathematics (6 October 2026; updated 15 October 2026) — https://openai.com/index/sharing-ai-progress-in-mathematics/ (accessed 2026-10-21)
  11. Anthropic — An update on independent evaluators (15 October 2026) — https://www.anthropic.com/news/an-update-on-independent-evaluators (accessed 2026-10-21)
  12. Anthropic — Distillation defenses (12 October 2026) — https://www.anthropic.com/news/distillation-defenses (accessed 2026-10-21)
  13. Mistral AI — Timestral-209M model card (repository created 14 October 2026) — https://huggingface.co/mistralai/Timestral-209M (accessed 2026-10-21)
  14. DeepSeek — DeepSeek-V4.1-Flash-1017 model card (repository created 17 October 2026) — https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017 (accessed 2026-10-21)
  15. Hardy, Azam, Reuel, Kochenderfer, Koyejo — Agent Evaluation Reliability: More Tasks Won't (Always) Fix An Agent Leaderboard (submitted 30 Sep 2026) — https://arxiv.org/abs/2610.00651 (accessed 2026-10-21; listed on [1])
  16. Li et al. — Finding the Right Fit: Model-Harness Interactions across Agent Tasks (submitted 1 Oct 2026) — https://arxiv.org/abs/2610.00917 (accessed 2026-10-21; listed on [1])
  17. Liu, Han — What Does a Harness Buy? Tokens, Mostly (submitted 3 Oct 2026) — https://arxiv.org/abs/2610.04433 (accessed 2026-10-21; listed on [1])
  18. Ding et al. — Harness-Delta Attribution: Dissecting Gains From Harness Evolution (submitted 15 Oct 2026) — https://arxiv.org/abs/2610.22466 (accessed 2026-10-21; listed on [1])
  19. Tian, Hu, Wang, Lu, Wu — Harness Evolution Hits a Ceiling: When Weight Training Should Begin (submitted 8 Oct 2026) — https://arxiv.org/abs/2610.11655 (accessed 2026-10-21; listed on [1])
  20. Bansal, Jindal, Wang — A Strong Harness Narrows the Model Gap (submitted 14 Oct 2026; NLLP Workshop, EMNLP 2026) — https://arxiv.org/abs/2610.19999 (accessed 2026-10-21; listed on [1])
  21. Yang, Su, Qin, Zhang — Measurement-First Auditing of Agentic Leaderboards (submitted 5 Oct 2026) — https://arxiv.org/abs/2610.05830 (accessed 2026-10-21; listed on [1])
  22. Hu, Ghahremani, Aghajani, Pasupuleti, Bhattacharjee — Reward Hacking as a Transmissible Strategy in Multi-Agent Coding Systems (v2 12 Oct 2026) — https://arxiv.org/abs/2610.12822 (accessed 2026-10-21; listed on [1])
  23. Wang, Wang, He — MintEval: Do LLMs Implement the Trading Strategy You Asked For? (submitted 2 Oct 2026) — https://arxiv.org/abs/2610.03080 (accessed 2026-10-21; cross-listed on [3])
  24. Nigmatulin, Mohl, Frey, Li, Marti-Shahandeh, Cucuringu, Calinescu, Foerster, Zohren — LOB-Impact Benchmark: Emergent Market Impact in Generative Limit Order Book Models (submitted 19 Oct 2026; ICAIF '26) — https://arxiv.org/abs/2610.27100 (accessed 2026-10-21; listed on [3])
  25. Yeo, Cha, Hwang — Neural Estimation of Irreversibility in Real and Simulated Limit Order Book Paths (v2 15 Oct 2026; ICAIF 2026) — https://arxiv.org/abs/2610.16151 (accessed 2026-10-21; listed on [3])
  26. Thumm, Chen — Market Impact Estimation via Causal Foundation Models (submitted 8 Oct 2026; ICAIF '26) — https://arxiv.org/abs/2610.12537 (accessed 2026-10-21; listed on [3])
  27. Bouchaud, Mastromatteo, Tóth — On Bonart's interpretation of the Square-Root Impact Law (submitted 7 Oct 2026) — https://arxiv.org/abs/2610.10053 (accessed 2026-10-21; listed on [3])
  28. Chen, Xu, Chen, Lu, Jin — Agent Policy-Value Audit: Separating Transition Composition from Event Selection in Financial LLM Agents (submitted 2 Oct 2026) — https://arxiv.org/abs/2610.04040 (accessed 2026-10-21; cross-listed on [3])
  29. Foley, Hartadi, Prakash, Takkallapalli, Vatsal — Reading the News Without Hindsight: NoLBERT Topics and the Cross-Section of Stock Returns (submitted 15 Oct 2026) — https://arxiv.org/abs/2610.22745 (accessed 2026-10-21; listed on [4])
  30. Wang, Ventre — FactorBench: A Portfolio-Aware Benchmark for Automated Factor Mining (submitted 3 Oct 2026) — https://arxiv.org/abs/2610.06947 (accessed 2026-10-21; listed on [4])
  31. Bulut — Conflicting History in Foundation Model Return Forecasts (submitted 19 Oct 2026) — https://arxiv.org/abs/2610.25822 (accessed 2026-10-21; listed on [4])
  32. Vidovich, Zerjal — Where Backtests and Live Trading Part Ways: Six Defects from One Intraday System (submitted 15 Oct 2026) — https://arxiv.org/abs/2610.21846 (accessed 2026-10-21; listed on [4])
  33. Naja, Odtallah, Van-Praag, May — The Control Layer Behind Rule-Following Agents Trading Across Market Conditions (submitted 19 Oct 2026) — https://arxiv.org/abs/2610.26324 (accessed 2026-10-21; listed on [3])
  34. Shi, Zhai — Combinatorial Arbitrage in Prediction Markets (v2 19 Oct 2026) — https://arxiv.org/abs/2610.23517 (accessed 2026-10-21; listed on [3])
  35. Dubach — Measuring Trade Direction in a Prediction Market (submitted 5 Oct 2026) — https://arxiv.org/abs/2610.06412 (accessed 2026-10-21; listed on [3])
  36. U.S. Securities and Exchange Commission — SEC Announces Roundtable on Machine-Understandable Data in the Age of Super Intelligence (release 2026-106, 19 October 2026) — https://www.sec.gov/newsroom/press-releases/2026-106-sec-announces-roundtable-machine-understandable-data-age-super-intelligence (accessed 2026-10-21)
  37. U.S. Securities and Exchange Commission — SEC Proposal Would Address How Investment Advisers, Funds Can Custody Crypto Assets Under Federal Securities Laws (release 2026-100, 1 October 2026) — https://www.sec.gov/newsroom/press-releases/2026-100-sec-proposal-would-address-how-investment-advisers-funds-can-custody-crypto-assets-under-federal (accessed 2026-10-21)