← All research

Research F12 sources

Monthly Research · April 2026

The Verification Gap — April 2026

Frontier models now beat experts in adversarial domains while the benchmarks that measure them prove gameable — independent evaluation becomes the scarce capability.

#benchmark-gaming #independent-evaluation #model-evals #adversarial-domains

Abstract

April was the month the industry began stress-testing its own claims: Anthropic disclosed an unreleased frontier model that finds and exploits software vulnerabilities better than all but the most skilled humans, days before UC Berkeley researchers showed that all eight leading AI-agent benchmarks can be gamed to near-perfect scores. The quant arXiv record was dominated by LLM-agent work — simulated markets of LLM traders that bubble and destabilize, constrained factor-discovery agents, and structural warnings about homogeneous AI-driven markets. AI capability in finance is no longer the bottleneck; trustworthy evaluation of that capability is.

Executive Summary

April 2026 read as the month the industry began stress-testing its own AI claims. Anthropic disclosed that an unreleased frontier model, Claude Mythos Preview, can find and exploit software vulnerabilities better than all but the most skilled humans, and convened a twelve-member coalition — Project Glasswing — to direct that capability defensively [9]. Days later, UC Berkeley researchers showed that every one of eight leading AI-agent benchmarks can be gamed to near-perfect scores without solving a single task, undercutting the headline numbers that drive model selection and, increasingly, valuations [10]. The infrastructure layer kept consolidating: Cloudflare unified 70+ models from 12+ providers behind a single agent-oriented inference API with automatic cross-provider failover [11]. On the quant side, the April arXiv record is dominated by LLM-agent work — simulated markets of LLM traders that bubble and destabilize [3], constrained LLM agents for factor discovery [8], and structural warnings about homogeneous AI-driven markets [4] — alongside agentic portfolio architectures from established practitioners [6]. The takeaway for a partner: AI capability in finance is no longer the bottleneck; trustworthy evaluation of that capability is, and firms that can independently verify what their models actually do will hold the edge.

AI — Latest Approaches

Anthropic's Project Glasswing: frontier models cross the offensive-security threshold

On April 7, Anthropic announced Project Glasswing, an initiative joining Amazon Web Services, Anthropic, Apple, Broadcom, Cisco, CrowdStrike, Google, JPMorganChase, the Linux Foundation, Microsoft, NVIDIA, and Palo Alto Networks to secure the world's most critical software [9]. The trigger was Claude Mythos Preview, an unreleased general-purpose frontier model that Anthropic says surpasses all but the most skilled humans at finding and exploiting software vulnerabilities, and which has already found thousands of high-severity vulnerabilities — including some in every major operating system and web browser [9]. The announcement frames proliferation of these capabilities as near-term and inevitable, and the coalition as an attempt to put them to defensive use first [9].

Berkeley RDI breaks every major AI-agent benchmark

Researchers at UC Berkeley (Wang, Mang, Cheung, Sen, Song) published an April study showing that an automated scanning agent could exploit all eight of the most prominent AI-agent benchmarks — including SWE-bench, WebArena, OSWorld, GAIA, and Terminal-Bench — achieving near-perfect scores without solving a single task [10]. The exploits are practical, not theoretical: a ten-line conftest.py "resolves" every instance of SWE-bench Verified through the official evaluation pipeline [10]. Since these leaderboards are cited in press releases, used to justify valuations, and consulted for deployment decisions, the result materially weakens benchmark scores as a due-diligence signal [10].

Cloudflare ships a unified, agent-first inference layer

On April 16, Cloudflare turned its AI Gateway and Workers AI into a single inference layer: one API exposing 70+ models across 12+ providers (OpenAI, Anthropic, Google, Alibaba Cloud, ByteDance, and others), with one-line model switching, consolidated billing, and automatic failover to an alternate provider when one goes down [11]. The design explicitly targets agents, where chained calls amplify latency and a single failed request cascades — streamed responses are buffered server-side so interrupted agents can reconnect without re-paying for tokens [11]. It is further evidence that the agentic stack is commoditizing at the routing layer while differentiation moves to models and orchestration [11].

AI-generated content and open releases reach industrial scale

The month's popular-press record (as indexed by Hacker News' April 2026 story data) shows the supply side of generative AI scaling fast: TechCrunch reported on April 20 that Deezer says 44% of songs uploaded to its platform daily are AI-generated, and Microsoft released VibeVoice, an open-source "frontier voice AI," in late April [12]. The same index records growing institutional entanglement — a reported Google–Pentagon agreement covering "any lawful" use of AI — and mounting friction in open-source communities over AI-authored contributions [12].

Quantitative Trading — Latest Approaches

"Machine Spirits": LLM trading agents bubble, adapt, and destabilize

Saxena, Pangallo, Hommes, Caccioli, and del Rio-Chanona (submitted April 9) simulate asset markets populated by 15 different LLMs and find behaviour ranging from stable coordination on fundamental value to human-like speculative bubbles — generally inconsistent with rational expectations [3]. In mixed populations, even the most advanced models fail to consistently stabilize prices; instead they adapt their forecasting to other agents, profitably exploiting less sophisticated counterparts while amplifying volatility [3]. The authors conclude that heterogeneous LLM populations can generate endogenous instability — directly relevant to any desk deploying, or trading against, LLM-driven flow [3].

Structural risk from AI homogeneity in markets

Qiu and Han's "Representation Homogeneity and Systemic Instability in AI-Dominated Financial Markets" (q-fin.TR, April) takes a structural approach to the same concern from the other direction: when many market participants run models with similar internal representations, correlated behaviour becomes a systemic property rather than a coincidence [1][4]. Together with the Machine Spirits results, April's literature treats AI-crowding as a measurable risk factor, not a hypothetical [3][4].

LLM agents as constrained factor researchers

Huang, Fan, Hu, and Ye's "From Hypotheses to Factors: Constrained LLM Agents in Cryptocurrency Markets" (q-fin.PM/TR, April) puts LLM agents inside a disciplined factor pipeline — generating economic hypotheses that are then constrained and tested as systematic factors in crypto markets — rather than letting models pick trades directly [8]. The same group's companion work on LLM-augmented semantic networks for cross-stock predictability appears in the same month's q-fin.PM listing [2][8]. The pattern matches our internal stance: LLMs as hypothesis generators under statistical supervision, not as discretionary traders.

Early detection of microstructure regimes in limit order books

Hiremath and Hiremath (cs.LG/q-fin.TR, April) propose a method for early detection of latent microstructure regimes in limit order books, combining identifiability and early-detection theoretical guarantees with a 200-run simulation study and preliminary real-data evaluation on BTC/USDT order books, with code and data released [1][5]. Regime detection with provable latency bounds is directly applicable to execution and market-making risk controls in fast crypto microstructure.

Agentic architectures and tail risk in portfolio construction

On the buy-side architecture front, Ang, Azimbayev, and Kim circulated "The Self Driving Portfolio: Agentic Architecture for Institutional Asset Management" (cs.AI/q-fin.PM, April), sketching how multi-agent systems can run institutional portfolios end-to-end [2][6]. In the same listing, Liang, Koyluoglu, Alican, and Ihlamur's "Beyond Picking Winners" shows that correlation-driven tail risk — not single-name selection — dominates venture-capital portfolio construction outcomes [2][7]. Both bear directly on how a quantitative VC should size and diversify rather than merely select.

Cross-cutting Signals / Relevance to SteadyHash

The dominant cross-cutting theme in April is the verification gap. Anthropic demonstrated that frontier models now exceed expert humans in a high-stakes adversarial domain [9], while Berkeley demonstrated that the public instruments used to measure such capabilities are trivially gameable [10]. For a quantitative firm, the implication is the same one the q-fin literature reached this month about LLM stock-picking [8]: published scores and demos are marketing surfaces; only in-house, adversarial evaluation against our own tasks and data constitutes evidence. This applies equally to model selection for our research stack and to diligence on AI-native portfolio companies whose pitch decks lead with benchmark numbers.

Second, the LLM-agent market literature is converging on a risk we should price: crowding and homogeneity. Simulated markets of LLM traders generate endogenous bubbles and volatility amplification [3], and structural analysis suggests representation homogeneity across AI participants is itself a source of systemic instability [4]. As more flow becomes LLM-mediated, correlation regimes may shift faster than historical covariance estimates capture — an argument for regime-detection tooling of the kind appearing in the April microstructure work [5], and for stress tests that assume AI-correlated deleveraging.

Third, the infrastructure layer is commoditizing in our favour. Cloudflare's unified inference API — many providers, one endpoint, automatic failover, consolidated cost telemetry [11] — lowers the operational cost of running multi-model research pipelines and weakens single-vendor lock-in. Combined with the April portfolio literature's emphasis on correlation-driven tail risk in venture portfolios [7], the practical posture for SteadyHash is unchanged but sharpened: diversify model dependencies like positions, verify capability claims like backtests, and treat AI-crowding as a first-class risk factor.

Sources

  1. arXiv — q-fin.TR (Trading and Market Microstructure), Authors and titles for April 2026https://arxiv.org/list/q-fin.TR/2026-04 (accessed 2026-06-12)
  2. arXiv — q-fin.PM (Portfolio Management), Authors and titles for April 2026https://arxiv.org/list/q-fin.PM/2026-04 (accessed 2026-06-12)
  3. Saxena, Pangallo, Hommes, Caccioli, del Rio-Chanona — Machine Spirits: Speculation and Adaptation of LLM Agents in Asset Marketshttps://arxiv.org/abs/2604.18602 (accessed 2026-06-12)
  4. Qiu, Han — Representation Homogeneity and Systemic Instability in AI-Dominated Financial Markets: A Structural Approachhttps://arxiv.org/abs/2604.22818 (accessed 2026-06-12; listed in [1])
  5. Hiremath, Hiremath — Early Detection of Latent Microstructure Regimes in Limit Order Bookshttps://arxiv.org/abs/2604.20949 (accessed 2026-06-12; listed in [1])
  6. Ang, Azimbayev, Kim — The Self Driving Portfolio: Agentic Architecture for Institutional Asset Managementhttps://arxiv.org/abs/2604.02279 (accessed 2026-06-12; listed in [2])
  7. Liang, Koyluoglu, Alican, Ihlamur — Beyond Picking Winners: Correlation-Driven Tail Risk in Venture Capital Portfolio Constructionhttps://arxiv.org/abs/2604.23087 (accessed 2026-06-12; listed in [2])
  8. Huang, Fan, Hu, Ye — From Hypotheses to Factors: Constrained LLM Agents in Cryptocurrency Marketshttps://arxiv.org/abs/2604.26747 (accessed 2026-06-12; listed in [1][2])
  9. Anthropic — Project Glasswing: Securing critical software for the AI erahttps://www.anthropic.com/glasswing (accessed 2026-06-12)
  10. Wang, Mang, Cheung, Sen, Song (UC Berkeley RDI) — How We Broke Top AI Agent Benchmarks: And What Comes Nexthttps://rdi.berkeley.edu/blog/trustworthy-benchmarks-cont/ (accessed 2026-06-12)
  11. Lu, Chen (Cloudflare) — Cloudflare's AI Platform: an inference layer designed for agentshttps://blog.cloudflare.com/ai-platform/ (accessed 2026-06-12)
  12. Hacker News (Algolia search API) — Top AI stories, 2026-04-01 to 2026-04-30https://hn.algolia.com/api/v1/search?query=AI&tags=story&numericFilters=created_at_i%3E1775001600,created_at_i%3C1777593600&hitsPerPage=30 (accessed 2026-06-12)