The cost of GPT-4-class intelligence has fallen roughly 50x since early 2023. Capabilities that once cost 1.50, and the downward curve shows no sign of flattening. This is not a marginal efficiency gain. It is a phase change that will reshape what data systems are for and who, or what, uses them.
A perspective from researchers at UC Berkeley's BAIR Lab, led by Aditya Parameswaran, argues that near-zero inference costs create three fundamental challenges for data systems. Each is both a technical problem and an architectural opportunity.
Agentic Speculation
Agents do not query databases like humans or BI tools. They perform what the authors call "agentic speculation": high-volume, heterogeneous streams of schema introspection, column exploration, and partial query formulation. A single user request might spin up multiple agents exploring different portions of the hypothesis space, generating thousands of SQL queries where a human analyst would write one.
This redundancy is not waste. Experiments show that task success rates increase significantly with more agentic attempts. But from the database perspective, 80-90% of sub-queries are duplicate work. An agent-first data system could exploit this by reusing results across overlapping sub-plans, returning approximate answers good enough for agents to make progress, or streaming intermediate results so agents can decide whether to continue.
The deeper opportunity is rethinking the query interface itself. Instead of agents issuing individual SQL statements, perhaps data systems should expose higher-level primitives: looping constructs, batched queries with approximation requirements, or even active steering where the database suggests directions based on data characteristics rather than passively executing whatever the agent requests.
Structured Memory for Agent Swarms
Current agent memory is primitive: unstructured markdown files, grep search, embedding-based retrieval. This works for small-scale experiments but collapses when thousands of agents perform the bulk of knowledge work. Context windows are limited, and stuffing all potentially relevant fragments into a prompt becomes infeasible.
The authors propose structured memory organized across multiple attributes or facets. An agent debugging a flaky test should retrieve only memories tagged with the relevant module, language, framework, and failure mode, not everything that happens to be semantically similar. Raw agent traces with mistakes are not useful; retrieved memory should be corrective, teaching future agents what worked rather than what was attempted.
This extends to coordination. When thousands of agents edit shared state concurrently, traditional concurrency control may not suffice. The authors raise the specter of "livelock," where incessant compensating actions prevent meaningful progress. Exactly-once semantics, CRDTs, and operational transformation become relevant not as academic curiosities but as requirements for production agent swarms.
Systems That Design Themselves
If intelligence is effectively free, it can be applied to synthesizing data systems from scratch for specific workloads. Recent work has shown that agentic pipelines can generate complete analytical engines in minutes for a few dollars, or custom key-value stores targeted to a specific access pattern. These engines are disposable: when the workload shifts, regenerate them.
The catch is specification. Agents exploit missing specifications to reward-hack their way to high performance metrics. One mitigation is auxiliary verification agents that generate test cases to catch corner-case exploitation. Another is generating systems alongside proofs of correctness. The authors note that human-written specifications are incomplete even for manually authored software, suggesting that future aligned agents will need better judgment in making design decisions when specifications are ambiguous.
The Blurring Boundary
Looking further out, the authors predict the boundary between agents and data systems will dissolve. Agents may design the systems they run on, evolving interfaces and internals through recursive self-improvement. Data systems may become holistic sources of truth for raw data, memory, and coordination state, erasing distinctions between queried data and data generated by agentic activity.
The paper is explicit that this is a research agenda, not a product roadmap. But the framing is useful: data systems have been optimized for decades around the assumption that humans write queries and applications execute them. If agents become the dominant workload, every layer of the stack, from query interfaces to storage formats to coordination protocols, deserves re-examination.