Can today’s most powerful AI coding tools actually handle the deep physics embedded in scientific imaging workflows? A new benchmark called Imaging-101 was built to find out — and the answer, at least for now, is a clear no. Submitted in July 2026 by a team of twelve researchers including Siyi Chen, Jiahe Ying, and He Sun, the study puts benchmarking LLM coding agents at the center of a field where getting the math wrong doesn’t just produce buggy software — it produces scientifically meaningless results.
Summary
Key takeaways
- Imaging-101 is a benchmark of 57 expert-verified computational imaging tasks spanning six scientific domains, each grounded in a peer-reviewed paper.
- Every task follows a standardized four-stage pipeline: preprocessing, forward physics modeling, inverse solver, and visualization.
- Evaluation covers three tracks — planning, function-level unit tests, and end-to-end reconstruction — probing distinct agent capabilities.
- Seven frontier large language models were evaluated, revealing systematic challenges that go beyond what general coding benchmarks expose.
- The study identifies concrete gaps in algorithm selection, physical convention handling, and pipeline integration, and points toward domain-specialized agents as the way forward.
What Imaging-101 Actually Tests
Computational imaging sits at the intersection of physics, mathematics, and software engineering. The core challenge is recovering hidden signals from indirect, noisy measurements — think reconstructing a medical image from sensor readings, or recovering a structure from scattered light. It underpins quantitative discovery across scientific disciplines, yet building a correct reconstruction pipeline demands deep domain expertise. Even experienced domain scientists find it laborious.
Imaging-101 was designed to stress-test whether LLMs can meaningfully assist with that work. The benchmark compiles 57 expert-verified tasks drawn from six scientific domains, with each task grounded in a peer-reviewed paper. That grounding matters: it means the benchmark isn’t measuring abstract coding competence but rather whether an AI agent can translate real, published scientific methods into working code.
Standardized Four-Stage Pipeline
To make tasks comparable across domains, every one of the 57 problems is canonicalized into the same structure. The four-stage pipeline moves through preprocessing, forward physics modeling, inverse solver, and visualization. Each stage carries its own complexity. Forward physics modeling, for instance, requires an agent to encode the physical laws governing how a signal is measured — not just write syntactically correct code, but capture the right equations. The inverse solver stage then asks the agent to mathematically reverse that process.
This pipeline structure is one of Imaging-101’s most deliberate design choices. By standardizing the workflow, the benchmark makes it possible to isolate exactly where an LLM breaks down — whether it fails at understanding the physical setup, struggles with numerical methods, or simply can’t integrate the stages into a coherent end-to-end solution.
How the Evaluation Was Structured
The research team evaluated seven state-of-the-art large language models optimized for coding tasks. Rather than measuring performance with a single metric, the evaluation splits into three distinct tracks designed to probe different agent capabilities.
The first track tests planning — whether an agent can correctly reason about the overall approach before writing any code. The second uses function-level unit tests, isolating individual components of the pipeline to assess granular coding accuracy. The third and most demanding track measures end-to-end reconstruction, requiring the agent to produce a complete, working pipeline that actually recovers a meaningful signal from raw measurements.
This three-track design is analytically smart. A model could perform well on planning — articulating the right strategy — while failing completely when it has to implement that strategy in code. Separating the tracks makes those failures visible rather than averaging them away.
Where LLMs Fall Short on Scientific Imaging Tasks
The results surface a set of systematic challenges that general coding benchmarks simply don’t expose. Three problem areas stand out clearly in the findings.
- Algorithm selection: LLMs struggle to choose the appropriate reconstruction algorithm for a given physical setup, often defaulting to generic or incorrect approaches.
- Physical convention handling: Scientific imaging relies on precise conventions — coordinate systems, unit definitions, sign conventions in equations — and models frequently get these wrong in ways that silently corrupt results.
- Pipeline integration: Even when individual stages are coded correctly, connecting them into a functioning end-to-end system exposes additional failure modes.
What makes these findings significant is that they represent a qualitatively different class of difficulty from general software development tasks. Writing a web scraper or a sorting function doesn’t require an understanding of the physics of wave propagation or the mathematics of Fourier inversion. Computational imaging does. The gap between general coding competence and domain-specific scientific coding turns out to be wider than existing benchmarks suggest.
Why This Gap Matters Beyond Academia
The implications reach further than a single research paper. LLM coding agents are increasingly being positioned as general-purpose scientific assistants — tools that researchers could use to accelerate the implementation of new methods. If those agents systematically fail to handle physical conventions or select inappropriate inverse solvers, deploying them without careful human oversight could introduce hard-to-detect errors into scientific pipelines. The kinds of mistakes that don’t throw exceptions but quietly produce wrong answers.
For fields where computational imaging drives discovery — from medical diagnostics to materials science — that’s a meaningful reliability concern, not a theoretical one.
The Path Forward: Domain-Specialized Agents
The study doesn’t stop at identifying problems. The research team points toward skill-augmented and domain-specialized agents as the practical improvement path. The framing suggests that a general-purpose LLM, no matter how capable at conventional coding tasks, carries structural limitations when applied to physics-grounded scientific workflows. Agents that are specifically equipped with domain knowledge — whether through fine-tuning, retrieval-augmented tools, or structured skill modules — represent the more promising direction.
Imaging-101 itself is positioned as the infrastructure to measure progress along that path. By providing a standardized benchmark with expert-verified tasks and a reproducible evaluation framework, it gives researchers a concrete target for improving agent performance on LLM reconstruction challenges in computational imaging. The benchmark’s grounding in peer-reviewed papers also means it reflects real scientific practice, not synthetic toy problems.
Whether the field moves quickly toward specialized agents or continues relying on general-purpose models with human correction, Imaging-101 now provides the clearest available picture of where the gaps actually lie — and how deep they go.
FAQ
What is the purpose of the Imaging-101 benchmark?
Imaging-101 benchmarks the performance of large language model coding agents on 57 expert-verified computational imaging tasks across six scientific domains. Each task is standardized into a four-stage pipeline, allowing systematic evaluation of where AI agents succeed and fail in scientific imaging workflows.
Which stages compose the computational imaging pipeline in Imaging-101?
The pipeline consists of four stages: preprocessing, forward physics modeling, inverse solver, and visualization. Each stage represents a distinct technical challenge, from encoding physical measurement laws to reconstructing hidden signals from noisy data.
What challenges did LLM coding agents face in the evaluation?
The evaluated models struggled with three main areas: selecting the appropriate algorithm for a given physical setup, correctly handling physical conventions such as coordinate systems and sign definitions, and integrating individual pipeline components into a working end-to-end reconstruction system.
What future improvements are suggested for LLM coding agents in computational imaging?
The study proposes skill-augmented and domain-specialized agents as the practical path forward. Rather than relying on general-purpose models, the researchers suggest that agents equipped with domain-specific knowledge and structured capabilities are better suited to the demands of scientific imaging pipelines.
Article produced with the assistance of artificial intelligence and reviewed by the editorial team.
