EditAgent: Rethinking Agentic Image Editing
Complex image-editing requests expose a weakness in many agentic systems. A user may ask an editor to move a subject, widen the frame, remove background objects, and change the lighting in one instruction. Existing agents typically decompose that request into steps, run an image editor, inspect the result, and retry when something goes wrong. The retry often uses the same editor and evaluates one output at a time.
Haiyang Xu and colleagues at UC San Diego and Adobe Research argue that this design is constrained in two ways: by the agent’s action space, or what it can do, and by the judge’s observation space, or what it can reliably assess. Their proposed system, EditAgent, expands both rather than simply repeating the same operation.
Why one general editor is not enough
The authors first build a diagnostic benchmark containing 600 candidate sets covering 30 edit types. Five professional Photoshop editors label which candidates are usable, rather than merely selecting a single favorite. The candidates combine outputs from a general-purpose editor with results from specialized methods.
In this benchmark, the general editor fails to produce a usable result in 42.7% of sets. Specialized methods recover 175 of those 256 failures. The gap is especially pronounced for geometry and enhancement tasks: operations such as rotation, zooming, sharpening, or restoration often benefit from tools designed for those specific transformations.
Sampling the general editor more often does not solve the problem. A pool restricted to that editor delivers a usable result in 45.2% of sets, compared with 68.2% when specialized methods are included. This supports the authors’ central claim that broader tool selection can matter more than additional samples from one model.
Parallel candidates and richer evidence
EditAgent routes each planned step to a capability pool. Depending on the request, that pool can include masked local editing, object removal or movement, photometric operators, style filters, geometric tools, cropping, and canvas extension, alongside the general editor.
Rather than repeatedly editing the latest output, the system generates candidates in parallel from the same source image. This preserves a clean reference point and reduces the risk that an early artifact will propagate through later iterations. An off-the-shelf vision-language model then compares the candidates in one listwise judgment.
The judge also receives a verification record for every candidate. Records can include a change map, a same-scale close-up of the edited region, measured brightness or color statistics, geometric information, and programmatic checks. Code may verify, for example, that pixels outside a mask remain unchanged or that a flip is an exact transformation. Visual questions such as realism, seam quality, and whether the requested object actually looks correct remain for the VLM.
This division is important because the correct amount of change depends on the task. A source-similarity metric may be useful for a subtle enhancement but harmful for a request that requires a visible rotation or style change.
What the evaluations show
In a blind comparison involving 144 Complex-Edit samples, professional editors preferred parallel search over sequential refinement in 102 cases, versus 28 for sequential refinement, with 14 ties. Among decided comparisons, parallel search was preferred in 78.5% of cases and used no more than half as many VLM calls under the tested setup.
The judging experiments also favor comparative evaluation with evidence. On the authors’ diagnostic benchmark, a Qwen3.8-27B judge reaches 71.6% Winner-Set Hit@1 when evaluating candidates pointwise. Listwise comparison raises this to 73.6%, while adding verification records raises it to 81.3%. Across seven off-the-shelf VLM judges, evidence improves accuracy by 6.3 to 9.3 percentage points and allows every tested model to outperform the finetuned reward models in that comparison.
EditAgent improves two open-weight editing backbones, Qwen-Image-Edit-2511 and FLUX.2-klein-9B, on EdiVal, Complex-Edit, and ComplexBench-Edit. On EdiVal’s third-turn output score, the reported results rise from 47.64 to 55.46 for Qwen-Image-Edit-2511 and from 53.20 to 59.45 for FLUX.2-klein-9B. The system also performs well against the agentic baselines tested with the same base editors and VLM, achieving the strongest third-turn score for both backbones in the reported comparison.
A data engine, not just an editor
The authors use EditAgent’s accepted sessions as filtered training data. From roughly 25.7 thousand training sessions, they report that models trained on EditAgent outputs outperform models trained on images generated directly by FLUX.2-klein-9B under the same general training comparison. For Qwen-Image-Edit-2511, the third-turn EdiVal-O score increases from 49.66 without finetuning to 51.61 with EditAgent data; for FLUX.2-klein-base-9B, it increases from 54.53 to 56.53.
These results suggest a practical role beyond inference: a slower agent can produce higher-quality supervision for a faster standalone editor. That conclusion is supported by the reported experiments, although it should not be read as proof that every generated dataset will transfer equally well.
Limitations and practical implications
EditAgent is substantially more expensive than standalone editing. The authors report roughly ten times the inference time, and performance remains bounded by the underlying editing methods. A planner or judge cannot recover an edit that no available tool can produce. The benchmark’s expert labels and the chosen VLM evaluators also define what counts as success, so results may vary with other image domains, judges, or task distributions.
For developers, the paper offers a useful design principle: when an image agent fails, do not assume it needs another identical attempt. Add tools suited to different edit types, preserve the original source for candidate generation, compare alternatives together, and expose machine-checkable evidence to the evaluator. For production systems, the trade-off is clear. Parallel search and verification may improve reliability for high-value edits, while accepted trajectories can potentially amortize their cost by training more capable and efficient editors later.
Sources
AI-assisted research summary. Check the original paper for complete methods, results, and limitations.