WebAnything: From the Wild Web to Scalable Agent Environments
Web agents need more than screenshots. To train and evaluate reliably, they need websites they can open, manipulate, reset, and revisit under controlled conditions. Live websites provide rich and realistic workflows, but their changing data, external dependencies, authentication requirements, and side effects make them difficult to use as repeatable research environments. Manually mirroring each site is expensive, while purely generative approaches may produce functional pages that no longer correspond closely to the specified source.
The authors of WebAnything propose a different route: automatically reconstructing local, executable environments from authorized real-world URLs. Their central idea is to treat browser observations as structured evidence rather than relying on screenshots or model assumptions alone. The system is designed to preserve observed page structure, resources, interactions, and state changes within a declared site contract.
From browser evidence to executable environments
WebAnything uses an Explorer–Builder pipeline. The Explorer visits a source website and records browser-observable information, including page structure, computed styles, element geometry, screenshots, resource references, and before-and-after outcomes for actions. These observations distinguish states that share a URL but differ in behavior, such as a closed menu and an expanded menu.
The Builder uses that evidence to implement local routes, components, resources, and state transitions. Structural evidence constrains layout and relationships between elements; resource evidence grounds fonts, images, and media; behavioral evidence informs navigation, filtering, forms, persistence, and other state changes. When the available evidence does not establish a required outcome, the system is supposed to request targeted recapture instead of presenting an inferred behavior as observed.
The framework supports two exploration policies. Full Coverage expands through accessible page types, interaction states, and nearby navigation branches. Just-in-Time Coverage focuses on the dependencies of a specified task set. The construction experiments reported by the authors use Full Coverage.
Why independent verification matters
A notable part of the method is the separation between construction and checking. After the Builder completes a candidate, an Independent Verifier examines a frozen version in a separate context. It cannot edit the candidate while testing it, and it compares fresh observations with source evidence and the site contract. Checks can cover pages, controls, resources, workflows, persistence, and state restoration.
The resulting feedback is routed in one of two ways. If existing evidence proves that the candidate is wrong, the system sends a local repair request describing the affected control or page, the expected outcome, the observed outcome, and the supporting evidence. If the evidence is insufficient, it creates a bounded recapture request. The revised candidate is frozen and checked again before release.
This distinction is important because a failed check can have two very different causes: an implementation defect or a gap in what was observed from the source. The authors present examples involving Figma, Ctrip, and Stashmedia to illustrate repairs, runtime problems, and discrepancies that Builder self-checks missed.
Scale, fidelity, and functional trade-offs
The reported resource pool contains 244 website artifacts spanning 14 application domains and 53 categories. The authors also report 12,984 executed and accepted tasks. In an audited historical collection of 85 sites, they report no recorded per-site manual implementation, debugging, or repair after initial setup and authorization. Recorded model-call costs for 84 sites totalled $220.62, but the paper explicitly describes this as a lower bound that excludes unpriced calls, browser execution, and other system overhead. It is not extrapolated to the full artifact pool.
On five sites and 14 historical states matched against WebHarbor, WebAnything achieved higher aggregate Pixel Similarity, SSIM, and Text F1 scores. The reported values were 0.9240, 0.7904, and 0.8541 for WebAnything, compared with 0.8083, 0.6260, and 0.2617 for WebHarbor. A blind audit preferred WebAnything in 12 cases, tied in two, and recorded no losses for overall fidelity.
Those results do not mean that WebAnything was more functional in every comparison. On the WebHarbor interaction subset, WebHarbor had higher capability coverage and all-test pass rates: 80% versus 75%, and 76.7% versus 63.3%. On a separate five-site VeriEnv subset, selected WebAnything artifacts passed 29 of 33 tests, compared with 21 of 33 for VeriEnv. The contrasting results support the authors’ warning that visual fidelity and interaction validity must be measured separately.
Evidence for downstream training value
The authors fine-tuned Qwen3.5-9B on 5,856 WebAnything interaction trajectories, represented as 60,517 step-level examples. On 91 jointly valid REAL tasks, success rose from 22/91 to 31/91, a 9.9 percentage-point increase. On 44 jointly valid Odysseys-Easy tasks, Rubric Average increased by 6.25 percentage points and Task Average by 6.32 points.
WebArena-Verified showed a 13.33-point increase in Overall success under the fixed harness, from 16.41% to 29.74%. Gains were especially large for Navigate and Mutate tasks, while Retrieve declined under strict scoring. A supplementary rescoring that ignored JSON-format judgments changed the Retrieve result, but that was a scoring change rather than a new execution result. The authors therefore present the SFT findings as evidence of utility, not as proof that the reconstruction process alone caused every improvement.
Limitations and practical implications
The evidence remains bounded. Quality comparisons use small, selected subsets, historical references have rendering and JavaScript inconsistencies, and the 244-site inventory is not a preregistered random sample. The cost analysis is incomplete, and the authors do not report a unified construction success rate. Training validation centres on a 9B model and does not systematically test larger models, other post-training methods, or larger budgets.
Maintenance is another unresolved issue. Real websites continually change, while WebAnything reconstructs and verifies states at particular times. Long-term synchronization, incremental recapture, and update costs are left for future work. The authors also stress that public accessibility does not automatically permit copying or redistributing content, branding, images, fonts, or user data.
For researchers, the practical lesson is to build agent environments around explicit evidence, frozen versions, reset mechanisms, and independent evaluation. For engineering teams, the framework suggests a way to generate broader training data without hand-building every site, provided that authorization, privacy, sandboxing, and side-effect controls are treated as first-class requirements. WebAnything demonstrates a promising path toward scalable web-agent infrastructure, while its own results show why realistic environments must be judged on both what agents can see and what they can reliably do.
Sources
AI-assisted research summary. Check the original paper for complete methods, results, and limitations.