WorkFactory: Turning Professional Intents into Executable Work for Scalable Agent Training
Training an AI agent to perform useful professional work requires more than generating a large collection of prompts. The agent must receive enough evidence to complete the assignment, while still having to perform the calculations, tool use, and judgments that make the task meaningful. If a spreadsheet omits a critical pricing rule or two reference files disagree, a teacher model may produce an unreliable demonstration. If the environment includes the answer directly, the task may become a shortcut rather than useful supervision.
In WorkFactory: Turning Professional Intents into Executable Work for Scalable Agent Training, the authors propose a framework for constructing work environments around this boundary. The system starts with a professional intent, such as evaluating suppliers and recommending one, then works backward from the desired deliverable to identify the information required to produce it.
Planning tasks around information dependencies
The central device is an information support graph. Rather than treating a workspace as a loose collection of files, the graph represents information units, the relationships among them, and the locations where they should appear. It distinguishes supplied facts and rules from values the agent is expected to derive.
In the paper’s supplier-selection example, demand forecasts, vendor quotations, freight terms, and discount rules are visible inputs. Annual spending, discounted lifecycle costs, and the final recommendation remain solver-derived outputs. This distinction helps the construction process provide the premises without doing the substantive work in advance.
The graph is translated into a construction plan covering the user request, reference artifacts, deliverables, and dependencies among them. An environment worker then reuses, adapts, or synthesizes materials. Automated and language-model-based checks assess whether the resulting workspace matches the plan, but the authors note that these checks cannot guarantee every semantic dependency survived materialization.
Repair through blind execution
WorkFactory adds a second quality-control stage: execution-based environment repair. An isolated probe agent attempts the task using only the request and solver-visible resources. It cannot see the support graph, construction plan, or earlier diagnoses. If it identifies a blocking problem, a refiner edits the request or existing reference files, then the revised environment is checked and probed again.
The repair process is constrained to preserve the intended work. For example, if a quotation does not say whether freight is included, the system can add a freight charge so that costs are comparable. It should not add the completed cost comparison or recommendation, because that would remove the work the training agent is supposed to perform. Tasks with unresolved blockers are discarded.
Scale and reported results
The authors constructed 112,686 tasks spanning 612 occupations, 23 major occupational groups, 53 deliverable file extensions, and 317 reference-file extensions occurring at least 10 times. The average task included 4.15 reference files. Teacher trajectories averaged 57.36 assistant turns and 62.95 tool calls, indicating that the corpus targets extended workflows rather than short question-answer exchanges.
Using these trajectories, the authors fine-tuned Qwen models at approximately 4B, 27B, and 35B-A3B scales to create the WorkPilot family. Each WorkPilot model outperformed its corresponding base model on four professional-work benchmarks. For the 35B-A3B model, GDPval-AA increased from 879 to 1565 Elo, JobBench from 28.91 to 62.35, Workspace-Bench from 47.12 to 56.53, and OmegaUse-OfficeVal from 7.14 to 21.18. WorkPilot-27B reached a locally estimated 1612 GDPval-AA Elo.
These results demonstrate that the collected trajectories were useful for supervised fine-tuning in the tested settings. They do not establish that WorkFactory alone explains every improvement over unrelated systems, since the strongest primary comparison is between each base model and its fine-tuned counterpart.
What the ablations reveal
A controlled 5,000-task comparison supports the framework’s design choices. Replacing direct artifact-level planning with information-level planning reduced several probe-diagnosed defects, including broken derivations and underspecified requests. However, some categories, including leakage or shortcuts and self-inconsistency, initially increased. This is an important qualification: making dependencies explicit does not automatically make generated files coherent.
Adding execution-based repair reduced all eight reported defect categories relative to information-level planning alone. Cross-file inconsistency fell from 25.74% to 4.84%, while execution-feasibility defects fell from 6.64% to 0.48%. Downstream GDPval-AA rose from 1245 with artifact-level planning to 1293 with the support graph and 1326 after repair. The authors interpret these results as evidence that planning and runtime diagnosis are complementary.
Scaling, costs, and limitations
More data helped, but with diminishing returns. For the 35B-A3B model, the first 5,000 tasks contributed 447 GDPval-AA points, while expanding the dataset from 75,000 to 110,000 tasks added 13 points. Execution length first increased and later decreased as training data grew, while some tool-call failure rates declined. The authors caution that these patterns are model-dependent rather than universal scaling laws.
The paper’s cost analysis places the three WorkPilot models on the reported non-dominated cost-performance frontier, with estimated task-solving costs of approximately $0.012, $0.031, and $0.396 for the 4B, 35B-A3B, and 27B models respectively. These are standardized estimates, not provider invoices: they exclude judges, tools, web fees, sandbox compute, storage, and orchestration.
There are other meaningful limitations. Training and evaluation use text-only task-solving interfaces because the teacher model does not accept visual input. The study focuses on supervised fine-tuning, not reinforcement learning from execution feedback. The tasks are constructed environments, so success on these benchmarks may not predict performance amid the ambiguity, privacy constraints, and changing systems of real workplaces. Teacher choice also mattered: on a matched 30,000-task comparison, students trained on GLM trajectories scored higher than those trained on DeepSeek trajectories.
Practical implications
For teams building agent-training data, WorkFactory suggests that environment design deserves the same rigor as prompt design. A useful pipeline should map each deliverable to visible evidence, preserve the calculations and judgments that define the task, and test the workspace with an independent execution before collecting demonstrations. Repair logs can also become a valuable evaluation signal, revealing whether failures arise from the agent or from the environment.
The broader lesson is pragmatic: scaling professional agents is not simply a matter of adding more examples. It requires coherent, auditable workspaces in which the agent has what it needs to work, but not the answer it is meant to discover.
Sources
AI-assisted research summary. Check the original paper for complete methods, results, and limitations.