What If You Could Simulate the Internet Before Interacting With It?

Key takeaway: Qwen-AgentWorld is the first family of language world models capable of simulating agent environments across seven domains — MCP, Search, Terminal, Software Engineering, Android, Web, and OS — through a three-stage training pipeline (CPT → SFT → RL). The 397B-parameter model achieves 58.71 on AgentWorldBench, outperforming GPT-5.4 (58.25), and enables agents trained in fictional environments to surpass those trained in real ones.

The problem: agents that can't predict what happens next

A world model predicts how the environment will change in response to an action. In reinforcement learning, this is fundamental: an agent that can anticipate the consequences of its actions can plan, avoid mistakes, and explore more efficiently. AlphaGo doesn't just evaluate board positions — it simulates possible futures. DreamerV3 masters diverse control tasks by learning to predict the next state before deciding what to do.

Now consider today's LLM agents. They interact with terminals, search engines, web browsers, and software repositories. They issue tool calls, run commands, and navigate complex digital environments. But unlike AlphaGo or Dreamer, they lack a predictive model of their environment. They don't simulate what would happen if they ran curl on a misconfigured server, or anticipate the error message they'd get after deleting a nonexistent file. They operate without foresight.

This is a strange limitation. The environments agents interact with are deterministic, rule-governed, and often highly structured. The terminal's behavior is specified by POSIX standards. Search engines follow consistent retrieval and ranking algorithms. Web browsers render according to HTML and CSS specifications. These are not chaotic physical systems — they are systems of rules that could, in principle, be modeled.

The Qwen-AgentWorld paper asks: can we build a single foundation model that simulates all these environments, predicting the next observation given the current state and action? And if we can, what can agents do with that simulation?

What a language world model is

A language world model (LWM) is a conditional text generator that predicts the next environment observation given the interaction history and the agent's current action. Formally:

ô(t+1) = f_θ(c, o≤t, a≤t)

Where c is the system prompt (task description, action space, initial state, demonstrations, and simulation instruction), o_t is the observation at turn t, and a_t is the agent's action. The LWM learns to generate the ground-truth observation o_(t+1).

This is different from a general-purpose LLM being asked to role-play as an environment. The LWM is trained end-to-end with environment modeling as the explicit objective, starting from continual pre-training. It doesn't just infer what a terminal output might look like — it learns the state transition dynamics of each domain.

The seven domains

Qwen-AgentWorld covers seven domains with diverse observation and action representations:

For GUI domains, observations are represented as accessibility trees and UI view hierarchies rather than pixel frames. This keeps the problem text-based while preserving the structural information needed for state prediction.

The three-stage pipeline

The training recipe follows a principle: "CPT injects, SFT activates, RL sharpens."

Stage 1: Continual pre-training (CPT)

CPT injects environment world knowledge through next-token prediction. The model sees multi-turn environment trajectories framed as world-modeling tasks: the system prompt defines the simulation context, user turns carry the agent's actions, and assistant turns carry the environment's responses.

The key innovation here is information-theoretic loss masking. Many turns in tool-use trajectories are boilerplate — tools that echo input, APIs that mirror request parameters. The gradients from these turns are low-quality. The paper computes four statistics per (action, observation) pair — Overlap, Novelty, Jaccard similarity, and length ratio — and assigns each turn to one of seven semantic categories with different keep ratios. Turns that carry genuine world knowledge (e.g., read_file returning file contents) have a 100% keep ratio. Boilerplate turns (API echoes) drop to 5–10%. This decouples "learning the next state" from "learning the next token."

Stage 2: Supervised fine-tuning (SFT)

The SFT data is curated through rejection sampling. Each query generates three rollouts from a general-purpose reasoning model. An independent judge scores them and selects the highest-quality trajectory. Starting from 10,250 candidate queries, the pipeline retains 7,094 trajectories (69.2% retention rate).

Stage 3: Reinforcement learning (RL)

Five-dimensional rubric (LLM judge). Each predicted observation is scored on Format, Factuality, Consistency, Realism, and Quality — each on a 1–5 scale. The judge receives the ground-truth observation alongside the prediction, making the evaluation a factual comparison rather than an open-ended quality judgment.

Rule-based verifier. A subset of the data carries executable verifier code that produces a binary correctness signal. This provides an objective anchor, mitigating reward hacking from the open-ended rubric.

The two signals are combined at a 9:1 ratio (rubric:rule). The paper identifies three critical failure modes in RL training:

1. Reward collapse from multi-turn expansion. When training trajectories are expanded into multiple samples, they share long common prefixes, causing training to collapse. The fix: restrict expansion to exactly one turn per trajectory in the RL pool.
2. Reward shaping. Binary reference-reward and Turing-test reward converge slowly or barely converge. The five-dimensional rubric provides consistent, informative gradients.
3. Reward hacking through self-praise. The policy learns to embed self-praising phrases to inflate scores. The rule-based verifier, content-type classification, and strict tag extraction prevent this.

AgentWorldBench: evaluating world models

Construction principles

1. Widely-used queries. All task queries are drawn from established high-quality agentic benchmarks rather than self-constructed tasks.
2. Frontier-agent trajectories. All trajectories are generated by frontier-model agents whose actions are complex enough to stress-test world-model fidelity.
3. Real observations. Every trajectory is paired with ground-truth observations from real environment execution.
4. Out-of-distribution. Training data and benchmark queries are partitioned at the data-source level.

The evaluation protocol

An LLM judge scores each predicted observation on five dimensions, scaled to [0, 100]:

• Format: does the output obey structural conventions?
• Factuality: are stated facts correct?
• Consistency: is the output internally coherent, and coherent with prior turns?
• Realism: does the simulation match the behavioral characteristics of the real environment?
• Quality: is the output complete and appropriately concise?

The judge receives the ground-truth observation alongside the prediction. This reference-grounded design converts evaluation from an open-ended quality judgment into a factual comparison. Differentiated matching criteria handle deterministic content (must match exactly), pre-existing environment content (format and plausibility only), and runtime metadata (format and range verification).

The results

Qwen-AgentWorld-397B-A17B achieves 58.71 on AgentWorldBench, surpassing GPT-5.4 (58.25) and all other frontier models. On text-based domains, the advantage is most pronounced on Terminal (57.73 vs. 53.69) and SWE (68.49 vs. 66.29) — the two domains requiring accurate modeling of code execution state and tool API behavior.

The effect of world-model training is substantial. At the 397B scale, the overall average rises from 54.74 to 58.71. At 35B, the gain is 8.66 points (47.73 to 56.39), lifting Qwen-AgentWorld-35B-A3B above Claude Sonnet 4.6 (56.04). The improvement is consistent across both text and GUI domains, confirming that the gains are not explained by the base model's general capability alone.

Cross-domain generalization

When Stage 3 (RL) is trained on Terminal data alone, Terminal improves by +14.2 points within 100 steps. But all three held-out text-based domains improve in parallel: SWE gains +11.5, Search gains +11.8, and MCP gains +5.0.

This is non-trivial transfer. Terminal shell commands and MCP tool calls differ in syntax, state representation, and response structure. Yet the gains emerge within the first 10 RL steps and remain stable. The pattern suggests that RL reinforces generalizable world knowledge — how environments respond to actions, how errors propagate, how state transitions compose across turns — rather than domain-specific output formats.

Application I: environment simulator

Controllability

Controllable simulation uses natural-language instructions to shape the simulator's behavior. This is where the results become striking.

MCP: environment adaptation. Sim RL without control instructions provides no gain (Tool Decathlon actually drops from 32.4 to 31.5). With controllable simulation — injecting targeted perturbations like intermittent API errors and paginated responses — Tool Decathlon improves by +3.7 and MCPMark by +12.3. Controllability isn't just a factor in the magnitude of improvement; it's a prerequisite for Sim RL to work at all in this domain.

Search: fictional-world construction. Qwen-AgentWorld generates entirely fictional, self-consistent worlds from a compact initial specification. Agents trained in these fictional environments — where every search result is invented — generalize to real-world search tasks. On WideSearch, controllable Sim RL raises F1 by Item from 34.02 to 50.31 (+16.29) on the 35B model and from 70.11 to 73.98 (+3.87) on the 397B model.

Fictional-world construction offers two structural advantages. First, since the answers exist only within the fictional setting, the agent cannot bypass the search tool by answering from parametric memory. Second, because all facts are invented, the agent cannot confuse training-time search results with real-world knowledge.

Controllable Sim RL surpasses real RL

Figure 9 in the paper compares controllable Sim RL against Real RL (trained against a live search engine) on WideSearch. Sim RL reaches 50.3% F1 by Item, compared to 45.6% for Real RL. The more informative signal comes from agent behavior: both training regimes reduce web_search calls, but Sim RL increases web_extractor calls while Real RL decreases them. Because the simulated search snippets deliberately withhold detailed content, the Sim-RL-trained agent learns that extracting full pages is necessary for assembling complete answers. The Real-RL-trained agent finds that real search snippets often contain sufficient information and skips extraction.

The controllable simulation design shapes targeted agent behavior. By constructing adversarial environment conditions where specific capabilities are required, Sim RL trains those capabilities more effectively than uncontrolled real-environment training.

Application II: agent foundation model

The results are striking. LWM RL warm-up on single-turn, non-agentic trajectories transfers to multi-turn, tool-calling agentic tasks across seven benchmarks — including three out-of-domain benchmarks absent from LWM training. Terminal-Bench 2.0 improves from 33.25 to 39.55 (+6.30). SWE-Bench Verified from 64.5 to 67.9 (+3.4). SWE-Bench Pro from 42.2 to 47.4 (+5.2).

The out-of-domain gains are particularly compelling: Claw-Eval improves from 53.6 to 64.9 (+11.3), QwenClawBench from 39.8 to 49.4 (+9.7), and BFCL v4 from 62.3 to 71.3 (+9.0). The LWM training pipeline contains no Claw or function-calling data whatsoever — yet gains of +11.3, +9.7, and +9.0 emerge on domains entirely absent from world-model training.

Prediction accuracy

The paper quantifies the mechanism behind these gains. On Terminal-Bench 2.0 trajectories, RL training improves the model's explicit prediction accuracy from 69.9% to 78.3% (+8.4%). The model's thinking traces show systematic mental simulation of environment responses before executing actions, using the internalized world model to predict outcomes, identify infeasible approaches, and refine action plans — all within the thinking trace and before any real execution.

The case study on the mailman task (Figure 11) illustrates this. Both models encounter the same Postfix recipient-rejection error. The model after LWM RL correctly predicts that configuring transport_maps alone will not work, because Postfix rejects unknown recipients before consulting transport routing. It refines its action toward modifying local_recipient_maps. The model before LWM RL incorrectly predicts that transport routing precedes recipient validation, leading to futile exploration.

What the model actually learned: reasoning patterns

Deliberative self-correction

The model uses "Wait!" as an explicit cognitive interrupt to re-examine an intermediate prediction and revise before committing. Across 129 turns, the paper counts 1,347 such interrupts (10.4 per turn on average; peak: 56 in a single SWE turn). These self-corrections decompose into three functional subtypes: factual (catching incorrect API response formats), epistemological (recognizing the limit of in-context computation), and perspective-taking (modeling the evaluator's intent).

Information leakage prevention

In the Search domain, the model holds a reference answer that the agent is trying to find. When the agent's query is unrelated to this answer, the model explicitly prevents leakage: it identifies the topic mismatch and ensures that generated snippets do not accidentally reveal the target information. This is the world-model equivalent of theory of mind — the model distinguishes what the agent knows from what the environment should reveal.

The limitations

The paper is transparent about several limitations.

GUI domains still lag. The gap between Qwen-AgentWorld and top frontier models is largest on GUI domains — Android, Web, and OS. The authors attribute this to multimodal pre-training that text-only world modeling does not capture. GUI observations are represented as accessibility trees, which are text-based but derive from visual layouts. Future work on multimodal extension could close this gap.

Search remains the hardest domain. The best score on Search (37.82) is roughly half the best score on SWE (68.49) or MCP (70.10). Search requires modeling constantly evolving web content, and factual consistency across long retrieval chains remains difficult for all models.

Training is expensive. Running the full three-stage pipeline requires substantial compute. The RL stage alone involves on-policy rollouts with up to 128k-token prompts. The paper reports cost estimates in Appendix D.2.

What this means for agent development

The Qwen-AgentWorld paper opens a new axis for scaling agent training: not by collecting more real-world trajectories, but by simulating environments at scale and with controllability.

The fictional-world construction result is the most striking. Agents trained entirely in invented environments — where every search result, web page, and factual record is generated by a world model — generalize effectively to real-world search tasks. This is counterintuitive. You might expect that training on fictional data would produce agents that are confused by real-world patterns. Instead, the agents learn generalizable capabilities: query reformulation, multi-source cross-referencing, and iterative result aggregation.

The implication is that world modeling might enable agent training at a scale that real-world interaction cannot provide. Real environments are bounded by infrastructure, cost, and safety. Simulated environments can be instantiated on demand, perturbed systematically, and generated in unlimited quantities — provided the world model is sufficiently faithful.

This is the core insight of the paper, stated at the beginning:

LWM enables turn-level scaling of diverse environments without requiring dedicated infrastructure, spanning extreme scenarios, real-world tasks, and high-value professional domains where real execution is infeasible due to irreversible operations, proprietary deployments, or the absence of public implementations.

The agent foundation model results are equally significant. LWM warm-up improves downstream agent performance across seven diverse benchmarks, including domains absent from LWM training. The mechanism — prediction-driven action refinement — is transferable. An agent that can predict what will happen before acting is an agent that can avoid mistakes, explore more efficiently, and refine its plans.

Questions for further research

• Can language world models close the GUI gap? The paper suggests multimodal extension — fusing GUI screenshots with text-based state representations — as a natural next step. But is that sufficient, or does GUI simulation require fundamentally different approaches?
• How far can controllable simulation go? The paper shows that agents trained in fictional search environments generalize to real tasks. Could we train agents entirely in fictional worlds for web navigation, terminal operations, or software engineering — and have them transfer effectively?
• Can the world model improve the agent, and the agent improve the world model, in a virtuous cycle? The paper mentions agent-LWM co-evolution as a future direction: self-play where the agent discovers novel states that push the world model's boundaries, and the world model generates increasingly challenging scenarios for the agent.
• What happens when world models are deployed as inference-time simulators? The paper shows that LWM warm-up improves agent performance, but what about using the world model at test time — simulating multiple action candidates and selecting the best one through planning?

Source: Qwen Team (2026). Qwen-AgentWorld: Language World Models for General Agents. Qwen Technical Report.