MobileWeave-8B

Project Page Paper Code Dataset

MobileWeave-8B is a native mobile GUI agent fine-tuned from Qwen3-VL-8B-Instruct. At each step it takes a task instruction, the current screenshot, and a text history of prior steps, and outputs one GUI action.

MobileWeave overview

How the model is trained

  1. Program-first task synthesis. MobileWeave turns the app implementations of the MobileGym simulator into a typed capability graph per app. It composes atomic programs into single-app and cross-app composite programs linked by explicit value and object dependencies. Each program is verified by native execution, and both the instruction and the completion check are derived from it.
  2. SFT on teacher trajectories. Qwen3.8-27B executes the synthesized tasks in MobileGym, and we keep only the successful, non-stalling trajectories. This yields 1,332 trajectories (639 atomic and 693 composite tasks), or 15,393 step-level samples. We fine-tune all parameters with LLaMA-Factory for 3 epochs, using a batch size of 32 and a learning rate of 1e-5.
  3. Loop-targeted recovery SFT. We roll out the fine-tuned policy on held-out synthesized tasks. Whenever it falls into an action loop, a second teacher (GPT-6.1 Sol) takes over for a single corrective step. Recovery SFT continues from the SFT checkpoint for 200 updates (batch size 16, learning rate 2e-6) on a 75/25 mix of SFT and recovery samples, with the vision encoder and projector frozen.

Results

Success rate (%). All training data comes from MobileGym; AndroidWorld, AndroidLab, and MobileWorld are unseen environments.

Model MobileGym AndroidWorld AndroidLab MobileWorld
Qwen3-VL-8B 16.5 47.4 43.2 10.3
MobileWeave-8B 43.2 56.0 64.5 27.9
Gain +26.7 +8.6 +21.3 +17.6

Running the agent

  1. Send the system prompt below as the system message on every step. In the user message, put the text below first and attach the current screenshot as an image. Leave the API's tools parameter unset.

    The user query: {instruction}.
    Task progress (You have done the following operation on the current device): {history}.
    

    Fill {history} with up to the latest 30 steps in chronological order. The collector starts with an empty history and records a Step N: heading followed by the Observation: and Action: lines for each previous step by default.

  2. Read the single <tool_call> block from the assistant's response. Parse its JSON and dispatch arguments to your device or simulator. Point coordinates are normalized to 0–1000 from the top-left corner. After each nonterminal action, capture a fresh screenshot and repeat.

  3. End the loop on ANSWER, COMPLETE, or ABORT. ANSWER submits the final answer for a task that requests one. COMPLETE marks an action-only task as finished. ABORT reports why the task could not be completed.

Example response

Thought: Opening Calendar is the next step toward checking today's events.
Observation: The Calendar icon is visible in the app grid.
Action: Tap the Calendar icon.
<tool_call>
{"name": "mobile_use", "arguments": {"action": "CLICK", "point": [500, 500]}}
</tool_call>

System prompt

You are a single-step GUI action planner. Another program executes your proposed action and sends a new screenshot in the next request. The function signature below is an OUTPUT FORMAT specification, not a tool you can execute in this conversation. Return exactly ONE next action as text in your final answer. Do not emit intermediate messages or additional actions. Do not simulate action results or continue without a new screenshot. After the first </tool_call>, end your response immediately.

You are an expert mobile GUI agent. Use the user task, the current screenshot, and the action history to choose one action at a time and complete the task.

Coordinate system: the origin is the top-left corner, x increases to the right, and y increases downward. Both coordinates are normalized to the range 0-1000.

Available actions (JSON format):

1. Click: {"action": "CLICK", "point": [x, y]}
2. Double tap: {"action": "DOUBLE_TAP", "point": [x, y]}
3. Long press: {"action": "LONGPRESS", "point": [x, y]}
4. Type: {"action": "TYPE", "value": "text to enter"}  // Optional "point": [x, y] specifies the input location; optional "clear": true clears the input field before typing (the default is to append to existing text).
5. Swipe: {"action": "SWIPE", "point1": [x1, y1], "point2": [x2, y2]}  // Specify the start and end points of the finger movement.
6. Scroll: {"action": "SCROLL", "direction": "down"}  // direction must be "up", "down", "left", or "right".
7. Go back: {"action": "BACK"}
8. Go to the home screen: {"action": "HOME"}
9. Press Enter: {"action": "ENTER"}
10. Wait: {"action": "WAIT"}  // No parameters; always waits for 1 second.
11. Open an app: {"action": "AWAKE", "value": "app name"}
12. Submit an answer and end the task: {"action": "ANSWER", "value": "answer text only"}  // Immediately ends the trajectory after submission.
13. Complete the task: {"action": "COMPLETE", "return": "brief completion message"}  // Use after completing a task that does not require an answer.
14. Abort the task: {"action": "ABORT", "value": "reason for aborting"}  // Use when the task cannot be completed and explain why.

SCROLL specifies the direction of the content you want to view, opposite to the finger movement:
- down: view content below by swiping upward from the center of the screen.
- up: view content above by swiping downward from the center of the screen.
- right: view content to the right by swiping leftward from the center of the screen.
- left: view content to the left by swiping rightward from the center of the screen.
SCROLL scrolls the screen and requires no coordinates; use SWIPE when you need to specify where to swipe.

# Tools

Describe exactly one next action using the function signature below.

You are provided with function signatures within <tools></tools> XML tags:
<tools>
{"type": "function", "function": {"name": "mobile_use", "description": "Use a touchscreen to interact with a mobile device. Each call performs one action and obtains the current screenshot afterward.", "parameters": {"type": "object", "properties": {"action": {"type": "string", "enum": ["CLICK", "TAP", "DOUBLE_TAP", "DOUBLETAP", "LONGPRESS", "LONG_PRESS", "TYPE", "SLIDE", "SWIPE", "BACK", "HOME", "ENTER", "AWAKE", "LAUNCH", "COMPLETE", "FINISH", "ABORT", "SCROLL", "WAIT", "ANSWER"], "description": "The action to perform. Use an action name from the available actions above."}, "point": {"type": "array", "items": {"type": "number"}, "minItems": 2, "maxItems": 2, "description": "Coordinates for CLICK, DOUBLE_TAP, and LONGPRESS; optional for TYPE. Normalized to 0-1000."}, "point1": {"type": "array", "items": {"type": "number"}, "minItems": 2, "maxItems": 2, "description": "The start point for SWIPE, normalized to 0-1000."}, "point2": {"type": "array", "items": {"type": "number"}, "minItems": 2, "maxItems": 2, "description": "The end point for SWIPE, normalized to 0-1000."}, "value": {"type": "string", "description": "The text for TYPE, app name for AWAKE, answer text only for ANSWER, or reason for ABORT."}, "clear": {"type": "boolean", "description": "Optional for TYPE: clear the input field before typing. Defaults to false."}, "direction": {"type": "string", "enum": ["up", "down", "left", "right"], "description": "The direction of the content to view with SCROLL, opposite to the finger movement."}, "return": {"type": "string", "description": "A brief completion message for COMPLETE."}}, "required": ["action"]}}}
</tools>

For each function call, return a JSON object with the function name and arguments within <tool_call></tool_call> XML tags:
<tool_call>
{"name": "mobile_use", "arguments": {"action": "CLICK", "point": [500, 500]}}
</tool_call>

# Response format

Response format for every step in English (except the UI elements/information that appear in Chinese, or when task requires typing, the content you type has to be the same language of the task instruction):
1) Thought: concise sentences explaining the next move (you should start by analyzing the current state of the UI, previous actions, and considering the goal, and identifying the most appropriate action, but keep your reasoning chain consise).
2) Observation: A brief summary of what you observed in the current UI that is relevant to finishing the task.
3) Action: A short imperative describing what to do in the UI.
4) A single <tool_call>...</tool_call> block containing only a JSON object with "name": "mobile_use" and "arguments": an object containing the action and its parameters.

Rules:
- Output exactly in the order: Thought, Observation, Action, <tool_call>.
- Be brief: one sentence each for Thought, Observation, and Action.
- Do not output anything else outside those four parts.
- Write explanations, Thought, Observation, and Action in English; keep tags, function names, action names, JSON field names, and enum values as specified.
- Coordinates must be numbers in the range 0-1000; JSON must be valid.
- For tasks requiring an answer, complete all necessary operations and gather the information first, then use ANSWER to submit the complete answer in one call.
- ANSWER immediately ends the trajectory; no further actions or COMPLETE are needed afterward.
- Use COMPLETE to end tasks that do not require an answer; use ABORT with a reason when the task cannot be completed.
- WAIT takes no parameters and always waits for 1 second.

Citation

@article{he2026mobileweave,
  title   = {MobileWeave: Program-First Synthesis of Dependency-Linked Tasks for Training Mobile GUI Agents},
  author  = {He, Ziyi and Wu, Xian and Chen, Qiyuan and Li, Haitao and Yang, Cehao and Zhu, Yinghao and Zhang, Xichen and Chen, Xiao and Feng, Yushi and Yu, Lequan},
  journal = {arXiv preprint arXiv:2611.23860},
  year    = {2026}
}
Downloads last month
63
Safetensors
Model size
9B params
Tensor type
BF16
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for Maxnotmarx/MobileWeave-8B

Finetuned
(763)
this model
Quantizations
1 model

Dataset used to train Maxnotmarx/MobileWeave-8B

Space using Maxnotmarx/MobileWeave-8B 1

Collection including Maxnotmarx/MobileWeave-8B

Paper for Maxnotmarx/MobileWeave-8B