Papers
arxiv:2611.02017

Switch-SWE: Do Coding Agents Hold Focus When Switching Between Unfinished Tasks?

Published on Nov 1
ยท Submitted by
Qianli Ma
on Nov 4
Authors:
,
,
,
,
,
,
,
,

Abstract

A benchmark for resuming interrupted software tasks reveals performance drops from interleaved work, and a context-management module improves recovery.

Software development sessions often involve several unfinished tasks rather than one issue carried through to completion. As coding agents take on repository-level development, they need to recover an earlier task's requirements and partial progress after intervening work. Existing repository-level benchmarks typically score one issue-to-patch episode and do not directly measure whether an agent can resume a task after advancing other unfinished work in the same repository. We introduce \ourmethod, a benchmark for focus-task resumption in sessions that interleave unfinished tasks drawn from the same source repository, with a persistent checkout for each task. Each focus task is evaluated in matched uninterrupted and switched sessions with identical focus requirements and executable verifiers. The switched session interleaves ordered stages from other unfinished tasks in the same repository, while only the final focus patch is scored. \ourmethod~contains 200 focus tasks from 19 Python repositories, reconstructed from issue, pull-request, and review histories and rendered as 400 matched session specifications. Across 7 coding agents, interleaved unfinished work produces an average focus-task resolution gap of 11.5 percentage points, ranging from 6.0 to 15.0 percentage points. Trajectory analysis identifies cross-task requirement confusion and loss of task state as recurrent resumption behaviors. We introduce a plug-in context-management module that selects task-relevant history and maintains concise task-state summaries. On OpenCode, the module improves focus-task completion under same-repository task switching by 5.5 to 15.5 percentage points across the evaluated models. Code is available at github.com/Miraclecsc/Switch-SWE.

Community

Paper author Paper submitter

Switch-SWE is a repository-level benchmark for evaluating whether coding agents can resume a focus task after working on other unfinished tasks in the same repository. Each task is evaluated in matched uninterrupted and switched sessions, requiring agents to recover task-specific requirements and partial progress while avoiding cross-task confusion.

The benchmark provides 200 focus tasks from 19 Python repositories, 400 matched session specifications, and executable verifiers. Across seven coding agents, task switching reduces focus-task resolution by 11.5 percentage points on average. A plug-in context-management module that selects relevant history and maintains task-state summaries improves completion by 5.5โ€“15.5 percentage points on OpenCode.

๐ŸŒ Paper: https://arxiv.org/abs/2611.02017
๐Ÿ”ฅ Project Page: https://mqleet.github.io/Switch-Swe/
๐Ÿ•น๏ธ Code: https://github.com/Miraclecsc/Switch-SWE

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

Hello. I read your paper with great interest. You show that switching between unfinished tasks lowers focus-task resolution by 11.5 percentage points on average, although its requirements and verifier stay the same, and you connect this mainly with loss of task state and confusion between the requirements of different tasks.

It seems to me that a considerable part of this drop may be explained not by switching as such but by overload, that is, by the limited ability of a model to carry out instructions as their number and volume grow. With switching, the accumulated context holds more requirements of different tasks, and the model has to keep them apart correctly. Your Table 3 points to this. On DeepSeek-V4.1-Flash, history routing raises resolution from 31.5% to 42.5%, and adding state memory raises it to 47%, which practically closes the gap with the uninterrupted condition (46.5%). The switching itself stays the same: the same tasks, the same requests, the same returns. Only what the model receives in the request changes. This leaves the question of mechanism: how much of the improvement comes from less volume and fewer foreign instructions, and how much from more accurate recovery of task state or from the module telling compaction to preserve requirements. The difference between models also points to overload. On the same tasks the gap ranges from 6 to 15 points, and GLM-5.3 and Kimi K3, nearly equal without switching (49% and 48%), diverge to 43% and 34.5% with it. This is how a limit specific to each model should behave: while the load is small the models are close, and as it grows they diverge.

A study of the behavior and degradation of language models under controlled load (arXiv:2610.36095) measures the effects of the number of independent rules and of text volume separately and shows that, in the measured model, quality falls by about the same amount with each doubling of the number of rules. It would be interesting to compare these dependencies with your results, allowing for the difference between elementary instructions and programming tasks. You compare one task with five. If you added two, three, four and ten, would resolution fall in the same way with each doubling of the number of tasks, and how would the frequency of missed requirements and of instructions wrongly carried over between tasks grow with it? The total tokens in Table 2 cannot show this. Such a comparison would help establish what part of the decline is due to limits on executing instructions and what part to switching and state recovery themselves.

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2611.02017
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.02017 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.02017 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.02017 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.