2026-08-31

On Qwen3-1.7B, keeping the full task prompt while evicting old thoughts preserves reported quality and speeds long decoding

Prefix Sliding for efficient test-time scaling

The researchers test whether a reasoning model can keep the complete task prompt while retaining only a recent slice of its generated thoughts, instead of storing the whole chain in its key/value cache. They find that this Prefix Sliding rule preserves reported Qwen3-1.7B quality at long outputs while making decoding much faster, because the retained state stops growing after the prompt and local window are filled. This changes how one can judge long test-time scaling: in the tested setup, many old reasoning tokens can be evicted, although the tested LiveCodeBench case shows a boundary: code written early can be lost after a long interval of reasoning, and the paper does not report an evaluation of delayed factual-reference tasks.

In full attention, each new token can access every earlier token, so the cache keeps the entire generated trace. Prefix Sliding permanently retains the system and task prompt, including available tool definitions, and adds only the most recent generated window; older generated tokens leave the cache. The retained decoding cache is therefore bounded by the fixed prefix length plus the local window, even when reasoning continues for hundreds of thousands of tokens. This does not reduce the prompt pre-fill cost: extremely long prefixes can still consume substantial memory. A custom Hopper/FlashAttention-compatible kernel skips attention tiles outside this region. The authors also use “Continue PE”: retained tokens keep their original positional encoding after eviction, avoiding cache reconstruction.

This is an extension of an existing streaming-cache pattern. StreamingLLM already retained a small set of initial tokens plus a rolling recent cache for bounded-cost decoding. A closer predecessor is H2O, an earlier training-free KV-cache policy that dynamically retains attention heavy hitters together with recent tokens and reports end-to-end generative-inference results. Prefix Sliding instead fixes the retained prefix in advance and combines it with a recent window; its increment is this prefix-preserving policy, the custom kernel, and its application to long reasoning and truncated-backpropagation training, not the first bounded-cache decoder. The fixed prefix is intended to preserve instructions and tools that a small set of attention-sink tokens or retrospectively selected heavy hitters might not retain.

The training recipe applies the same distinction to long reinforcement-learning rollouts. The trainer receives the last four local windows: the earlier three provide context, while the final window receives the learning loss. Before using this approximation, the authors compare generator and trainer log probabilities as the supplied context grows. This is a useful research operation because it measures whether the trainer is reproducing the generator’s local behavior; it does not establish equivalent credit assignment for actions far before the final window.

The clearest training-free evidence is Appendix C, Table 1. On Qwen3-1.7B with a 4,096-token window, Prefix Sliding reports AIME25 avg@64 of 33.9 versus 34.2 for full attention, where avg@64 is the mean result over 64 stochastic runs. In the same serving benchmark, throughput at a 32K sequence length is 5,479 versus 1,477 tokens per second on one 80GB H100 using vLLM auto-batching and FlashAttention. The result supports a strong quality-throughput operating point, while the table does not establish equal-token quality: the methods produce different average output lengths, and no uncertainty intervals are reported.

The transferable operation is to classify context before evicting it: preserve immutable task state, bound recent working state, and test the eviction rule with delayed-reference tasks. The paper’s LiveCodeBench limitation gives the important boundary: when code written early is needed after a long interval of reasoning, a short window can lose necessary state. Prefix Sliding is therefore a practical cache policy for suitable long-horizon traces, not evidence that historical reasoning is universally disposable.

Read Appendix C, Table 1 to verify how the near-full-attention quality point is paired with long-sequence throughput and unequal generated lengths.

Section 2, “Prefix Sliding without training” and “Prefix Sliding kernel implementation” · Appendix C, Table 1 · Limitations, “Information loss”; Figure 11 · Abstract; Sections 1 and 3.2; Figure 1(d)

A fresh-batch test motivates SAMuon: keep the strongest update direction at Muon’s scale and give larger steps to the rest

Spectral Allocation: Why Muon Outperforms Adam, and How to Improve Muon

Researchers ask whether every direction in a matrix update should use the same step size. They split each update into its strongest direction and the remaining directions, then compare how much step size each part can tolerate on a fresh batch rather than on the batch that produced the update. The strongest direction tolerates roughly Muon’s current scale, while the remaining bulk tolerates larger steps, so SAMuon keeps the strongest direction near Muon’s scale and enlarges the steps for bulk directions. For the comparison below, the paper calls its tuned Scion implementation “Muon.” This makes assigning equal update amplitude to every direction look conservative in the bulk, while leaving the broader explanation of Muon versus Adam and SGD as suggestive rather than fully established.

Muon keeps a momentum buffer for each matrix and whitens it, so its strongest direction and the remaining singular directions receive approximately equal update amplitude. The paper’s added operation is an out-of-sample, rank-resolved probe. At a checkpoint, one gradient batch updates the buffers; the authors decompose 72 matrix buffers into singular directions and assemble one model-wide probe from the same rank across all matrices. A disjoint held-out batch then determines how useful a step along that probe is. A local quadratic estimate provides the preferred step size without constructing a Hessian, and an appendix checks the estimate against direct one-dimensional step searches in the relevant range. This extends the concurrent curvature account of Muon by measuring transferable per-rank allocation, rather than only comparing aggregate in-batch curvature quantities.

The probes show a volatile head and a tolerant bulk. On the 64M Muon trajectory, the leading direction’s preferred step coincides with Muon’s uniform scale, while bulk directions prefer steps several times larger. SAMuon keeps the leading directions near the Muon scale and increases the bulk scale. The full version estimates several leading directions with randomized low-rank SVD and applies a gradual rank-based profile. SAMuon-lite estimates only the leading singular pair with power iteration, keeps that direction anchored, and boosts everything else uniformly. Both retain the same persistent momentum-buffer state as Scion/Muon and warm in the shaping during early training.

The main comparison uses seven FineWeb pretraining cells from 124M to 1B-parameter, 12-layer modded-nanogpt models. Against the paper’s tuned Scion implementation—the implementation it calls Muon—SAMuon reaches lower final validation loss in every evaluated cell and is estimated to require 13.3%–24.0% fewer tokens to reach Scion’s final loss. The comparison is asymmetric: hyperparameters were tuned per batch size on the 124M model and transferred across scales; SAMuon used a fixed radius of 50 and transferred gamma without retuning at larger scales, and the authors note that its Muon-optimized schedule and larger-scale settings leave it relatively under-tuned. SAMuon-lite achieves a 13.3%–22.1% token-efficiency improvement relative to the Scion schedule, retaining most of SAMuon’s 13.3%–24.0% gain. These estimates come from separately trained shortened, fully annealed schedules. Each configuration has one seed, and the comparison is against Scion rather than a separately tested vanilla-Muon implementation. In one timing test, lite added 0.5% per iteration, whereas full SAMuon’s SVD added 7.4%; this was on one RTX 6000 Ada setup, not an end-to-end time-to-target measurement.

The reusable research operation is to form directions from one batch, measure their useful step sizes on another, validate the cheap estimator against direct search, and then test the simplest allocation rule suggested by the profile. That separation can expose apparent tail capacity caused by fitting the construction batch. The evidence remains local and one-direction-at-a-time: cross-rank interactions, module-specific behavior, larger deployed widths, and multi-seed variability are unresolved, and the Adam comparison omits off-diagonal couplings while the main grid contains no SGD run.

Read Sections 4.1–4.2 and Figure 1 of the paper to see how the authors split matrix updates into their strongest and remaining directions and test their step sizes on a disjoint batch; then check Section 6.2 and Table 1 to verify that SAMuon is compared with the tuned Scion implementation called Muon, not a separately tested vanilla-Muon run.

Sections 4.1-4.2 and Figure 1 (PDF pp. 5-7); Section 5.1 and Algorithm 1 (PDF pp. 9-11); Section 6.2 and Table 1 (PDF pp. 12-13). · Sections 4.1-4.2 and Figures 1-2; Sections 5.1-5.3 and Theorem 5.5.

On selected matched bug-fix tasks, adding expected behavior improved resolution across four models

RealSWE: A Compositional Evaluation of Coding Agents under Realistic User Requests

RealSWE turns one executable repository task into several request variants, keeping the code, tests, and gold patch fixed while changing which parts of the request are present or how they are worded. It finds that adding an explicit statement of the behavior the software should have consistently improves sparse bug-fix requests across four tested models. This affects how we interpret coding-agent benchmarks: a score difference can reflect missing specification content rather than model capability alone.

For a bug report, the problem statement describes what appears to be wrong, while desired behavior states what a correct result should look like. RealSWE also separates reproduction steps, environment information, and additional context. Earlier work had already shown that mutating polished benchmark issues into more user-like or underspecified requests can reduce agent success, including Saving SWE-Bench. RealSWE’s methodological addition is compositional control: it creates multiple variants from the same task so that field content can be tested separately from a bundled rewrite into a more conversational style.

The authors derive 381 task families from SWE-bench Verified and Pro, including 192 bug fixes, and evaluate them with the same mini-SWE-agent setup, source containers, step limit, and repeated runs. Their realism target comes from a filtered first-turn sample of SWE-chat: most retained requests contain only a problem statement or that statement plus limited additional context. The families are selected source tasks whose required fields are present, with Environment Information additionally taken from the released execution container for 101 bug-fix tasks. The reported “Original” condition is also source-derived: for some tasks it supplements environment information from the container and combines parts of the Pro description.

The clearest matched comparison adds desired behavior to the problem statement while holding the underlying task fixed. Across the four models, the majority-of-three-run resolution gain ranges from 6.8 to 9.9 percentage points, with every model-level comparison surviving the paper’s multiple-comparison correction. Removing reproduction steps, environment information, or additional context along the tested paths produces no consistent average loss; this does not show that such details are useless for individual incidents. In the parallel all-field rewrite, changing the expression toward the target casual style produces no stable resolution effect, supporting the interpretation that the robust result is tied more closely to missing expectations than to this bundled linguistic rephrasing.

The transferable research operation is to build matched task families before attributing performance changes to realism. Preserve the executable task and acceptance target, alter one communication factor at a time, and use paired per-task outcomes with a semantic-preservation audit. For a sparse single-turn bug report, the matched results suggest that expected behavior may be a high-value clarification target; whether an agent should ask for it first was not tested. That recommendation remains conditional on these selected source families, four models, and the single-turn scaffold; RealSWE does not test an interactive clarification policy.

Read Appendix E, Table 9, Panel C to verify the matched problem-statement versus desired-behavior comparison and its uncertainty across models.

§4.3; Appendix E, Table 9, Panels B–C · §3.1.2; §3.2.1; §4.1–§4.2 · §3.3–§4.4; Limitations Appendix A

Among episodes where the forced closed-book answer was correct, four 7–9B instruction-tuned models retained it after a plausible near-incorrect tool return in only 6.5–17.1% of eligible cases

MemToC: Benchmarking Memory-Tool Conflict Resolution in Large Language Models

MemToC fixes a question and executable tool call, records the model’s answer before seeing the tool, and then substitutes a tool return whose correctness is known. Among episodes where the model’s forced closed-book answer was correct, four 7–9B instruction-tuned models retained that answer after a plausible near-incorrect tool return in only 6.5–17.1% of eligible cases. This changes how researchers should judge tool use: following a tool is useful only when the tool is correct, so evaluation must separate source preference from source correctness.

Earlier work already elicited memory-only and tool-conditioned answers and labeled their correctness against task gold, including cases where either, both, or neither source was correct TMC, §3.5 and Table 1. MemToC’s narrower addition is a controlled post-call intervention. It derives factual questions from ToolHop, which supplies the executable-tool substrate and verified answers ToolHop, §2.1–§2.3, fixes the invocation, and replaces only the observed return with either a verified-correct value or a plausible, type-consistent wrong value. For each model, the authors first force a closed-book best guess. They then classify the episode by comparing that answer and the tool return with the verified answer: only the model answer is correct, only the tool is correct, both are correct, or neither is correct. The final response is scored separately for retaining a correct answer, following a correct tool, and repeating an incorrect tool.

The main comparison covers 542 curated factual questions, three instruction formulations, and four instruction-tuned models. In the memory-only-correct cases, models retain the correct answer in only 6.5–17.1% of eligible episodes. When the model answer is wrong and the tool is correct, they follow the tool in 86.0–93.1% of cases. When both sources are wrong, they still repeat the tool return in 78.4–86.0% of cases. These pooled ranges average the three instruction formulations for each model; each model’s formulation-specific estimates use its own eligible source-correctness cases. They therefore support a strong conclusion about deference in this controlled setting, while avoiding a universal model ranking. The forced closed-book answer is a best guess rather than a direct measurement of belief, so retention should be interpreted as a protocol-specific upper bound on ordinary-interaction behavior.

The reusable research operation is to label both competing sources before measuring arbitration. For a retrieval or agent benchmark, collect a pre-evidence answer, verify the external value independently, inject correct and plausible incorrect evidence under the same question and interface, and report the three behaviors separately. If the resulting data are used for supervised fine-tuning or preference optimization, MemToC also demonstrates why splits should follow complete tool chains and related factual groups, rather than question IDs alone. The evidence remains bounded by one ToolHop-derived, mostly biographical benchmark, synthetic short returns, and open-weight models, so it establishes a measurement pattern and a sharp diagnosis of tool deference rather than a general law of agent behavior.

Read Table 1 to verify retention, correct-tool following, and incorrect-tool following under the three source-conflict conditions, and how abstention after incorrect tool returns differs across the four models.

§3.1–§3.3; §5.1; Table 1 · §3.5; §4; Table 1 · §2

FaithSieve checks local transitions in generated Olympiad proofs and reports the first wrong step more accurately than a one-pass judge

FaithSieve: Fine-Grained Evaluation of Math Proofs with Faithful Formal Evidence

FaithSieve evaluates a natural-language mathematical proof by breaking each numbered step into smaller transitions, translating selected transitions into formal obligations, and checking them with Lean, a theorem prover. It reports better exact localization than a one-pass language-model judge on its Olympiad benchmark. This changes the evaluation question from whether some formal statement can be proved to whether that statement still expresses the local reasoning step being judged.

First-error localization and fine-grained proof evaluation already have precedents in ProcessBench and Hard2Verify, while Chain-of-States work already checks adjacent proof states with Lean during formal proof construction. FaithSieve’s concrete increment is to combine these ideas for auditing an existing, possibly incorrect informal proof. The original numbered step remains the output address, while an internal unit called an EdgeUnit records the incoming assumptions, current goal, resulting context and goal, and the kind of transition performed.

This finer unit supports typed checks. A derived fact must follow from the context available before it; a goal reduction must genuinely suffice for the preceding goal; a case split must cover the relevant cases; and a chosen witness must satisfy its target. When one step introduces a fact and uses it to reduce a goal, FaithSieve checks the introduced fact first. A suspicion-search stage selects likely problematic units and a short preceding context window, limiting the amount of formalization while preserving a path back to the original step.

Generated Lean statements pass through a semantic gate before their proof or refutation is used as evidence. The gate checks whether assumptions, objects, witnesses, direction, logical roles, and the conclusion remain aligned with the natural-language obligation. Compilation or proof search alone is insufficient; failed formalization, timeouts, and gate failures are treated as inconclusive. The resulting local evidence is then mapped back to the coarse numbered steps.

The main comparison is on ProofLoc-Olympiad: 350 algebra and number-theory problems, each evaluated with a GPT-4o-generated numbered proof and expert annotation. Against a one-pass GPT-5.4 judge given only the problem and numbered proof, FaithSieve reaches 81.43% exact first-error accuracy, compared with 72.29%. Exact accuracy requires the correct “correct” label or the exact earliest erroneous numbered step. This is evidence for the full pipeline—decomposition, scheduling, typed obligations, semantic filtering, formal checking, and synthesis—rather than for Lean checking alone, since all of those components and a much larger token budget change together.

The safeguard also has a clear weakness. In an audit of 135 generated formal statements, the semantic gate accepted 13 of 20 statements that experts labeled unfaithful. FaithSieve therefore supplies conditionally useful formal evidence, not a guarantee that a successful Lean proof represents the source reasoning. The reported 50-problem cost measurement also used 137.22 times as many recorded tokens as direct judging for GPT-5.4.

The transferable research operation is to separate the unit used for reporting from the unit used for verification. Preserve a deterministic mapping between them, check newly introduced facts before using them in later obligations, and record formalization or proof-search failure as uncertainty instead of as an error label.

Read Appendix F.4, Table 10 to verify the central first-error comparison and see exactly what the reported exact-accuracy gain measures.

Sections 3–4; Section 5.1, Table 3; Appendix E.2, Table 8; Appendix F.4, Table 10. · Sections 3.1–3.2 and Section 4.1. · Sections 1 and 3.1–3.6, especially Section 3.6. · Sections 4.1–4.3; Sections 5.1–5.2 and Algorithm 1; Appendix A.2.

Prepared retrospectively from the 2026-08-31 candidate papers; verified on 2026-09-12.