STOP: Structured On-Policy Pruning of Long-Form Reasoning in Low-Data Regimes
Abstract
We introduce STOP, an on-policy approach to reducing redundant reasoning when fine-tuning with limited data. STOP organizes a model’s own reasoning traces into structured nodes and prunes each trace after its earliest correct answering conclusion. Across two DeepSeek-R1 distilled models and three mathematical reasoning benchmarks, STOP reduces generated tokens by 19.4–42.4% while largely preserving accuracy.
Type
Publication
EMNLP 2026
STOP uses a model’s own reasoning traces to create more efficient fine-tuning examples. It segments and annotates each trace, builds a reasoning tree, and retains the prefix ending at the earliest correct answering conclusion. This removes unnecessary reasoning after a solution while preserving a coherent path to the answer.
