LLMs Keep Thinking

Project page

LLMs Keep Thinking When Told Not To

No-thinking controls are not a uniform switch: they compress Boolean verification, weaken on multiple choice, and fail most visibly on open-ended generation.

Dianqiao Lei1, Kevin Qinghong Lin2, Pan Lu3, Philip Torr2, James Zou3

1Tsinghua University · 2University of Oxford · 3Stanford University

Paper coming soon Code & Demo Contact

Overview

No-thinking must be measured at the response level.

We split each model response into visible pre-answer text T and final answer A. A model is closer to no-thinking only when it preserves task accuracy while exposing little or no question-conditioned payload before the answer.

Acc.

Task accuracy of the final answer.

ETR.

Empty-thinking ratio for visible pre-answer text.

Sim.

Semantic relevance between the question and visible payload.

Core result

Answer-space support governs compressibility.

Animated chart showing ETR dropping and Sim rising as the answer space opens from Boolean to MCQ to open-ended tasks.
Under native think-off, ETR. drops toward zero as the answer space opens, while Sim. rises: open-ended tasks keep question-conditioned work in the answer channel.
01

Think-on holds; think-off leaks.

Native think-off controls compress Boolean verification most reliably, are inconsistent on multiple choice, and fail almost completely on open-ended tasks.

02

Strong suppression can remove needed computation.

Stricter answer-only constraints may raise ETR. on open-ended tasks, but can also remove work needed for accuracy.

03

Task structure controls no-thinking.

Rewriting the same math questions into Boolean, MCQ, and open-ended forms changes how much visible payload remains.

Experiments

We test controls, answer spaces, and matched rewrites.

Thinking inertia results across task families.
Thinking inertia across answer-space families.
Comparison of no-thinking intervention variants.
No-thinking variants and accuracy trade-offs.
Matched answer-space rewrite results.
Matched Boolean, MCQ, and open-ended rewrites.

Resources

Code, project page, and contact.

Contact: kevin.qh.lin@gmail.com

Code & Demo: github.com/LeiDQ/ThinkZero

Paper: coming soon

Website: leidq.github.io/ThinkZero/

Citation

Cite this work.

@misc{lei2026llmskeepthinking,
  title = {LLMs Keep Thinking When Told Not To},
  author = {Dianqiao Lei and Kevin Qinghong Lin and Pan Lu and Philip Torr and James Zou},
  year = {2026},
  note = {Preprint. Citation details coming soon.}
}