We post-trained Kimi K2.7 (Max reasoning) using reinforcement learning only, then evaluated the final checkpoint on five external coding benchmarks.
It improved on all five:
Three things stood out.
1. The gains transferred. They held across three different agent harnesses, with different tools, prompting, and loop structures. Some of the largest gains appeared on evaluation formats we never targeted in training: SWE-Marathon asks agents to build complete systems over multi-hour trajectories; Terminal-Bench 3 spans hardware, scientific computing, systems, security, and software work. Our tasks were collected before either benchmark existed.
2. A much smaller model surpassed much larger models on several evals. On Kimi’s own reported scores, our post-trained K2.7 is ahead of K3 on Terminal-Bench 2.1 and Terminal-Bench 3. On SWE-Bench Pro, it scores slightly ahead of GPT-5.6 Sol (Max reasoning). Since harnesses and task versions can differ, we treat these as contextual rather than controlled head-to-head comparisons.
3. And it used fewer steps to get there. Median trajectory length fell from 150 to 98 steps on DeepSWE and from 102 to 78 on Terminal-Bench 3. The model became more efficient.
To understand what new behaviors the model learned to achieve these gains, we studied its trajectories.
Why Kimi K2.7 failed to write shippable code
In order to understand our post-trained model’s learned behaviors, we paired base and post-trained trajectories on the same tasks and analyzed the models’ reasoning, commands, tests, and outputs.
Kimi K2.7 is already a strong agentic coding model. The base model often passed most of the target tests. Across DeepSWE failures, the median failed run still passed 86% of the target tests; 49 passed at least 80%, and 70 preserved every existing pass-to-pass test.
The model knew generally what to do. It just didn’t know how to reliably finish the job.
Its failures clustered into four recurring patterns, which the post-trained model learned to mitigate. They map to a basic engineering loop: understand, test, maintain, and verify.
One example captures the shift unusually well. On SWE-Marathon, the model had to build a Zstandard decompressor in an environment with no reference zstd binary. The trained model responded by writing a compressor first, giving itself a way to manufacture valid compressed files and test whether its decompressor recovered them correctly.
In other words, when no ground truth existed, it built it itself.
Learned Behavior #1:
Remember and implement the entire spec
Before RL. One FastAPI task had twenty requirements. Requirement 18 asked the tracking middleware to expose get_stats() and reset_stats(). The base model implemented essentially the whole feature, but put those functions at the module level instead of making them methods on the middleware object. It passed 121 of 137 target tests; the remaining sixteen called the methods exactly where the specification said they should exist, and all sixteen failed.
After RL. The trained model made them instance methods and passed 137 of 137. The base model clearly understood the feature—it had the right architecture and wrote working code—but one interface requirement disappeared somewhere between reading the spec and finishing the implementation. The trained model carried that last constraint all the way through.
Why the training data rewards this. The tasks are dense with explicit, testable requirements: named edge cases, prohibitions, exact interfaces, output formats, and constraints backed by hidden checks. Reward increases criterion by criterion, so the model repeatedly gets feedback on the difference between understanding the gist of a task and successfully satisfying the whole contract.
Why this matters. The hard part of real software is implementing every requirement, instead of declaring the job done early.
Learned Behavior #2:
Test the requirement, not your implementation
Before RL. Both models tested frequently. The difference was what they chose to test.
One task required default arguments to work for both named and anonymous functions. The base model chose an implementation that worked for:
func(a = 1)Every test it wrote used that form. But the specification also required:
func f(a = 1)Its implementation did not handle the identifier between func and (, so both hidden test suites failed. The model had designed its tests around the path its own implementation already supported.
After RL. The trained model started with the named example from the specification itself, then tried anonymous functions, multiline parameters, spread calls, immediately invoked functions, and other variants. Instead of asking whether its implementation worked on the cases it already supported, it looked for cases that could prove its interpretation of the requirement wrong.
Why the training data rewards this. The grader is hidden. During a rollout, the model sees the specification but not the tests that determine reward. It cannot simply code against visible cases; it has to infer what the specification implies and decide for itself what behavior is worth testing.
That repeatedly rewards a subtle distinction: test what was asked for, not just what you happened to build.
Why this matters. Good testing goes beyond the cases you already had in mind and uncovers the edge cases you hadn’t thought of.
Learned Behavior #3:
Protect against regressions
Before RL. One task asked the model to eliminate layout shift while preserving the same visible elements, styling, and analytics side effects. The base model achieved zero layout shift on every page, but partly by deleting two components. Its own final review judged the deletion harmless because some of their styling had been inlined.
After RL. The trained model explicitly tracked what the specification said had to remain before it started editing. It preserved the components while changing how their visual effects were applied. Instead of optimizing only for the requested change, it tracked both what needed to change and what was not allowed to.
Why the training data rewards this. Preserving existing behavior is built directly into the training signal. Many specifications explicitly say what must remain unchanged, and the repository tasks carry pass-to-pass tests that act as a hard regression gate: break required existing behavior and the reward goes to zero, no matter how many new requirements you satisfy.
Why this matters. Real engineering is rarely just “make X happen.” It is “make X happen without breaking Y, Z, and everything the user already depends on.”
Learned Behavior #4:
Reconstruct ground truth from the specification
Before RL. Some tasks do not provide an obvious reference implementation or trusted answer the model can use to verify its work. In those situations, the base model would sometimes make an assumption and then construct a test that inherited the same assumption.
On a two-dimensional spectral-solver task, the official reference functions were unavailable inside the container. The base model decided the problem was probably effectively one-dimensional, built a one-dimensional solver, and then validated it against reference data constructed with the same assumption. Its test had no way to expose the original mistake.
After RL. The trained model chose a two-dimensional solution whose correct answer it could derive mathematically, constructed an input for which that solution had to be correct, and used the pair as an independent test. That exposed an instability before submission.
We saw the same instinct elsewhere. On a Verilog console task, the trained model wrote a Python emulator and compared the hardware implementation against it instruction by instruction. And on the Zstandard task above, it wrote a compressor solely to generate valid inputs for its decompressor.
Why the training data rewards this. Nothing in the training set explicitly teaches the model to build PDE reference solutions, hardware emulators, or compressors. But every task puts it in the same basic situation: a precise specification, a hidden checker, and no access to the answer key.
To succeed consistently, the model has to develop its own evidence that the result is correct: what should be true if my solution works, and how can I check it?
Why this matters. Real work often does not come with a unit test, reference implementation, or someone nearby with the answer. Being able to create your own check is part of what makes an agent useful outside a benchmark.
It wrote better code by writing less code
One obvious explanation for the gains could be that post-training simply made the model more persistent: more reasoning, more experiments, more tests.
The trajectories point in the opposite direction. On paired DeepSWE runs, median trajectory length fell from 150 steps to 98 after training. On Terminal-Bench 3, it fell from 102 to 78.
The trained model won by being more targeted in its work: less time debugging the wrong interpretation and testing irrelevant cases.
What was in the training set?
The 1,700 tasks came from a small subset of the coding collections in Surge’s dataset catalog.
1,000 repository tasks start from repositories at pinned commits and ask the model to make specific changes. Hidden tests check the requested new behavior, while regression tests make sure existing functionality still works.
700 terminal tasks put the model inside working environments and ask it to produce real deliverables. Expert-written hidden verifiers inspect the final state.
The formats are familiar SWE-Bench and Terminal-Bench formats, but we did not build the training data to target the benchmark distributions. The repository tasks were not selected to improve SWE-Bench, and the terminal tasks were not selected to improve Terminal-Bench 2.1; they mirrored the format only. For the other evaluations, the separation is even cleaner: the training tasks were collected before DeepSWE, SWE-Marathon, and Terminal-Bench 3 existed.
When performance improves across all five external evals, we are seeing behaviors learned in one set of environments generalize to somewhere else, which is the kind of transfer we care about.
The training signal: partial credit, zero tolerance for regressions
Training was RL only, with no supervised fine-tuning stage. In order to focus on the effect of the data, we kept the training setup deliberately simple. We used GSPO, a rank-32 LoRA adapter, eight rollouts per task per step, a 65,536-token per-turn context and response cap, and maximum reasoning effort.
The reward was:
R = 1[all pass-to-pass tests pass] × fraction of target checks passed
A rollout that satisfies 3 of 12 requested checks without breaking existing behavior gets 0.25. A rollout that satisfies all 12 new checks but breaks one pass-to-pass test gets zero.
That gives the model dense feedback about progress, but a hard gate against regressions. Over many rollouts, it gets repeated feedback about which implementation choices move an incomplete solution toward a complete one. Our hypothesis is that this combination lets the model learn from near-misses rather than treating every incomplete attempt as equally wrong.
We have not run the ablations needed to claim that this reward design alone caused the change. But the correspondence between what the training signal rewards and what changed in the trajectories is hard to miss.
How post-training shapes behavior
This isn’t the first time we’ve seen high-quality environments teach models new capabilities beyond the tasks they trained on.
In one post-training run, we trained a model on long-horizon office work RL environments, with no coding tasks at all. SWE-Bench Pro still improved +5.8pp. When we analyzed the trajectories, the model appeared to have improved at a broader capability we call Goal-Directed Execution.
We saw a different kind of transfer in our ComplexConstraints post-training run. All 1,000 training examples were single-turn, yet some of the largest gains on MultiChallenge appeared in multi-turn behaviors: Instruction Retention (+22.1pp), System Steerability (+12.4pp), and Self Coherence (+10.5pp).
Our hypothesis here was that surface format changed, but the underlying skill did not. Keeping track of many interacting requirements in one dense prompt (ComplexConstraints) and keeping track of requirements that accumulate over nine turns (MultiChallenge) both depend on the same basic capability: holding onto constraints without dropping them. Complex, high-quality tasks taught the model that underlying skill.
This coding run adds the same kind of evidence: +4.7pp to +20.0pp across five external benchmarks, spanning very different kinds of coding and three different agent harnesses. The model was not just getting better at the repositories it trained on; it appeared to be learning broader software-engineering habits that survived changes in format, horizon, domain, and scaffold.
Across these studies, the question we care about is: which experiences teach which capabilities?
Intelligence emerges from experience. Our goal is to understand how post-training shapes the way models behave.
The last mile is the job
Kimi K2.7 already knew a lot about coding. Yet 1,700 expert-built tasks and a straightforward RL recipe improved it by an average of 12.5pp across five external benchmarks. The gains held across three agent harnesses, including evaluations we did not target.
The trajectories show what changed. Before training, the model would often solve the core problem and still fail somewhere around the edges: it dropped a requirement, tested only the case its implementation already handled, broke behavior that was supposed to stay intact, or never checked an assumption it was relying on. After training, it was better at finishing the whole job.
That’s the difference between writing code and shipping software. The last mile isn’t polish; it’s where “almost right” becomes either working software or a failed task.
Want to train smarter SWE agents? The coding datasets used in this post-training run are available from Surge. Explore Surge's dataset catalog or get in touch.
Appendix
Training Run Configuration
