Kimi K3 matches GPT-5.6. It takes 4.6 times as long

Artificial Analysis gives Kimi K3 the same Coding Agent Index score as GPT-5.6 medium, but Kimi needs 23.8 minutes where GPT-5.6 needs 5.2, which is 4.6 times as long [1]. GLM-5.2 fares worse: its $1.40 input price becomes $6.51 per attempted task [2]. My read is that both models are capable, but neither offers the bargain its token price suggests.

The comparison in one table: Kimi competes, GLM-5.2 trails

Kimi K3 is competitive rather than dominant: it ties GPT-5.6 Sol medium on the combined index, leads the terminal test, falls behind on repository comprehension and runs much slower. GLM-5.2 trails all three comparison models at these settings and leads none of the measurements [1] [2].

Measurement	Kimi K3	GLM-5.2	Sol medium	Opus 5 medium
Coding Agent Index	61	43	61	62 (Best value in this row)
DeepSWE, %	64	29	64	63
Terminal-Bench v2, %	84 (Best value in this row)	72	78	79
SWE-Atlas Q&A, %	37	29	40	44 (Best value in this row)
Cost per task, $	3.18	6.51	2.99 (Best value in this row)	3.14
Active time, min	23.8	25.1	5.2 (Best value in this row)	12.2

Kimi’s tie with Sol medium is the most useful direct comparison: both score 61, and their measured task costs are close at $3.18 and $2.99 respectively [1]. The difference is how they get there. Kimi uses 23.8 active minutes and 10.6 million tokens per task, compared with 5.2 minutes and 5.8 million tokens for Sol. Opus 5 medium tells a similar story, beating Kimi by one index point while costing four cents less and finishing in roughly half the time [2].

Western models still set the highest index scores. Sol max and Opus 5 xhigh both reach 67, at $7.08 and $8.23 per task, while Opus also leads repository comprehension with 55% on SWE-Atlas Q&A [1] [2]. Fable 5 max follows at 66 but costs $11.71 per task [2]. Kimi remains competitive where it is strongest: its 84% on Terminal-Bench v2 beats Sol high’s 83% and comes within a few points of the max-effort models [1] [2].

Repository comprehension is the clear weakness. Kimi scores 37% on the codebase Q&A component, below Sol medium’s 40% and well behind the Claude configurations [1] [2]. That matters less when starting a new project, but on a 300,000-line existing codebase, understanding what is already there is most of the job.

Why is Kimi K3 getting so much hype? Browser results

Kimi’s arena rankings created most of the excitement because they show that people prefer its browser work, even when broader agent tests expose weaknesses. It leads Arena’s fullstack coding leaderboard [3] and ranks second on the WebDev Arena, ahead of Claude Fable 5 and GPT-5.6 [4]. Open weights and apparently low token prices add to the appeal.

The arena results matter because they measure what people prefer, not whether a test suite passes. On the fullstack board dated July 24, 2026, Kimi K3 max ranks first at 1,664, ahead of GPT-5.6 Sol at 1,633 and Fable 5 at 1,623 [3]. Four days later, Opus 5 max led the WebDev Arena at 1,712, while Kimi placed second at 1,682 across 3,777 votes, ahead of Fable 5, Sol and GLM-5.2 [4]. Users clearly like what Kimi builds in a browser.

The releases back up the open-weight claim. Moonshot describes Kimi K3 as the first open-weight model with 2.8 trillion parameters, alongside a 1-million-token context window, built-in vision and API prices of $3 per million input tokens and $15 per million output tokens [5]. Z.ai publishes GLM-5.2 at a smaller 753 billion parameters under an MIT license [6], with list prices of $1.40 for input and $4.40 for output [7]. On list price alone, both appear much cheaper than their Western competitors.

Moonshot’s own launch post is more restrained than that comparison suggests. It says K3 still trails Fable 5 and GPT-5.6 Sol, with “a noticeable gap in user experience” against both [5]. The independent agent measurements make that gap concrete, particularly in speed and repository comprehension. Moonshot’s own caution is worth remembering whenever a Chinese lab is named as the reason for something, and that is what I argue in Claude’s text watermark answers the EU rather than the Chinese labs.

What does the Coding Agent Index measure? Complete agent systems

The Coding Agent Index measures complete model-and-agent systems across software engineering, terminal work and repository questions. Artificial Analysis combines 113 DeepSWE tasks, 84 Terminal-Bench v2 tasks and 124 SWE-Atlas Q&A questions with equal weight in version 1.3 [8].

That boundary matters because Kimi ran inside Kimi Code CLI, GLM-5.2 inside Claude Code, Sol inside Codex, and the Claude models inside Claude Code [1] [2]. Each score therefore reflects the model, its agent and the chosen effort setting together. The table can compare those working configurations, but it cannot tell us how the same weights would rank inside a neutral harness.

Are Kimi K3 and GLM-5.2 cheaper? Not per completed task

No. The cost of a completed task depends on how many tokens the agent needs and how often it succeeds, not just what each token costs. GLM-5.2 is the clearest example: it has the lowest list price here, yet costs more per attempted task than every medium-effort Western configuration [2] [7].

Anthropic made Claude Sonnet 5’s introductory pricing permanent at $2 for input and $10 for output per million tokens [9]. Kimi K3 costs $3 and $15 [5], so both its input and output prices are 50% higher than Sonnet’s. Kimi does not have a list-price advantage against that model.

Further up the price list, Anthropic charges $5 for Opus 5 input and $25 for output, while Fable 5 costs $10 and $50 [9]. OpenAI prices GPT-5.6 at $5 and $30, or $10 and $45 in its long-context tier [10]. Z.ai undercuts them all with GLM-5.2 at $1.40 and $4.40, plus cache reads at $0.26 [7]. All figures are per million tokens.

Those units are not perfectly comparable. Vendors use different tokenizers, and Anthropic says its current one produces roughly 30% more tokens for the same text than its previous tokenizer [9]. Moonshot’s quickstart also says Kimi K3 always reasons, cannot disable that mode and defaults to max effort [11]. Its hidden reasoning tokens still appear on the bill.

list price, per million input tokens

$1.40

the cheapest model in this comparison

measured, per attempted task

$6.51

inside Claude Code, 25.1 active minutes

derived, per solved task

$15.14

cost per attempt divided by the 43% pass rate

Once I compare solved tasks rather than attempted ones, Sol medium, Opus 5 medium and Kimi K3 stay close at roughly $4.90, $5.06 and $5.21. GLM-5.2 jumps to $15.14, about three times as much. The lowest token price in the group produces the highest estimated cost for a successful result.

That estimate is still kind to failed runs because it excludes engineer time. Ten minutes spent reviewing one failure costs more than any model charge in the figure, so success rate matters far more than token price once a person has to inspect the output. With the cost claim settled, the next question is which tasks fit Kimi well enough to justify its slower runs.

What is Kimi K3 good at? Terminals and fullstack work

Kimi K3 makes the strongest case for terminal-heavy tasks, fullstack web development and long agent runs, especially when a team needs open weights and local hosting. Independent measurements, arena votes and Moonshot’s own results all point toward those uses [1] [3] [5]. They do not support treating Kimi as a cheaper drop-in replacement for work on large existing repositories.

The terminal result appears in both independent and vendor testing: Artificial Analysis measured 84% in Kimi’s own CLI, while Moonshot measured 88.3% in its preferred configuration [1] [12]. K3 also leads Moonshot’s SWE-Marathon table at 42, although the footnotes matter. Fable 5 used fallback models on 35% of its tasks, and some Kimi rows ran a prerelease branch calibrated for H20 hardware [12]. Add Kimi’s dedicated vision encoder and 1-million-token context window, and Moonshot’s proposed workflow between code and live screenshots has credible technical support [5] [12].

Those strengths come with practical limits. The 37% codebase Q&A score makes Kimi a questionable choice when repository comprehension drives the task [1], and 23.8 active minutes per run means either running agents in parallel or waiting. Self-hosting is hardly casual either: 2.8 trillion parameters require roughly 1.4 TB for the weights at 4-bit quantization, before the KV cache and serving overhead. A practical deployment therefore needs a server cluster rather than a workstation.

The legal terms add another check before adoption. A model-as-a-service business above $20 million in total revenue needs a separate Moonshot agreement, while products above 100 million monthly active users or $20 million in monthly revenue must display “Kimi K3” prominently. Internal use is exempt [13]. Moonshot’s standard API terms also allow customer content to be used to provide, maintain, develop, support and improve its services, with model-training restrictions handled through separate enterprise arrangements [14]. I would get those restrictions in writing before sending a proprietary repository to the public API.

Why does GLM-5.2 struggle in Claude Code? The agent matters

GLM-5.2’s nine-point Terminal-Bench drop is best read as a warning about the agent setup, not proof that the model weights are weak. The model scores 81% in Z.ai’s Terminus harness and 72% inside Claude Code [6] [2]. Nobody outside Z.ai can identify the exact cause, but tool calling, error recovery, caching or the agent’s conventions may explain the gap.

Whatever the cause, that configuration performs poorly. GLM-5.2 scores 43 on the index and 29% on both DeepSWE and codebase Q&A; at $6.51 and 25.1 active minutes per attempt, it is also the slowest configuration in the comparison [2]. Its $1.40 input and $4.40 output prices never translate into a low task cost.

Z.ai’s model card shows why the surrounding system matters. Its 62.1 SWE-Bench Pro result uses OpenHands with a tailored instruction prompt, DeepSWE runs in isolated containers with two-hour timeouts, and Terminal-Bench uses Terminus with a four-hour budget [6]. Those are valid measurements of GLM-5.2 in a setup chosen for it, but they do not predict how it will behave in another agent.

GLM-5.2 still has a plausible use in self-hosted, automatically verified work. Its 753 billion parameters and MIT license make it the easier of the two models to host [6], while the $1.40 input price suits bulk codemods or migration candidates backed by deterministic checks. Those checks can reject the 57% of failed attempts without manual review. If an engineer has to inspect them, the token discount no longer saves money. Hosting is its own cost, as I found with DeepSeek V4.1 Flash, whose low API prices come with enormous open weights.

How to read a coding leaderboard: compare the harness first

Start with the harness, benchmark version and task being measured, because a leaderboard score belongs to that setup. The gap between Kimi’s arena success and GLM’s harness-dependent results shows why the model name and final number are not enough.

Signal	What it rewards	What it cannot tell you
DeepSWE, 113 tasks	completing end-to-end engineering tasks	fit with your codebase’s conventions
Terminal-Bench v2, 84 tasks	driving a shell to a verified end state	comprehension of a large existing repository
SWE-Atlas Q&A, 124 tasks	answering technical questions about code	the ability to land the change it describes
WebDev and Fullstack Arena	what people prefer in head-to-head votes	correctness, tests, security, maintainability
Vendor model cards	the model at its best, in its own setup	comparability between rows

Infrastructure is the first source of uncertainty. Anthropic’s engineering team changed a Terminal-Bench 2.0 score by six percentage points through resource limits alone, while infrastructure errors fell from 5.8% to 0.5% as those limits increased [15]. The team concluded that gaps below about three points deserve skepticism until configurations are documented and matched. Kimi and Sol medium sit inside that range, so their shared score means parity rather than a win for either model.

The second issue is inconsistent test setup. Moonshot runs each competitor in a different harness, uses fallback models on some rows, and includes its own in-house benchmark. Fable 5 used fallbacks 13 times and refused one of 80 tasks on that test [12]. Moonshot discloses the details, but a comparison loses much of its value when those footnotes disappear from the summary.

Finally, the benchmark version has to match. Artificial Analysis versions its index because the task mix changes, with v1.3 current here [8]. Scores from different versions are not directly comparable, even when the benchmark name stays the same. A useful comparison therefore states the harness, effort setting, task budget and version. The Muse Spark 1.3 rescore on the Artificial Analysis index shows how far a version change can move a rank.

My routing map: Kimi for UI, GLM behind automatic checks

I would trial Kimi K3 on greenfield fullstack and UI-heavy work, while GLM-5.2 stays behind automatic checks in a native harness. For the rest, I would still route work to Claude or GPT-5.6 according to the task:

The work	My pick today
Routine changes with solid tests	Sonnet 5, at its permanent $2/$10 price
Fast agent loops and terminal work	GPT-5.6 at medium or high effort
Brownfield work in a large repository	Opus 5 at medium or high effort
Greenfield fullstack and UI-heavy builds	Kimi K3, supported by the arena results
Bulk work with automatic verification	GLM-5.2 in its native harness, as a pilot
Escalation when everything else fails	Fable 5, sparingly

The table follows the same rule I used during Opus 5’s launch week: compare the cost of approved changes, not token prices alone. That puts Kimi where its terminal and arena results are strongest, while keeping GLM behind automatic checks.

Two measurements would change those choices. Kimi K3 needs to hold its score of 61 in a neutral harness such as Claude Code or Codex, which would separate the model’s ability from the advantage of Kimi Code CLI. GLM-5.2 needs the opposite test: an independent run in its native agent by someone who does not sell it. Until then, both models show useful capability in specific tasks, but neither offers the broad cost advantage suggested by its token price.