Category · AI models for coding

Which AI model would you recommend for coding?

14 AI models · 230 answers · ranked by score

ordered by brand score: 100 pts for rank 1 → 20 for rank 5 · no mention = 0

  1. #1Anthropic
    90
    brand score
    87first choice
    12345
  2. #2OpenAI
    74
    brand score
    49first choice
  3. #3Google
    52
    brand score
    29first choice
  4. #4DeepSeek
    25
    brand score
    16first choice
  5. #5Meta
    13
    brand score
    11first choice
Show 10 more
  1. #6GitHub
    13
    brand score
    11first choice
  2. #7Mistral AI
    9
    brand score
    9first choice
  3. #8Amazon CodeWhisperer
    6
    brand score
    4first choice
  4. #9Tabnine
    4
    brand score
    2first choice
  5. #10xAI
    3
    brand score
    3first choice
  6. #11Replit Ghostwriter
    3
    brand score
    2first choice
  7. #12Alibaba
    2
    brand score
    1first choice
  8. #13Microsoft
    0
    brand score
    0first choice
  9. #14Cursor
    0
    brand score
    0first choice
  10. #15Sourcegraph Cody
    0
    brand score
    0first choice

Each bar covers the middle half of one model’s answers (Q1–Q3); the line inside it is that model’s median rank for the times it recommended the brand. The badge under a brand says how far apart the models are on it, and “first choice” scores rank 1 far above the rest — a brand can place well overall and still rarely lead an answer.

Swipe left for the full report

Category · AI models for coding

Brand by model

Median rank each model gave each brand.

Scroll the table sideways to see every model →

deepseek-v4.1-flashgpt-6-astragemini-3.8-flashgemini-3.7-flashgrok-4.6qwen3.8-maxclaude-opus-5gemini-3.5-flash-litekimi-k3gpt-5.6-lunagpt-5.6-solgpt-5.6-terramistral-medium-3.5claude-haiku-4.5
Anthropic1.01.01.01.01.01.01.01.01.01.01.01.0—2.0
OpenAI2.02.02.02.02.02.02.02.02.02.02.02.0—1.0
Google3.03.04.04.03.03.03.03.03.03.03.03.0—3.0
DeepSeek4.04.03.03.04.0—4.04.04.04.04.04.0——
Meta5.0—5.05.04.55.05.04.55.0————4.0
GitHub4.0————4.04.0—4.04.0—4.01.0—
Mistral AI5.05.05.05.05.04.55.05.0—5.05.05.05.05.0
Amazon CodeWhisperer————————————2.0—
Tabnine————————————3.0—
xAI————4.0—————5.05.0——
Replit Ghostwriter————————————4.0—
Alibaba—5.0————5.0—5.0—————
Microsoft————4.0————4.0————
Cursor—————————3.0————
Sourcegraph Cody————————————5.0—
1st
2
3
4
5th

A dash means that model never named the brand in any of its runs. Deeper blue is better: rank 1 is the brand a model would recommend first.

Category · AI models for coding

Where the models disagree

4 of 15 brands split the panel. Each mark is one model's score for the brand, on the same 0–100 scale as the ranking.

  1. #6GitHub

    claude-haiku-4.5 0 → mistral-medium-3.5 100 · 7 of 14 never named it

  2. #1Anthropic

    mistral-medium-3.5 0 → qwen3.8-max 100 · 1 of 14 never named it

  3. #2OpenAI

    mistral-medium-3.5 0 → claude-haiku-4.5 100 · 1 of 14 never named it

  4. #4DeepSeek

    claude-haiku-4.5 0 → gemini-3.8-flash 50 · 3 of 14 never named it

  5. #3Google

    mistral-medium-3.5 0 → claude-haiku-4.5 62 · 1 of 14 never named it

  6. #5Meta

    gpt-5.6-luna 0 → claude-haiku-4.5 40 · 5 of 14 never named it

A hollow mark is a model that never named the brand in any of its runs, which scores 0. Agreement is not endorsement — a brand every model ignores equally agrees just as tightly as one they all rank first.

Category · AI models for coding

Named, and named first

Across → how frequently the panel names the brand at all.
Up ↑ how often the answers that name it put it first.

the default answera narrow favouritelisted, rarely ledthe long tail12345678
  1. 1Anthropic91/92
  2. 2OpenAI91/8
  3. 3Google91/0
  4. 4DeepSeek60/0
  5. 5Meta52/0
  6. 6GitHub20/14
  7. 7Mistral AI44/0
  8. 8Amazon CodeWhisperer7/0

The horizontal line sits at 20% — the rate a named brand would lead at if the models were picking one of its 5 slots at random. Above it they are choosing it first on purpose. Both figures average across models, so a thinly sampled model counts the same as a heavily sampled one.

Category · AI models for coding

What the models actually said

Models
14
Answers
230

Anthropic leads with a score of 90 of 100, ranked by 13 of 14 models.

The AI models show clear consensus at the top and gradual fragmentation toward the bottom. Anthropic and OpenAI hold the first two positions across nearly all models, Google claims a stable third, and then the field splinters: models diverge sharply on whether to recommend open-weight options, niche products, or simply to stop ranking.

Anthropic

per-model scores 0–100 of 100 · mean 90 across 14 models

Anthropic achieves near-unanimity, with thirteen of fourteen models placing it first and the lone dissenter ranking it second. The panel agrees that Claude dominates repository-scale reasoning, multi-file refactoring, and agentic workflows, drawing heavily on SWE-benchwww.swebench.com×173 and Aider LLM Leaderboardsaider.chat/docs/leaderboards×155 to anchor claims of benchmark leadership and production reliability. The single outlier, claude-haiku-4.5, cites slightly weaker mainstream workflow integration but still rates Anthropic highly.

OpenAI lands in second across most models, recognized for exceptional reasoning and the widest ecosystem reach yet judged narrowly behind on sustained, iterative coding. The split is subtle: models cite the same performance sources but differ on whether raw versatility or task-specific consistency matters more. GitHub Copilot integration and API maturity repeatedly surface as OpenAI's decisive practical edge, even when benchmark gaps favor Claude.

AnthropicOpenAI
  • 100gpt-5.6-terra76
  • 100claude-opus-580
  • 100gemini-3.5-flash-lite80
  • 100gemini-3.7-flash80
  • 100gemini-3.8-flash80
  • 100gpt-5.6-sol80
  • 100gpt-6-astra80
  • 100grok-4.680
  • 100kimi-k380
  • 100qwen3.8-max80
  • 80deepseek-v4.1-flash64
  • 98gpt-5.6-luna82
  • 0mistral-medium-3.50
  • 78claude-haiku-4.5100

Anthropic ahead on 12 of 14 models, level on 1

Google's Gemini earns consistent third-place rankings with minimal disagreement. Every model that ranked it emphasized massive context windows as the defining feature, valuable for whole-repository analysis but insufficient to overcome what the panel frames as weaker instruction-following and agentic reliability. Two Gemini models—gemini-3.7-flash and gemini-3.8-flash—rank Google fourth rather than third, a modest self-deprecation that may signal awareness of the gap their own reasoning describes.

claude-haiku-4.5
62
rates it highest
mistral-medium-3.5
0
never named it

Google · 62 points apart on a 0–100 scale

The field fragments beyond the top three. DeepSeek, Meta, and Mistral AI occupy contested middle ground, with models differing on whether open weights, cost efficiency, or self-hosting flexibility warrant inclusion at all. DeepSeek splits models into two camps: those emphasizing value-driven deployment (often placing it fourth) and those focusing on turnkey polish (ranking it lower or omitting it). The divide plausibly traces to whether a model prioritizes developer autonomy or managed convenience, though the data does not confirm this hypothesis directly.

DeepSeek

per-model scores 0–50 of 100 · mean 25 across 14 models

GitHub and xAI provoke category confusion. GitHub's brand score of 12.69 hides a stark split: mistral-medium-3.5 ranks it first in every run, while half the panel excludes it entirely. The tension centers on whether Copilot is a model or a product layer over other labs' backends. Models that included GitHub praised workflow integration but noted it "is a router over other labs' models rather than a frontier model itself," making recommendation awkward when the question asks for AI models. xAI faces a similar but milder skepticism, with models acknowledging capable general reasoning but citing thin coding-specific tooling and benchmarks.

mistral-medium-3.5
100
rates it highest
claude-haiku-4.5
0
never named it

GitHub · 100 points apart on a 0–100 scale

Meta and Mistral AI earn broad but lukewarm agreement. Both are framed as strong open-weight options valuable for local deployment, fine-tuning, and privacy-sensitive workflows, yet consistently trailing proprietary leaders on complex tasks. The panel converges on this trade-off without serious dissent, suggesting the open-versus-closed capability gap is accepted across model families. Sources cluster around Hugging Facehuggingface.co×61, Meta AIai.meta.com×43, and Mistral AI documentationdocs.mistral.ai×29, reinforcing the view that these brands excel at democratizing access but not at frontier performance.

The long tail—Amazon CodeWhisperer, Tabnine, Replit Ghostwriter, Alibaba, Microsoft, Cursor, Sourcegraph Cody—receives minimal and highly concentrated attention. Most appear in only one or two models' rankings, and several owe their entire presence to mistral-medium-3.5, which alone ranks CodeWhisperer, Tabnine, Replit, and Sourcegraph. This concentration suggests a hypothesis: models may differ in how they interpret the scope of "AI model" versus "coding tool," with some excluding product layers, IDE extensions, or platform-specific assistants while others include them. The data does not clarify this boundary, but the uneven coverage hints at definitional ambiguity rather than quality consensus.

Alibaba's Qwen models earn respect as top open-weight coding options from the few models that mention them, yet rank last because they "demand more setup work and still lag proprietary leaders on the hardest tasks." Microsoft and Cursor barely register, treated as product wrappers rather than model providers, while Sourcegraph Cody appears once and is dismissed as too immature.

The most frequently recalled sources—SWE-benchwww.swebench.com×173, Aider LLM Leaderboardsaider.chat/docs/leaderboards×155, LMArenalmarena.ai×81—anchor the top-tier consensus, with Anthropic and OpenAI dominating these benchmarks. Google, DeepSeek, and others cite the same leaderboards but are consistently judged to trail. The convergence on these sources suggests the panel shares a common frame for evaluating coding capability, even when downstream recommendations diverge on cost, deployment, or ecosystem fit.

Category · AI models for coding

What shaped the answers

239 sources across 3059 references, grouped by site from 802 recalled names. The top 5 carry 37% of them.

Sources are what each model recalled as having shaped its view. It does not mean that the sources were actually used in training. Recalled URLs might also be invalid or hallucinated.

SWE-bench×351 · 11%

swebench.com

Category · AI models for coding
#1

Anthropic

also named Anthropic (Claude), 1. Anthropic, Claude

90
brand score
87first choice
12345

Summary

Anthropic commands near-universal endorsement across the panel, achieving a brand score of 89.71 and a first choice score of 87.38, with thirteen of fourteen models ranking it and the vast majority placing it first. The models converge tightly on the view that Claude excels at repository-scale reasoning, multi-file refactoring, and agentic coding workflows, consistently citing its leadership on benchmarks such as SWE-benchwww.swebench.com×173 and Aider LLM Leaderboardsaider.chat/docs/leaderboards×155. Themes of careful instruction-following, low hallucination rates, and strong performance in tools like Cursor and Claude Code recur throughout the reasoning, with models emphasizing its suitability for production work where code quality and maintainability matter more than cost.

Anthropic

per-model scores 0–100 of 100 · mean 90 across 14 models

The lone dissenter, claude-haiku-4.5, ranks Anthropic second rather than first, citing strong reasoning and safety but noting it "may be slightly behind GPT-4 in some competitive benchmarks" and "remains slightly less integrated into mainstream development workflows." Every other model places it at the top, often describing Claude as the "default choice," "gold standard," or "premier recommendation" for professional software engineering, with particular praise for handling large codebases, complex debugging, and producing clean, readable implementations. Sources such as Anthropic documentationdocs.anthropic.com×19 and LMSYS Chatbot Arenachat.lmsys.org×70 reinforce the perception of technical strength backed by both benchmark results and developer community sentiment.

“Claude models (Sonnet/Opus 4.x) lead agentic coding benchmarks like SWE-bench Verified and are the default engine behind many coding tools, including Cursor and GitHub Copilot's agent modes, excelling at large-codebase work and multi-file refactors in real-world development.”

— claude-opus-5

“Anthropic's Claude 3.5 Sonnet is widely regarded by developers as the gold standard for coding assistants due to its exceptional reasoning capabilities and deep understanding of complex codebases.”

— gemini-3.5-flash-lite

Per model summary

  • claude-opus-5
    brand score
    100
    first choice
    100
    median rank
    1.0
    present in
    25/25

    Claude models (Sonnet/Opus 4.x) lead agentic coding benchmarks like SWE-bench Verified and serve as the default engine behind major coding tools, excelling at large-codebase work and multi-file refactors in real-world development.

  • deepseek-v4.1-flash
    brand score
    80
    first choice
    80
    median rank
    1.0
    present in
    8/10

    Claude models are valued for careful instruction-following and strong performance on complex refactors, debugging, and agentic coding workflows, making them a practical default choice for serious development work despite not always leading on raw benchmarks.

  • gemini-3.5-flash-lite
    brand score
    100
    first choice
    100
    median rank
    1.0
    present in
    10/10

    Claude models, particularly Claude 3.5 Sonnet, have established a gold standard in AI-assisted development through exceptional reasoning capabilities, deep contextual understanding, and consistent superiority in generating clean, accurate code across complex software engineering tasks.

  • gemini-3.7-flash
    brand score
    100
    first choice
    100
    median rank
    1.0
    present in
    25/25

    Anthropic models, particularly Claude 3.5 Sonnet, consistently lead industry benchmarks and developer preference for code generation, excelling at large-codebase comprehension, precise refactoring, and producing clean code with minimal hallucinations.

  • gemini-3.8-flash
    brand score
    100
    first choice
    100
    median rank
    1.0
    present in
    10/10

    Anthropic models lead the industry in complex software engineering benchmarks like SWE-bench, excelling at nuanced instruction-following, multi-file refactoring, and producing clean code with remarkably low hallucination rates, making them the premier choice for professional coding workflows.

  • gpt-5.6-luna
    brand score
    98
    first choice
    95
    median rank
    1.0
    present in
    20/20

    Anthropic's Claude models are top recommendations for understanding large codebases, following detailed instructions, and performing multi-file refactors, with consistent strength in debugging and producing readable, maintainable code.

  • gpt-5.6-sol
    brand score
    100
    first choice
    100
    median rank
    1.0
    present in
    20/20

    Anthropic is the top recommendation for understanding large codebases, making precise multi-file edits, and handling agentic software-engineering workflows, with strong coding-focused tooling and broad integration making it practical for serious development work.

  • gpt-5.6-terra
    brand score
    100
    first choice
    100
    median rank
    1.0
    present in
    20/20

    Anthropic is the top recommendation for demanding coding work due to Claude models' strength in repository-scale reasoning, code review, debugging, and following detailed engineering constraints, particularly when code quality and careful explanations matter more than cost.

  • gpt-6-astra
    brand score
    100
    first choice
    100
    median rank
    1.0
    present in
    25/25

    Anthropic is the default recommendation for general-purpose coding, particularly valued for understanding unfamiliar repositories, making coordinated multi-file changes, and providing clear explanations, with agentic development tooling making it especially suited for substantial software-engineering work.

  • grok-4.6
    brand score
    100
    first choice
    100
    median rank
    1.0
    present in
    10/10

    Anthropic's Claude models, especially 3.5 Sonnet, consistently lead coding benchmarks with superior reasoning, large context windows for handling entire codebases, and reliable code generation with fewer hallucinations, making them the strongest overall choice for professional development.

  • kimi-k3
    brand score
    100
    first choice
    100
    median rank
    1.0
    present in
    25/25

    Claude models consistently top real-world coding benchmarks like SWE-bench Verified and are widely praised by developers for code quality, multi-file reasoning, and agentic coding capabilities, making them the go-to choice for professional software development work.

  • qwen3.8-max
    brand score
    100
    first choice
    100
    median rank
    1.0
    present in
    10/10

    Claude models are valued as the top choice for coding due to strength in understanding large codebases, following complex multi-step instructions carefully, and producing coherent multi-file edits with careful attention to safety and maintainability in real development workflows.

  • claude-haiku-4.5
    brand score
    78
    first choice
    48
    median rank
    2.0
    present in
    10/10

    Claude models consistently emphasize strong reasoning abilities, code safety, and thoughtful explanations, with particular strength in handling large context windows for complex projects and code reviews.

Sources per model

Each model’s own references for this brand, grouped by site. Tile area is that site’s share of the model’s references; tap one for the pages behind it.

deepseek-v4.1-flash9 sites · 23 references
Anthropic Claude×7 · 30%

anthropic.com

gpt-6-astra6 sites · 53 references
Anthropic — Claude Code best practices×22 · 42%

anthropic.com

gemini-3.8-flash6 sites · 24 references
SWE-bench×10 · 42%

swebench.com

  • ×10/
gemini-3.7-flash8 sites · 69 references
SWE-bench×25 · 36%

swebench.com

  • ×25/
grok-4.69 sites · 30 references
SWE-bench×7 · 23%

swebench.com

  • ×5/
  • ×2named without a URL
qwen3.8-max8 sites · 27 references
Anthropic documentation×10 · 37%

docs.anthropic.com

claude-opus-58 sites · 75 references
SWE-bench leaderboard×25 · 33%

swebench.com

  • ×25/
gemini-3.5-flash-lite3 sites · 26 references
Hacker News×10 · 38%

news.ycombinator.com

  • ×10/
kimi-k36 sites · 75 references
SWE-bench×25 · 33%

swebench.com

  • ×25/
gpt-5.6-luna6 sites · 51 references
SWE-bench×18 · 35%

swebench.com

  • ×18/
gpt-5.6-sol7 sites · 60 references
SWE-bench×20 · 33%

swebench.com

  • ×20/
gpt-5.6-terra7 sites · 58 references
SWE-bench×20 · 34%

swebench.com

claude-haiku-4.522 sites · 30 references
Anthropic documentation×3 · 10%

anthropic.com

Sources are what each model recalled as having shaped its view. It does not mean that the sources were actually used in training. Recalled URLs might also be invalid or hallucinated.

Category · AI models for coding
#2

OpenAI

also named OpenAI (GPT/Codex), 2. OpenAI, GPT, OpenAI (GPT / o-series / Codex), OpenAI (GPT/Codex/o-series)

74
brand score
49first choice
12345

Summary

OpenAI commands a strong position in the coding AI landscape with a brand score of 74.43 and first choice score of 49.46, appearing in the rankings of 13 of 14 models. The panel's assessment reveals significant divergence: one model awarded it a perfect 100 for consistent first-place rankings, while another gave it zero, yielding a standard deviation of 21.78 that the study labels "models split." Most evaluators placed OpenAI second, recognizing its GPT-4, GPT-4o, and o-series reasoning models as highly capable across code generation, debugging, and algorithmic problem-solving, yet narrowly trailing Anthropic's Claude for sustained, repository-level work. The consensus holds that OpenAI offers the most mature ecosystem—powering GitHub Copilot×3 and delivering broad API, IDE, and tooling integrations—even when competitors edge ahead on specialized benchmarks or long agentic coding sessions.

OpenAI

per-model scores 0–100 of 100 · mean 74 across 14 models

The reasoning behind OpenAI's strong but not dominant standing centers on three themes: exceptional reasoning and versatility, unmatched ecosystem maturity, and narrow performance gaps in specialized tasks. Models praised its advanced reasoning capabilities on competitive programming and complex algorithms, citing SWE-bench×3 and LMSYS Chatbot Arenachat.lmsys.org×70 as evidence of robust general-purpose performance. At the same time, evaluators noted variability by model version and occasional verbosity, with several pointing out that Claude feels more consistent for iterative, multi-file edits. The ecosystem argument—anchored by GitHub Copilot integration, extensive API documentation, and the widest developer adoption—repeatedly emerged as OpenAI's decisive practical advantage, even when raw coding scores fell slightly short of the leader.

“OpenAI's GPT-4 and GPT-4o models have demonstrated exceptional coding capabilities with strong performance on competitive programming benchmarks and real-world development tasks.”

— claude-haiku-4.5

“GPT-5 class and the o-series reasoning models are excellent at algorithmic problem solving, debugging, and competitive-programming-style tasks, with Codex/Codex CLI providing a solid agentic coding workflow.”

— claude-opus-5

Per model summary

  • claude-haiku-4.5
    brand score
    100
    first choice
    100
    median rank
    1.0
    present in
    10/10

    GPT-4 and GPT-4o models consistently demonstrate exceptional coding capabilities with strong competitive programming benchmark performance, and their integration with GitHub Copilot has established industry-standard adoption among developers.

  • claude-opus-5
    brand score
    80
    first choice
    50
    median rank
    2.0
    present in
    25/25

    GPT-5-class and o-series reasoning models excel at algorithmic problem solving and competitive programming tasks, supported by the broadest ecosystem, API maturity, and tooling integrations in the industry.

  • deepseek-v4.1-flash
    brand score
    64
    first choice
    40
    median rank
    2.0
    present in
    8/10

    GPT and Codex models are strong general coding assistants with broad language coverage and deep ecosystem support, ranking as a very close second with mature API availability and widespread IDE integrations.

  • gemini-3.5-flash-lite
    brand score
    80
    first choice
    50
    median rank
    2.0
    present in
    10/10

    OpenAI remains a powerhouse through GPT-4 series models that offer robust multi-language support and seamless integration into major developer tools like GitHub Copilot, maintaining reliability despite being edged out in some recent nuanced coding benchmarks.

  • gemini-3.7-flash
    brand score
    80
    first choice
    50
    median rank
    2.0
    present in
    25/25

    OpenAI provides industry-standard coding capabilities with powerful reasoning models and the deepest integration into the widest developer ecosystem, maintaining exceptional reliability across complex algorithmic tasks despite tight competition.

  • gemini-3.8-flash
    brand score
    80
    first choice
    50
    median rank
    2.0
    present in
    10/10

    OpenAI delivers industry-leading reasoning and code generation with deep integration across developer ecosystems and popular IDE extensions, though occasionally requiring more iterative prompting or showing less consistency than top rivals on large-scale refactoring.

  • gpt-5.6-luna
    brand score
    82
    first choice
    55
    median rank
    2.0
    present in
    20/20

    OpenAI offers highly capable general-purpose models with strong performance across code generation, debugging, and tool-assisted workflows, ranking just behind Anthropic because coding quality can vary by model and task.

  • gpt-5.6-sol
    brand score
    80
    first choice
    50
    median rank
    2.0
    present in
    20/20

    OpenAI offers excellent code generation, reasoning, debugging, and tool use backed by a mature API and extensive developer ecosystem, ranking just below Anthropic because performance varies by model and workflow.

  • gpt-5.6-terra
    brand score
    76
    first choice
    48
    median rank
    2.0
    present in
    19/20

    OpenAI is an excellent general-purpose coding choice with strong reasoning models and mature API tooling, ranking just behind Anthropic because the best choice depends heavily on specific codebases, tasks, and workflow requirements.

  • gpt-6-astra
    brand score
    80
    first choice
    50
    median rank
    2.0
    present in
    25/25

    OpenAI is a close second offering strong all-round coding capabilities, debugging, and reasoning, particularly appealing for its broad developer ecosystem and API integrations that support both software development and broader technical workflows.

  • grok-4.6
    brand score
    80
    first choice
    50
    median rank
    2.0
    present in
    10/10

    OpenAI models power widely adopted tools like GitHub Copilot and deliver versatile, high-quality coding performance across languages, benefiting from a mature ecosystem and extensive training data despite trailing slightly on some recent specialized evaluations.

  • kimi-k3
    brand score
    80
    first choice
    50
    median rank
    2.0
    present in
    25/25

    OpenAI's GPT and o-series models deliver excellent coding performance with the broadest ecosystem adoption including GitHub Copilot integration, though recent benchmarks often place them slightly behind Anthropic on complex software engineering tasks.

  • qwen3.8-max
    brand score
    80
    first choice
    50
    median rank
    2.0
    present in
    10/10

    OpenAI's GPT models are highly capable, versatile coding assistants with strong general reasoning and a mature ecosystem of APIs and developer tools, ranking second because they are slightly less consistent than Claude for complex repository-level edits.

Sources per model

Each model’s own references for this brand, grouped by site. Tile area is that site’s share of the model’s references; tap one for the pages behind it.

deepseek-v4.1-flash8 sites · 24 references
OpenAI HumanEval×9 · 38%

github.com

gpt-6-astra6 sites · 56 references
OpenAI — Introducing Codex×16 · 29%

openai.com

gemini-3.8-flash6 sites · 23 references
OpenAI Research×8 · 35%

openai.com

gemini-3.7-flash8 sites · 69 references
OpenAI Research×22 · 32%

openai.com

grok-4.610 sites · 30 references
OpenAI×10 · 33%

openai.com

  • ×10/
qwen3.8-max8 sites · 27 references
OpenAI Platform documentation×11 · 41%

platform.openai.com

claude-opus-56 sites · 75 references
LMArena coding leaderboard×25 · 33%

lmarena.ai

gemini-3.5-flash-lite8 sites · 26 references
OpenAI×8 · 31%

openai.com

kimi-k37 sites · 75 references
LMArena×20 · 27%

lmarena.ai

  • ×20/
gpt-5.6-luna6 sites · 52 references
OpenAI×17 · 33%

openai.com

gpt-5.6-sol5 sites · 60 references
SWE-bench×20 · 33%

swebench.com

  • ×20/
gpt-5.6-terra7 sites · 55 references
OpenAI API documentation×19 · 35%

platform.openai.com

claude-haiku-4.519 sites · 30 references
GitHub Copilot documentation×4 · 13%

github.com

Sources are what each model recalled as having shaped its view. It does not mean that the sources were actually used in training. Recalled URLs might also be invalid or hallucinated.

Category · AI models for coding
#3

Google

also named Google (Gemini), Google DeepMind, Google DeepMind (Gemini), 3. Google, Gemini

52
brand score
29first choice
12345

Summary

Google achieves a brand score of 52.33 across the panel, with per-model scores spanning 0 to 62 and broad agreement among the models. Every model except one ranked Google, and none placed it first. The consistent theme is that Google's Gemini models offer exceptional long-context capabilities—often cited as industry-leading context windows of one to two million tokens—that make them uniquely valuable for analyzing entire repositories, extensive documentation, and large-scale refactoring tasks. Models repeatedly note tight integration with Google Cloud, Android Studio, and the broader Google developer ecosystem, alongside competitive pricing and generous free tiers via AI Studio and Gemini CLI. However, the panel uniformly agrees that while Gemini is highly capable, its code generation precision, instruction-following consistency, and agentic coding reliability trail Anthropic and OpenAI on complex multi-step tasks. Sources underpinning this view include Google DeepMind Gemini model pagedeepmind.google/technologies/gemini×19, SWE-benchwww.swebench.com×173, and Aider LLM Leaderboardsaider.chat/docs/leaderboards×155.

Google's Gemini earns third place by offering unmatched context windows for whole-repository reasoning, but every model judges its coding consistency and agentic workflows less reliable than the leaders.

The models frame Google as especially attractive for niche workflows—large codebase analysis, multimodal tasks involving diagrams or UI screenshots, and environments already standardized on Google Cloud—but not as the default choice for general-purpose coding assistance. Developer mindshare and ecosystem maturity are noted as weaker than the top two, and several models caution that coding output may require more manual refinement or validation. Despite these reservations, Gemini is recognized as rapidly improving and highly competitive on benchmarks, making it a strong third option and occasionally the best fit when context length or Google integration is the priority.

“Gemini 2.5/3 Pro's massive context window excels at repository-wide reasoning and offers strong value through free tiers, though it slightly trails Claude and GPT in agentic code-editing reliability.”

— claude-opus-5

“Gemini models offer strong coding with industry-leading context windows valuable for large codebases, but developer mindshare and consistency on multi-step work trail Anthropic and OpenAI.”

— kimi-k3

Per model summary

  • claude-haiku-4.5
    brand score
    62
    first choice
    35
    median rank
    3.0
    present in
    10/10

    Gemini models provide solid coding capabilities and integrate well with Google's ecosystem, but consistently lag behind top competitors in specialized coding benchmarks and developer adoption.

  • claude-opus-5
    brand score
    60
    first choice
    33
    median rank
    3.0
    present in
    25/25

    Gemini 2.5/3 Pro's massive context window excels at repository-wide reasoning and offers strong value through free tiers, though it slightly trails Claude and GPT in agentic code-editing reliability.

  • deepseek-v4.1-flash
    brand score
    48
    first choice
    27
    median rank
    3.0
    present in
    8/10

    Gemini models are highly capable with strong long-context and multimodal abilities, but their agentic coding polish and developer ecosystem consistency lag behind Anthropic and OpenAI.

  • gemini-3.5-flash-lite
    brand score
    60
    first choice
    33
    median rank
    3.0
    present in
    10/10

    Gemini's massive context windows enable unmatched whole-repository analysis and refactoring, though code generation precision occasionally lags behind top-tier competitors.

  • gpt-5.6-luna
    brand score
    58
    first choice
    33
    median rank
    3.0
    present in
    20/20

    Google models are strong for large-context workflows and ecosystem integration, but coding quality and consistency vary more by version and environment than the top two alternatives.

  • gpt-5.6-sol
    brand score
    60
    first choice
    33
    median rank
    3.0
    present in
    20/20

    Gemini models offer strong coding with large context windows and cloud integration, but their coding precision and consistency across model tiers feel less reliable than the top two choices.

  • gpt-5.6-terra
    brand score
    57
    first choice
    32
    median rank
    3.0
    present in
    19/20

    Gemini models excel at large-context tasks and Google Cloud integration, but coding quality and tooling fit vary more by model version and workflow than the top two options.

  • gpt-6-astra
    brand score
    60
    first choice
    33
    median rank
    3.0
    present in
    25/25

    Gemini is especially attractive for large-context coding tasks involving extensive documentation or repositories, ranking third generally but potentially first when long-context analysis is the priority.

  • grok-4.6
    brand score
    60
    first choice
    33
    median rank
    3.0
    present in
    10/10

    Gemini models offer very large context windows and solid multimodal coding support with good Google ecosystem integration, but trail the top two in raw coding accuracy on independent benchmarks.

  • kimi-k3
    brand score
    60
    first choice
    33
    median rank
    3.0
    present in
    25/25

    Gemini models offer strong coding performance with industry-leading context windows valuable for large codebases, but developer mindshare and consistency on multi-step work trail Anthropic and OpenAI.

  • qwen3.8-max
    brand score
    56
    first choice
    32
    median rank
    3.0
    present in
    10/10

    Gemini models offer strong coding with very large context windows and Google ecosystem integration, but coding consistency feels less reliable than the top two across everyday tasks.

  • gemini-3.7-flash
    brand score
    46
    first choice
    27
    median rank
    4.0
    present in
    25/25

    Gemini's industry-leading context windows allow simultaneous processing of entire repositories and documentation, though generated code occasionally requires more manual refinement than top competitors.

  • gemini-3.8-flash
    brand score
    46
    first choice
    28
    median rank
    4.0
    present in
    10/10

    Gemini's massive context windows enable effortless ingestion of entire multi-million token codebases, though code generation consistency on complex logic and edge cases occasionally trails Anthropic or OpenAI.

Sources per model

Each model’s own references for this brand, grouped by site. Tile area is that site’s share of the model’s references; tap one for the pages behind it.

deepseek-v4.1-flash12 sites · 24 references
Google AI for Developers×7 · 29%

ai.google.dev

gpt-6-astra5 sites · 51 references
Google — Gemini API documentation×32 · 63%

ai.google.dev

gemini-3.8-flash4 sites · 21 references
LMSYS Chatbot Arena×10 · 48%

chat.lmsys.org

  • ×10/
gemini-3.7-flash9 sites · 61 references
Google DeepMind×24 · 39%

deepmind.google

grok-4.614 sites · 28 references
LMSYS Arena×5 · 18%

lmarena.ai

qwen3.8-max10 sites · 26 references
Google AI Developers×11 · 42%

ai.google.dev

claude-opus-510 sites · 75 references
Google DeepMind Gemini model page×25 · 33%

deepmind.google

gemini-3.5-flash-lite9 sites · 21 references
Google Developers×8 · 38%

developers.google.com

kimi-k36 sites · 67 references
LMArena×25 · 37%

lmarena.ai

  • ×25/
gpt-5.6-luna7 sites · 51 references
Gemini Code Assist×15 · 29%

cloud.google.com

gpt-5.6-sol11 sites · 60 references
Google Gemini API documentation×19 · 32%

ai.google.dev

gpt-5.6-terra8 sites · 55 references
Google AI for Developers×21 · 38%

ai.google.dev

claude-haiku-4.524 sites · 30 references
Google AI blog×4 · 13%

no URL recalled

  • ×4named without a URL

Sources are what each model recalled as having shaped its view. It does not mean that the sources were actually used in training. Recalled URLs might also be invalid or hallucinated.

Category · AI models for coding
#4

DeepSeek

also named DeepSeek (DeepSeek-Coder/V3)

25
brand score
16first choice
12345

Summary

DeepSeek scores consistently across the panel, earning a brand score of 25 and appearing in 11 of 14 models' rankings, though never in first place. The models converge on a narrow assessment: DeepSeek occupies a value-driven niche, offering strong coding performance through open-weight models at dramatically lower cost than proprietary alternatives, but with less mature enterprise tooling and ecosystem polish. Every model that ranked it placed it fourth or fifth, and the standard deviation of 17.79 reflects broad agreement on both its strengths and limitations.

DeepSeek

per-model scores 0–50 of 100 · mean 25 across 14 models

The recurring theme is cost efficiency paired with deployment flexibility. Models repeatedly cite DeepSeek's open weights, self-hosting options, and competitive pricing as core appeals, often drawing on Hugging Facehuggingface.co×22, DeepSeek GitHubgithub.com/deepseek-ai×25, and Aider LLM Leaderboardsaider.chat/docs/leaderboards×155. The trade-off is consistent: while benchmark performance rivals frontier labs on code synthesis and algorithmic tasks, DeepSeek trails in enterprise support, IDE integrations, and agentic reliability. Models frame it as the default recommendation for budget-conscious teams, privacy-sensitive workflows, or high-volume use cases where hosting control matters, but not for organizations prioritizing polished managed services or the most complex multi-step coding agents.

“DeepSeek delivers state-of-the-art open-weight coding models that rival top proprietary offerings at a fraction of the inference cost, making it a premier choice for cost-conscious developers and self-hosted environments.”

— gemini-3.7-flash

“DeepSeek is an appealing option when cost efficiency and access to open-weight models matter, but realizing those advantages requires more attention to deployment, tooling, and model selection than the more polished alternatives.”

— gpt-6-astra

Per model summary

  • gemini-3.7-flash
    brand score
    49
    first choice
    28
    median rank
    3.0
    present in
    22/25

    DeepSeek delivers state-of-the-art open-weight coding models that rival top proprietary offerings at a fraction of the inference cost, making it a premier choice for cost-conscious developers and self-hosted environments.

  • gemini-3.8-flash
    brand score
    50
    first choice
    28
    median rank
    3.0
    present in
    9/10

    DeepSeek provides state-of-the-art coding performance that rivals top proprietary models at disruptive cost points with open-weight options, though its ecosystem tooling and platform maturity trail established commercial leaders.

  • claude-opus-5
    brand score
    37
    first choice
    24
    median rank
    4.0
    present in
    25/25

    DeepSeek's V3 and R1-lineage models deliver near-frontier coding performance at dramatically lower cost with open weights, making them the best value for budget-conscious or self-hosting teams, though they lag top proprietary labs on complex agentic tasks and tooling polish.

  • deepseek-v4.1-flash
    brand score
    28
    first choice
    18
    median rank
    4.0
    present in
    7/10

    DeepSeek offers strong price-performance and open-weight options attractive for cost-sensitive or self-hosted coding workflows, though its tooling, enterprise support, and coding-assistant ecosystem are less mature than top proprietary alternatives.

  • gemini-3.5-flash-lite
    brand score
    22
    first choice
    15
    median rank
    4.0
    present in
    6/10

    DeepSeek has emerged as a disruptive force offering open-weight models that punch well above their weight class in coding benchmarks at a fraction of the cost, though enterprise trust and adoption are still maturing.

  • gpt-5.6-luna
    brand score
    13
    first choice
    9
    median rank
    4.0
    present in
    7/20

    DeepSeek offers capable coding models at comparatively low cost with attractive open-weight availability, but reliability, hosted-service availability, and tooling consistency may be less predictable than leading commercial providers.

  • gpt-5.6-sol
    brand score
    40
    first choice
    25
    median rank
    4.0
    present in
    20/20

    DeepSeek provides strong coding performance at highly competitive prices with valuable open-weight and self-hosting options, though its enterprise support, integrations, and operational ecosystem are less mature than larger commercial providers.

  • gpt-5.6-terra
    brand score
    31
    first choice
    21
    median rank
    4.0
    present in
    17/20

    DeepSeek is a compelling value-oriented option offering strong coding performance with open-weight availability and deployment flexibility, though its enterprise tooling, support, and integration maturity are generally less comprehensive than leading proprietary platforms.

  • gpt-6-astra
    brand score
    38
    first choice
    25
    median rank
    4.0
    present in
    25/25

    DeepSeek is an appealing option when cost efficiency and access to open-weight models matter, but realizing those advantages requires more attention to deployment, tooling, and model selection than the more polished alternatives.

  • grok-4.6
    brand score
    4
    first choice
    3
    median rank
    4.0
    present in
    1/10

    DeepSeek achieves impressive benchmark scores matching larger proprietary systems at lower cost with openly available weights, though it is newer with less extensive production-tooling and enterprise validation than established leaders.

  • kimi-k3
    brand score
    38
    first choice
    25
    median rank
    4.0
    present in
    25/25

    DeepSeek delivers impressive coding performance at a fraction of the cost with strong open-weight models, making it excellent for budget-conscious and self-hosting teams, though its ecosystem tooling and enterprise maturity lag larger established providers.

Sources per model

Each model’s own references for this brand, grouped by site. Tile area is that site’s share of the model’s references; tap one for the pages behind it.

deepseek-v4.1-flash9 sites · 21 references
DeepSeek Coder×5 · 24%

github.com

gpt-6-astra3 sites · 55 references
DeepSeek-Coder repository×46 · 84%

github.com

gemini-3.8-flash7 sites · 19 references
Hugging Face×5 · 26%

huggingface.co

gemini-3.7-flash12 sites · 58 references
Hugging Face×20 · 34%

huggingface.co

grok-4.63 sites · 3 references
coding-benchmark papers×1 · 33%

no URL recalled

  • ×1named without a URL
claude-opus-57 sites · 75 references
Hugging Face DeepSeek model cards×24 · 32%

huggingface.co

gemini-3.5-flash-lite6 sites · 10 references
Hugging Face×4 · 40%

huggingface.co

  • ×3/
  • ×1named without a URL
kimi-k310 sites · 66 references
Aider LLM Leaderboards×17 · 26%

aider.chat

gpt-5.6-luna7 sites · 18 references
DeepSeek×6 · 33%

deepseek.com

gpt-5.6-sol8 sites · 60 references
DeepSeek API documentation×17 · 28%

api-docs.deepseek.com

  • ×17/
gpt-5.6-terra7 sites · 49 references
DeepSeek GitHub×14 · 29%

github.com

Sources are what each model recalled as having shaped its view. It does not mean that the sources were actually used in training. Recalled URLs might also be invalid or hallucinated.

Category · AI models for coding
#5

Meta

also named Meta (Llama), Meta (Llama/Code Llama), 5. Meta, Llama

13
brand score
11first choice
12345

Summary

Meta's Llama models earn a modest position in the coding recommendation landscape, with a brand score of 13.46 and broad agreement among evaluators—nine of fourteen models ranked it, though none placed it first. The consensus view treats Llama as a strong open-weights foundation valued for local deployment, privacy-sensitive workflows, and fine-tuning flexibility, but one that consistently trails proprietary frontier models on out-of-the-box coding performance and ease of use. Models frequently cite Hugging Facehuggingface.co×61 and Meta AIai.meta.com×43 when discussing the open ecosystem, customization potential, and community-driven development that define Meta's offering, while also noting that complex coding tasks typically require more setup, prompt engineering, or fine-tuning than specialized commercial alternatives deliver.

claude-haiku-4.5
40
rates it highest
gpt-5.6-luna
0
never named it

Meta · 40 points apart on a 0–100 scale

The divide in recommendations centers on what developers prioritize: control and transparency versus turnkey capability. Evaluators uniformly praise Llama for democratizing access to capable coding assistance, enabling complete data sovereignty, and serving as the backbone for self-hosted and privacy-compliant enterprise deployments. Yet that same consensus acknowledges the models generally underperform leaders like Claude, GPT, and Gemini on demanding benchmarks and agentic reasoning tasks, positioning Meta as the premier choice when openness, customization, or local hosting outweighs raw coding accuracy, but a secondary option when developers seek polished, hosted assistance for complex software engineering work.

“Meta's Llama models are valuable open-weight options for teams needing full control, local deployment, and fine-tuning capability, but out-of-the-box coding performance generally lags behind frontier proprietary models and requires more setup effort.”

— kimi-k3

“Llama models are the premier open-weights option enabling local, private deployment with complete data control and fine-tuning flexibility, but out-of-the-box coding performance generally requires more prompt tuning or compute than specialized proprietary models.”

— gemini-3.7-flash

Per model summary

  • claude-haiku-4.5
    brand score
    40
    first choice
    25
    median rank
    4.0
    present in
    10/10

    Llama models are open-source and useful for local or customized deployments, but consistently underperform leading proprietary models on coding benchmarks and require more setup or fine-tuning to match top-tier alternatives.

  • gemini-3.5-flash-lite
    brand score
    30
    first choice
    23
    median rank
    4.5
    present in
    10/10

    Llama models provide solid open-source foundations for local, privacy-focused coding assistance with unmatched customization potential, but generally require significant setup and fine-tuning to match the out-of-the-box performance of proprietary alternatives.

  • grok-4.6
    brand score
    30
    first choice
    23
    median rank
    4.5
    present in
    10/10

    Llama models are strong open-source alternatives that can be run locally, fine-tuned, and deployed privately with cost advantages, but generally trail leading closed-source frontier models on demanding coding benchmarks and out-of-the-box ease of use.

  • claude-opus-5
    brand score
    10
    first choice
    10
    median rank
    5.0
    present in
    12/25

    Llama models are valuable for open-weight, on-premise deployment and fine-tuning with strong community support, but recent releases have fallen behind both proprietary frontier models and Chinese open-weight competitors on coding benchmarks.

  • deepseek-v4.1-flash
    brand score
    12
    first choice
    12
    median rank
    5.0
    present in
    6/10

    Llama models are important open-weight options valuable for customization, local deployment, and privacy-sensitive environments, but generally trail top proprietary models in out-of-the-box coding performance on complex tasks.

  • gemini-3.7-flash
    brand score
    13
    first choice
    13
    median rank
    5.0
    present in
    16/25

    Llama models are the premier open-weights option enabling local, private deployment with complete data control and fine-tuning flexibility, but out-of-the-box coding performance generally requires more prompt tuning or compute than specialized proprietary models.

  • gemini-3.8-flash
    brand score
    16
    first choice
    15
    median rank
    5.0
    present in
    7/10

    Meta leads in open-source innovation, providing models that offer complete privacy, customization, and local hosting flexibility, but generally require significant setup and trail dedicated proprietary APIs in out-of-the-box coding precision on complex tasks.

  • kimi-k3
    brand score
    16
    first choice
    16
    median rank
    5.0
    present in
    20/25

    Llama models are valuable open-weight options for teams needing full control, local deployment, and fine-tuning capability, but out-of-the-box coding performance generally lags behind frontier proprietary models and requires more setup effort.

  • qwen3.8-max
    brand score
    22
    first choice
    21
    median rank
    5.0
    present in
    10/10

    Llama models are valuable open-weight options for self-hosting, customization, and privacy-sensitive coding workflows where control matters, but generally require more setup and trail top proprietary models for out-of-the-box coding quality on complex tasks.

Sources per model

Each model’s own references for this brand, grouped by site. Tile area is that site’s share of the model’s references; tap one for the pages behind it.

deepseek-v4.1-flash6 sites · 18 references
Meta Llama×8 · 44%

ai.meta.com

gemini-3.8-flash3 sites · 15 references
Hugging Face×7 · 47%

huggingface.co

gemini-3.7-flash5 sites · 39 references
Meta AI×16 · 41%

ai.meta.com

grok-4.68 sites · 28 references
Hugging Face×11 · 39%

huggingface.co

qwen3.8-max7 sites · 25 references
Meta AI×8 · 32%

ai.meta.com

claude-opus-58 sites · 36 references
Hugging Face Open LLM Leaderboard×12 · 33%

huggingface.co

gemini-3.5-flash-lite5 sites · 21 references
Meta AI×10 · 48%

ai.meta.com

  • ×10/
kimi-k38 sites · 52 references
Meta AI×18 · 35%

ai.meta.com

  • ×18/
claude-haiku-4.526 sites · 30 references
Hugging Face model comparisons×3 · 10%

no URL recalled

  • ×3named without a URL

Sources are what each model recalled as having shaped its view. It does not mean that the sources were actually used in training. Recalled URLs might also be invalid or hallucinated.

Category · AI models for coding

How this was measured

The question

  • Unaided brand recommendation question: “Which AI model would you recommend for coding?”
  • The prompt asks each model to return exactly 5 brands, ranked 1 to 5, and for each one a reason for the recommendation and the sources that informed it.

Sampling

  • Models are not deterministic, so the question is asked over and over — 16 answers per model
  • Spellings of the same brand are normalized to the most commonly used form before counting

Measures of Position: Median and Quartiles

  • Median value indicates that in 50% of answers the brand held this position or higher.
  • Quartiles help visualize the spread of rankings per brand. Q1 indicates that 25% of answers had this rank or higher. Q3 means that 75% of answers ranked the brand as X or better.
  • Medians and quartiles are computed only from answers where the brand was present.

Coverage

  • Count of answers in which the given brand was named, as a percentage.

Brand score - Normalized Borda score

  • The brand score is calculated using a normalized Borda score. Borda count is a voting method: each ballot awards points by position instead of naming one winner. Our ranking responses from the LLM are always fixed to 5 answers, so we assign 100 points for rank 1, then 80, 60, 40, 20 — and 0 if the brand is missing.
Brand score equals 100 divided by n, times the sum over the n answers of (6 minus the brand’s rank in that answer) divided by 5.

where

The rank r sub i is 1 through 5 if the brand appears in answer i, and 6 if it is not mentioned.
  • This way we can calculate a common score for all brands mentioned across model responses. The score reflects both how high the brand was ranked and how frequently it was mentioned.

First choice score - adjusted MRR - top-rank indicator

  • The first choice score is based on Mean Reciprocal Rank (a common search-engine measure), but adapted to measure the position of a specific brand across repeated ranked responses.
  • Each occurrence receives a reciprocal-position score: 100 for 1st, 50 for 2nd, 33.3 for 3rd, 25 for 4th, 20 for 5th, and 0 when the brand is not mentioned. The scores are averaged across all responses.
First choice score equals 100 divided by n, times the sum over the n answers of s sub i.

where

The score s sub i is 1 divided by the brand’s rank if the brand is present in answer i, and 0 if it is absent.
  • No rank below 1st place gets more than 50, so the score is heavily driven by first places.

Browse other categories

Each one is a single question put to every active AI model, many times over.

Or scroll for more

Your category

Ask your own question of every model

Tell us what to ask and how wide to sample it — we run it and send back the report.

Sample size
Display results
0/1000

Sampling every model takes up to 24 hours once we start.