DeepSeek’s AI benchmarking cluster: revised comparison archive
On this page
DeepSeek V4.1 Flash delivered a complete eleven-article cluster through CommandCode. Its frozen original received 73.82/100 in the fresh three-way review, behind Sol (94.27) and Luna (80.64). This archive contains publication revisions, including material billing/price-condition repairs and technical qualifications. Those repairs do not change its original score.
The configured model was the latest alias, with no proof of the served checkpoint. This is not labelled as a 1017 benchmark. One Sol judge shares the winner’s model designation, and no independent human validation was measured. Read the full findings and limitations.
Read the complete cluster
- Evaluating AI Models and Agent Systems: A Practical Guide for Technical Teams — master guide
- Public Benchmarks Are a Screening Tool, Not a Decision
- Designing Evaluation Tasks and Datasets That Reflect Your Workload
- Contamination, Leakage and Holdouts: Why Your Benchmark Score May Be Inflated
- Objective Metrics and Rubrics That Measure What Matters
- Human Evaluation and Model Judges: Getting Reliable Judgements at Scale
- Repeatability, Variance and Uncertainty in Model Evaluations
- Comparing Models Fairly: Prompts, Tools and Configuration
- Latency, Token Usage and Cost: The Other Half of the Scorecard
- Evaluating Agents and Tool Use: Model Benchmarks Are Not System Benchmarks
- Interpreting Results and Keeping Evaluations Useful Over Time
Inspect the original and the changes
- Download the scored original, all 29 files
- Original file hashes
- Publication edit log
- Review, scores, claim ledger and usage
- Featured Sol series
- Luna archive and its original-round history
The originals and historical source register are retained as submitted. Publication copies have their own source hashes. Recommendations and hypothetical examples are not measured customer results.