AI editorial benchmark extension: Sol, Luna and DeepSeek
On this page
A DeepSeek V4.1 Flash agent received the original eleven-article commission through CommandCode on 25 October 2026 UTC / 26 October Melbourne. A fresh blind review compared its original submission with the preserved Sol and Luna originals. This page records that extension; the original two-model round remains intact.
Findings and boundaries
| Producer | New review score / 100 | Original-round score / 100 | Production tokens |
|---|---|---|---|
| GPT-6.1 Sol | 94.27 | 92.05 | 3,087,961 |
| GPT-6 Luna | 80.64 | 85.93 | 3,157,774 |
| DeepSeek V4.1 Flash | 73.82 | Not entered | 32,056,365 |
The scores describe frozen originals, not edited publication copies. One Sol judge reviewed all 33 drafts; it shares the winner’s model designation. One run per producer and a different provider/harness/date for DeepSeek limit causal and general claims. No independent human validation, account-specific billed task cost or actual editing-time saving was measured.
- Read the results, findings and limitations
- Read the versioned methodology
- Read the featured Sol pillar
- Read Luna’s comparison archive
- Read DeepSeek’s corrected publication copies
Download the production and review evidence
- Shared production brief
- Frozen rubric
- DeepSeek original submission — all 29 files
- SHA-256 hashes of DeepSeek originals
- Blind review
- 162 substantive claim checks and bibliographic/source records
- Per-article scores
- Cluster scores
- Pairwise comparisons
- Blind decision
- Quality freeze record, curated public export
- Identity mapping and efficiency merge
- Editing plan, including estimated effort
- Publication edit log
- Public-export notes and limitations
- Download manifest
The quality archive was frozen before the coordinator combined it with identities and efficiency. The judge remains blinded. The public export removes private runtime locations and session identifiers; it has its own download hashes and is not represented as byte-identical to private review files. The original DeepSeek submission ZIP is byte-exact. Original Sol/Luna packages and their first-round review remain available from the historical evidence hub.
Recorded usage and reproducible estimates
- Production usage and comparison limits
- Judge usage, separate overhead
- Dated token-cost estimates and pricing rules
- Sol request-level usage and calculations
- Luna request-level usage and calculations
- DeepSeek request-level usage and calculations
- Fresh three-way review request-level usage and calculations
Cached input is included in input; reported reasoning is included in output. Both are recorded subsets, never extra additions. DeepSeek’s separate reasoning count and unpersisted verifier/background work are unavailable. The reference estimates exclude setup, tools, coordination/publication, taxes, subscription allocation and invoice reconciliation. They are not cost per editor-accepted publication.
Version identity
The DeepSeek run used the configured deepseek/deepseek-v4.1-flash latest alias through CommandCode. Its served snapshot is unverified. The September V4.1 release and October 1017 update have their own primary release evidence. Neither establishes CommandCode’s served build retrospectively.