AI editorial benchmark extension: Sol, Luna and DeepSeek
On this page
A DeepSeek V4.1 Flash agent received the original eleven-article commission through CommandCode on 25 October 2026 UTC / 26 October Melbourne. A fresh blind review compared its original submission with the preserved Sol and Luna originals. This page records that extension; the original two-model round remains intact.
Findings and boundaries
| Producer | New review score / 100 | Original-round score / 100 | Production tokens |
|---|---|---|---|
| GPT-6.1 Sol | 94.27 | 92.05 | 3,087,961 |
| GPT-6 Luna | 80.64 | 85.93 | 3,157,774 |
| DeepSeek V4.1 Flash | 73.82 | Not entered | 32,056,365 |
The scores describe frozen originals, not edited publication copies. One Sol judge reviewed all 33 drafts; it shares the winner’s model designation. One run per producer and a different provider/harness/date for DeepSeek limit causal and general claims. No independent human validation, account-specific billed task cost or actual editing-time saving was measured.
- Read the results, findings and limitations
- Read the versioned methodology
- Read the featured Sol pillar
- Read Luna’s comparison archive
- Read DeepSeek’s corrected publication copies
Quality and estimated production cost
USD token-only API-equivalent estimates using rates checked 26 October 2026; not invoices or cost per accepted task. Sol/Luna use OpenAI Standard reference rates; the CommandCode run uses DeepSeek direct-API off-peak reference rates. Provider, harness and production dates differ. No universal value or causal superiority claim is implied.
Download: SVG · PNG · CSV · Link to figure · Full-precision chart data
View chart data (display rounded)
| Producer | Review score / 100 | API-equivalent estimate, USD |
|---|---|---|
| GPT-6.1 Sol | 94.27 | 1.0051 |
| GPT-6 Luna | 80.64 | 0.0639 |
| DeepSeek V4.1 Flash | 73.82 | 0.3040 |
Sources: Frozen scores · Run identities · Reference costs.
Download the production and review evidence
- Shared production brief
- Frozen rubric
- DeepSeek original submission — all 29 files
- SHA-256 hashes of DeepSeek originals
- Blind review
- 162 substantive claim checks and bibliographic/source records
- Per-article scores
- Cluster scores
- Pairwise comparisons
- Blind decision
- Quality freeze record, curated public export
- Identity mapping and efficiency merge
- Editing plan, including estimated effort
- Publication edit log
- Public-export notes and limitations
- Download manifest
The quality archive was frozen before the coordinator combined it with identities and efficiency. The judge remains blinded. The public export removes private runtime locations and session identifiers; it has its own download hashes and is not represented as byte-identical to private review files. The original DeepSeek submission ZIP is byte-exact. Original Sol/Luna packages and their first-round review remain available from the historical evidence hub.
Recorded usage and reproducible estimates
- Production usage and comparison limits
- Judge usage, separate overhead
- Dated token-cost estimates and pricing rules
- Sol request-level usage and calculations
- Luna request-level usage and calculations
- DeepSeek request-level usage and calculations
- Fresh three-way review request-level usage and calculations
Cached input is included in input; reported reasoning is included in output. Both are recorded subsets, never extra additions. DeepSeek’s separate reasoning count and unpersisted verifier/background work are unavailable. The reference estimates exclude setup, tools, coordination/publication, taxes, subscription allocation and invoice reconciliation. They are not cost per editor-accepted publication.
Version identity
The DeepSeek run used the configured deepseek/deepseek-v4.1-flash latest alias through CommandCode. Its served snapshot is unverified. The September V4.1 release and October 1017 update have their own primary release evidence. Neither establishes CommandCode’s served build retrospectively.
Later review · 28 October 2026
The four-way follow-up adds Terra and preserves this three-way round’s scores, estimates, charts and evidence. Its new original-submission cost hold does not establish a defect in the already revised DeepSeek publication copies. Four-way evidence and Terra archive.