AI EDITORIAL BENCHMARK — SHARED PRODUCTION BRIEF v1.0 Date: 22 October 2026. Publisher: Digital Sanctum, digitalsanctum.com.au. OBJECTIVE Act as an editorial team and deliver a complete, coherent article cluster about AI model benchmarking for technical founders, engineering leads and practitioners choosing or evaluating models. Help readers understand benchmarks, design their own evaluations and interpret results. Exercise your own editorial judgment over titles, angles, cluster structure, research and examples. SCOPE AND COMPLETION Produce exactly one pillar article (1,800–2,200 body words) and ten supporting articles (1,000–1,500 body words each). Count article prose and exclude metadata, navigation and reference lists. Full drafts are required; outlines and briefs alone do not satisfy the task. Use Australian English and clear, concrete language. The reader should be able to act on the explanations. Avoid sales padding and claims about Digital Sanctum projects, customers or results that are not established in the brief. WORKFLOW Complete research, cluster planning, article briefs, full drafting, fact checking and one final editorial pass. Keep planning proportionate to producing all articles. Use the same model throughout. Perform the editorial roles yourself; do not spawn agents, sessions or delegate to other models. If essential information cannot be verified, label the limitation and finish the remaining deliverables. Do not stop after research or planning. Stop after the complete handoff and checks below; additional rounds of speculative expansion are outside scope. COVERAGE Determine a coherent set of distinct reader questions. The cluster as a whole must address public benchmarks versus workload-specific evaluation; task and dataset design; datasets, holdouts, contamination and leakage; objective metrics and rubric design; human evaluation and model judges; repeatability, variability and uncertainty; prompt/tool/configuration fairness; latency, token usage and cost; agent/tool-use evaluation; interpreting results and maintaining evaluations over time. These are coverage requirements, not prescribed article titles. Address tradeoffs and failure cases. Distinguish a model benchmark from evaluation of an entire agent system. RESEARCH AND EVIDENCE Both contestants receive live browsing and the same starting references: - https://developers.openai.com/api/docs/guides/evaluation-best-practices - https://learn.chatgpt.com/docs/long-running-work Use primary sources: original papers, benchmark maintainers and official documentation. Discover additional sources yourself. Verify changing claims such as model specifications, availability and pricing as of the research date. Cite material factual claims near the relevant text. Sources must actually support those claims, not merely mention the topic. Distinguish verified facts, editorial recommendations and illustrative examples. Label hypothetical measurements explicitly. Do not fabricate citations, benchmark results, search volumes, customer examples or quotes. Record source URLs, titles, access dates and what each source supports. Practical examples should be specific enough to follow without implying they are measured experimental findings. OUTPUT LOCATION AND FILES All deliverables are repository-local drafts in benchmark/submission/. This explicit repository destination applies to the article files and supporting documents. Do not save deliverables into connected Pages, change site source, build the site, publish, deploy or send messages to others. - editorial-strategy.md: audience, positioning, scope, cluster rationale and exclusions. - cluster-map.json: each article's stable ID, role, title, proposed slug, target query, search intent, reader question, unique purpose, proposed meta description, related article IDs and relative article file path. Exactly one pillar and ten supporting records. - briefs/: one brief per article covering reader intent, outline, distinct contribution, examples, evidence and internal links. - articles/: eleven complete Markdown article drafts with front matter for ID, role, title, proposed slug, description, target query and body word count. Use relative links that resolve within the submitted cluster. Metadata must not disclose which model produced the cluster; attribution will be added separately after review. - source-register.json: traceable source records and article/claim mappings. - internal-link-map.json: source article ID, destination article ID, anchor/context and purpose. - publication-sequence.md: ordered release plan and rationale. - editorial-handoff.md: contents, coverage, word counts, factual checks, unresolved issues and disclosure of anything not verified. - completion.json: write LAST, with status ready_for_independent_review or incomplete, completed_at_utc, article_count, list of files, per-article body word counts, unresolved issues and checks performed. Be truthful if incomplete. A complete marker is not a quality score. CHECKS AND INDEPENDENCE Check exactly eleven complete drafts exist; their stated word counts are honest and within range; metadata and article IDs agree; internal links resolve; articles have distinct purposes; source claims are supported; examples and terminology are consistent. These editorial checks are requested. Software tests, dependency installation and changes to the website are outside this content-only task. Do not inspect another contestant's worktree, conversation, drafts or output. Do not inspect the coordinator's private run registry or review package. Do not score or compare contestants or declare a winner. One independent reviewer will do that later. Token use is collected externally from runtime records. Never estimate token usage from word counts, invent usage figures or edit the external usage records. Report any tool outage, human intervention, model switch or unexpected interruption in the handoff. SCORING, DISCLOSED BEFORE PRODUCTION Factual accuracy and source quality 25%; cluster strategy/coverage/distinct purposes 20%; practical usefulness/depth 20%; writing/editorial quality 15%; cross-cluster consistency/internal linking 10%; search intent/metadata 5%; completeness/instruction following 5%. All substantive scores need evidence. Length alone earns no credit. Materially incorrect guidance, fabricated evidence and missing full drafts are publication blockers regardless of aggregate score. PUBLICATION OUTCOME The independently selected winning cluster is intended for a featured series on the Digital Sanctum blog. The runner-up is intended for a dedicated benchmark archive elsewhere on the same site. Both require factual checks and editorial approval. Original scored submissions will be preserved, and publication edits disclosed. This session is commissioned to produce drafts only. The methodology article and the comparative results are separate coordinator/editorial deliverables; do not add them as extra contestant articles.