Summary

  • Mistral publishes the provenance rule for a benchmark table rather than a provenance label on each number: across six addresses read on 22 November 2026, one row-level exception is named, in a caption, and no number carries its own source label.
  • The Medium 3 announcement prints both "we use numbers reported previously by other providers wherever available, otherwise we use our own evaluation harness" and "Performance accuracy on all benchmarks were obtained through the same internal evaluation pipeline", with no sentence reconciling them.

Two rules for one table

The Medium 3 announcement, published 7 May 2025, is the clearest case in the corpus (mistral.ai/news/mistral-medium-3). Before its benchmark section it sets out where the numbers come from:

In the evaluations below, we use numbers reported previously by other providers wherever available, otherwise we use our own evaluation harness.

A few lines later it prints the opposite arrangement, as a footnote repeated beneath the results:

Performance accuracy on all benchmarks were obtained through the same internal evaluation pipeline.

Both sentences govern the same rows. The first says part of what follows was produced by somebody else; the second says all of it came through one pipeline. A reader who wants to know which competitor figure was imported and which was re-run in-house has no row to consult, because the page never puts a label beside a number. This is not a contradiction proven. It is two printed statements, one table, and no sentence joining them, and it is the owner of the table who printed both.

The policy travels; the label does not

Small 3.1, published 17 March 2025, restates the same arrangement in different words: "Whenever possible, we show numbers reported previously by other providers, otherwise we evaluate models through our common evaluation harness" (mistral.ai/news/mistral-small-3-1). The vocabulary changed. The design did not. The page files its results under five headings, from text instruct benchmarks through multilingual, long context and pretrained performance, prints one scale note that MM-MT-Bench is scaled between 0 and 100, and says in prose that Small 3.1 "outperforms comparable models like Gemma 3 and GPT-4o Mini". Which of those comparator numbers came from Gemma 3's own publication and which from Mistral's harness is left where the policy put it: in general.

The one time a row gets named

Large Enough, published 24 July 2024, proves the label is available to the company when it wants one (mistral.ai/news/mistral-large-2407). Its MultiPL-E caption reads:

Performance accuracy on MultiPL-E (all models were benchmarked through the same evaluation pipeline, except for the "paper" row)

Four further captions on the same page make the general claim without an exception: code generation, GSM8K and MATH, general alignment, multilingual MMLU. One of them does a second job, which is to print shot counts: "GSM8K (8-shot) and MATH (0-shot, no CoT)". Nothing else in the six documents prints a shot count. That is the whole of the row-level provenance here: one caption, one exception, one named row, and two benchmarks out of the six named groups with a sampling figure attached.

When a procedure is printed, it is printed

Magistral, published 10 June 2025, shows the same selectivity pointing the other way (mistral.ai/news/magistral). It prints a test-time procedure beside a second score for the same model: "Magistral Medium scored 73.6% on AIME2024, and 90% with majority voting @64." Magistral Small gets "70.7% and 83.3% respectively". My arithmetic, not Mistral's: the two Medium figures differ by 16.4 percentage points and the two Small figures by 12.6, and the only condition the page prints between them is a majority vote over 64 samples. No other configuration is printed for either score, so the gap is visible while its full cause list is not. A reader who quotes one of those two numbers without the other is quoting a different instrument than the reader who quotes both.

The page that invites comparison and prints none

The documentation address whose path ends in /benchmark/ is the purest example of the shape (docs.mistral.ai/getting-started/models/benchmark). At retrieval it rendered as Models Overview and described itself as a place to "Explore the full lineup, compare benchmarks, and find the right model for your use case." What the retrieved text holds is a catalog: model families, licenses, versions, deprecation rows. What it does not hold is a benchmark score, a table caption, a harness name, a shot count or an interval. The address invites a comparison. The comparison lives somewhere else, and when a reader arrives at the somewhere else the first thing printed is a policy.

What a buyer can still check

Codestral 25.08, published 30 July 2025, is the one page here that reports deltas instead of scores (mistral.ai/news/codestral-25-08): a 30% increase in accepted completions, 10% more retained code after a suggestion, 50% fewer runaway generations, and two 5% gains, one on instruction following measured against IF eval v8 and one an average across MultiPL-E. A relative improvement means nothing without its baseline, and the retrieved excerpt prints no baseline for any of the five. The page also says the improvements "were validated in live IDE usage across production codebases", which is a different kind of evidence from a benchmark, and the excerpt names no number of codebases, no period and no prior version.

Across all six addresses, the same fields are missing: no evaluation harness is named, no prompt format is printed, no decoding configuration appears beyond that majority vote at 64 samples, no repetition count is given, and no score carries a confidence interval or a variance. Provenance appears as a rule on three pages and as a row label once.

The numeric comparator rows did not render in the retrieved text of any of these pages. That does not make them absent from the pages; it makes them unavailable to me. So this piece makes no claim about an individual score, about which comparator came out ahead, or about which row Mistral ran and which it imported. The absence findings are scoped to the retrieved text of six addresses read on 22 November 2026, not to the pages in every rendering, not to a PDF or an interactive chart, and not to any Mistral page outside this corpus. The Codestral page arrived as an excerpt, and an excerpt is a weaker instrument than a page.

No inspected page names a legal entity as the evaluating party either; they speak in the first person plural, and I am not converting "we" into a corporate signature.

The checkable question is small and sharp. For each row: was this number produced here or borrowed, and on what configuration was it produced? Mistral prints the rule, prints a procedure when it decides to, names one exception in one caption, and leaves the row unlabelled. A comparison table whose rows carry no source label asks the reader to trust the publisher's arrangement rather than inspect it. That is not a scandal. It is a reading instruction that was left out of the table.