# MechanismOS Benchmark Methodology (updated 2026-05-08)

## Scope
- Endpoint: MechanismOS evaluate pipeline (fast/refined modes)
- Objective: barrier and mechanism validation against experimental references

## Tier model
- GOLD:
  - Ground truth: directly reported experimental activation barriers (kJ/mol)
  - Role: primary release gate and external claim basis
- SILVER:
  - Ground truth: Arrhenius-derived activation barriers from measured k(T)
  - Role: secondary robustness/generalization monitor

## Pass criteria
- GOLD:
  - case must satisfy configured direct value tolerance checks on expected barrier fields
- SILVER:
  - case must satisfy configured direct value tolerance checks on derived barrier fields
- Any case with unmet required checks is marked failed

## Reported results
- GOLD: 154/154 passed (100.00%)
- SILVER: 1256/1323 passed (94.9358%)

## Current rebaseline
- Rebaseline date: 2026-05-08
- Public label: Flux Decision Engine
- Runtime basis: Flux Physics comparator, with no exact-reaction benchmark lookup
- GOLD report: `tests/artifacts/mechanism_os_current_check_gold_fluxphysics_20260508f/mechanism_os_ground_truth_report_20260508_154142.json`
- SILVER report: `tests/artifacts/mechanism_os_current_check_silver_full_fluxphysics_20260508f/mechanism_os_ground_truth_report_20260508_154336.json`
- Benchmark rows are validation references only, not runtime lookup entries.

## Dataset provenance
- Experimental-only benchmark rows are used for published claims.
- Source families include:
  - peer-reviewed mechanism/kinetics literature
  - curated experimental repositories (including NIST where available)
  - compiled experimental kinetics datasets
- Per-case evidence metadata is retained:
  - source type, citation, source reference
  - observation type and provenance level
  - DOI/URL/accession when available
- Synthetic sweeps are excluded from release-claim metrics.

## Interpretation policy
- GOLD is the production-quality barrier claim tier.
- SILVER is supporting evidence and physics-improvement discovery input.
- Both tiers are reported on each benchmark cycle.
