Model Match — methodology

Input rules

Inputs (WP-01). A field that is not stated takes its documented default and is listed in provenance.defaulted; a field that is stated but invalid — a count of 0, a negative, a fraction or a non-number, true in place of a count, an unknown value of an enumerated input, or an empty string — is rejected with VALIDATION_ERROR naming the field, never replaced by a default.

Input keys (WP-02). The engine accepts one list of input keys — the inputs documented here and their declared backward-compatible aliases. Unknown input keys are rejected with VALIDATION_ERROR naming the key: an invented or misspelt key is never silently ignored, a __ test hook never reaches the engine, and an alias sent together with the key it stands for is rejected too (the engine would read only one of them). export_json.inputs echoes every input the engine honoured, so replaying it reproduces the result.

Contents

1. What it decides, and what it does not
2. The candidate universe
3. The canonical model map
4. What each candidate is evaluated on
5. Reference accuracy and quantised accuracy
6. The vision memory model
7. The filter, and how UNKNOWN is treated
8. The four terms and the weight table
9. Measured and estimated accuracy
10. Winner, order, roles and ties
11. What the registries actually contain
12. What invalidates a result
13. Method changelog

1. What it decides, and what it does not

Model Match answers one question: on the platform you already have, under this load and these constraints, which model should you run — and why that one and not the next. It returns a ranked list of every model with a benchmark row for that silicon, the four term scores and the weight table that produced each total, the checks behind the winner, every excluded model with the constraint that excluded it, and the accuracy of each candidate with the metric and the evaluation set named.

It does not recommend a platform. When the question is which platform, that is Hardware Match. It does not compute a throughput, a latency or a memory figure of its own either: it owns exactly two things, the candidate universe and the ranking. Everything inside a candidate comes from an engine or a registry that already owns it, and re-running that engine directly with the same inputs reproduces the same numbers — which is what makes the sentence "the winner fits" checkable rather than asserted.

It also does not model the rest of the pipeline. The throughput figures are inference-only; decode, preprocessing, tracking and encoding are Camera Stream Capacity's question.

2. The candidate universe

A model is a candidate on a platform if and only if the benchmark database holds at least one row for it on that platform. The candidate is the triple (model, variant, precision), which is exactly how the database records a measurement, and its universe size is a fact about coverage rather than a design choice: 90 triples on the Jetson T5000, 58 on the Orin Nano Super, 48 on the Orin NX 16GB, 11 on the Orin NX 8GB — and zero on the Jetson T4000, which returns an answer naming the platform and the gap instead of a list.

The alternative rule — admit any model and model the ones nobody measured — was rejected. A modelled throughput is labelled as modelled by Compute Fit and can never confirm a fit, so every added row would be a candidate that cannot win on the strength of its own evidence; and a comparison whose deciding number was invented is worse than a shorter list. A model with no row here is reported in the gaps, never as a silently modelled entry.

Filters narrow the universe before evaluation, never after ranking: a task, a precision, a list of families to keep or to drop. When the task is one of the three language tasks — text generation, vision + language, speech recognition — the universe is instead the class A model-memory registry of that type at the quantisation asked for, because a language-model question on an edge module is a memory question first and the registry, not the benchmark rows, is what carries sourced memory facts for it.

Above max_candidates (default 40, cap 90) the universe is truncated by benchmark-row count, before any scoring, and the truncation is reported. Truncating after scoring would let the cap silently decide the answer.

3. The canonical model map

The benchmark database records the same checkpoint under more than one id, because it was assembled from sources that spelled models differently. YOLOv8n-seg has its throughput rows under yolov8_seg|n and its 0.367 mAP under yolov8_seg|n-seg. SSD MobileNet v1 appears under three family spellings. EfficientNet-EdgeTPU appears under three more. Without a map that joins them, one candidate has speed and no accuracy while a phantom second has accuracy and no speed, and neither can be compared to anything.

The map lives in the model-facts registry and is published in the dataset. Every entry names the non-canonical spelling, the canonical model and the reason the two are the same checkpoint; no entry points at another entry, and no canonical model is itself the source of an alias. After the map is applied, the merged candidate carries every row from every member spelling, and the result reports how many of its candidates are joins.

The map is Model Match's alone. The benchmark database is not rewritten by it, so no other engine's answers move because of it.

4. What each candidate is evaluated on

Every constraint is stamped with the engine that asserted it and the check it answers, so a reader can always say which engine is responsible for any number on the page. Nothing is recomputed here, and there is no constraint type that only this engine produces.

5. Reference accuracy and quantised accuracy

These are two different measurements and this engine never merges them.

The reference accuracy is the figure the model's own publisher puts on its model card, with the metric and the evaluation set named. For the detector families that is mAP50-95 on COCO val2017. Naming the metric is not decoration: it is what resolves the contradiction this engine was built on. YOLO11n has 0.395 published on the Ultralytics model page and 0.5085 published in the same publisher's Jetson guide. Eleven points apart is far more than quantisation explains, so they cannot be the same metric on the same set — and the guide does not say which set it used. The model card figure is the reference; the guide figure is recorded as a per-platform observation whose dataset is unstated, flagged as conflicting, and never presented as the model's accuracy.

The quantised accuracy is what was observed at the precision asked for. It is resolved in a fixed order: the benchmark row for this exact platform and precision if one exists; otherwise a per-platform figure the benchmark database dropped in its de-duplication; otherwise the publisher's own quantisation table, which also carries the drop in points, the quantisation method and the calibration set. Where the published drop at the requested precision exceeds two mAP points, the result names it in a warning — the ranking uses this quantised deployment figure, so the loss is priced into the score, and the reference stays on the row for context. A per-platform figure more than five points from the model card is withheld from the ranking: it is stated with its delta in the result's gaps (and in a warning when that model wins), and the model's accuracy term takes the unknown contribution (§9) — the reference is never substituted for it.

Evidence class. A vendor model-zoo or model-card accuracy table is an external measured benchmark: class C. Class B is reserved for EdgeAIStack's own measurements and class A for a vendor specification, so no accuracy figure on this page claims either. That is checked, not merely intended: a reference accuracy claiming class A or B fails a rule.

6. The vision memory model

Vision memory is built here rather than taken from the legacy memory estimator, because that estimator assigns a documented total peak figure to its weight term and then multiplies it again — a six-fold over-count on the one row backed by a real measurement (logged as engine bug #18, owned by the package that rewrites its consumers). Model Match never reads that table.

Instead the budget is:

The third row is the honest limitation: it is a five-point curve applied to architectures it was not measured on, which is an engineering heuristic and is labelled as one. That is why the size term is never evidence for a vision candidate, and why the memory constraint on such a candidate reports class E rather than borrowing the class of the split it was interpolated from. A candidate with no parameter count at all reports memory UNKNOWN rather than a guess.

Verdict: FITS below 85 % of module memory, TIGHT to 100 % inclusive, DOES NOT FIT above it — the same thresholds Model Memory Fit uses.

7. The filter, and how UNKNOWN is treated

A candidate is excluded if and only if a compute check FAILs or is UNSUPPORTED, or its memory check FAILs. That is the whole filter. There is no accuracy floor, no size ceiling, no throughput margin requirement and no preference anywhere in it: a preference changes the order of the ranked list and never its membership, which is a property a rule checks on every combination by re-running the same question at all four preferences.

Every excluded candidate is returned with the exact constraint that excluded it — the check, its status, what was required, what was available and the first assumption behind it — so an absence from the list is always explained.

A FAIL is only raised on evidence that can prove the miss (elimination authority). A candidate whose compute or memory miss rests on planning-only evidence — the multi-stream efficiency curve, a power-mode factor, a scaled or corpus-estimated throughput, the estimated activation and workspace memory — is not excluded: it is a planning candidate, planning_only: true, NEEDS VALIDATION with the modelled shortfall stated on its demoted row. Planning candidates are listed after every candidate without such a miss, and they anchor no cohort term (the accuracy range and the size anchor come from the others), so their presence never re-scores the rest. A planning candidate's own accuracy term reads against those anchors by the same groups: a measured figure beside measured anchors takes their floored min–max, held between the unknown contribution and 1; anything the anchors' group cannot rank takes the unknown contribution. coverage.planning_only counts them.

UNKNOWN never excludes anything and never passes anything free. A batch size nobody measured, a precision with no row, a model with no parameter count: each comes back UNKNOWN, caps the candidate's verdict at NEEDS VALIDATION — which ranks it after every FIT and FIT WITH RISKS candidate, whatever its score (§10) — and counts as class E in the evidence term. A candidate whose throughput and memory are both UNKNOWN is thin and sorts after every candidate with coverage in its tier, so it can only reach the top when nothing else has coverage either — which is what makes "the winner is the top of the list" safe.

Only a material check lowers a verdict. A candidate's verdict is the shared kernel's rollup of its own checks. A check the kernel reads as informational — a self-comparison such as "a row exists at the batch you asked for", or a catalog assertion — never lowers a verdict through its boundary, and a question the caller did not ask is not part of the verdict at all. These are the checks that can come back not established, and why each one decides something:

The precision check and the catalog runtime check are PASS or UNSUPPORTED, never not established. Accuracy, parameters, weight size and licence are not checks: a missing accuracy changes the accuracy term (§9), never the verdict.

8. The four terms and the weight table

Score is a plain weighted sum, Σ wᵢ·sᵢ, over four terms in [0,1]. No multipliers, no overlays, no per-preference code path. Each term is published beside the score, so the arithmetic can be recomputed from the numbers on the page — a ranking that cannot be checked from what it prints is not falsifiable.

The weight table below is class E — engineering judgement, not measurement. It is the only weight table in the engine: there is no overlay, no multiplier and no preference-specific branch anywhere else, and the applied row travels on every result so it can be read next to the scores it produced. Each row sums to 1.0.

The evidence weight never drops below 0.20 in any row. That is deliberate: a preference is a statement about what you want, not permission to prefer an unsourced number over a sourced one.

9. Measured and estimated accuracy

A deployment accuracy is either measured — a publisher quantisation table or a published per-platform figure, class A, B or C — or estimated — an EdgeAIStack benchmark-corpus estimate, class D or E. Measured accuracy outranks estimated accuracy as evidence, not as numeric quality: inside a task-and-metric cohort the two are separate groups, and an estimate is never ranked against a measurement on its number. Ranked figures sit on a floored scale from the unknown contribution (the weakest) to 1 (the strongest), so a figure that cannot be compared never sits above a measured one.

The unknown contribution is 0.25, class E. It is the class E evidence weight — the value the evidence term already gives an UNKNOWN check — and it is the floor of the accuracy scale: every ranked figure scores between 0.25 and 1, so an estimate, or a candidate with no comparable figure, never ranks above a measured figure on accuracy, and no accuracy term is ever 0. It never depends on the candidate's own number, so an estimate can never out-score a measurement because the estimate is larger. The price is stated, not hidden: between two measured models the accuracy spread is three quarters of the accuracy weight rather than all of it, so accuracy weighs a quarter less against margin, evidence and size than it did before method 1.6.0. The constant travels on every result as weights.accuracy_normalization.unknown_contribution with range [0.25, 1], normalisation version 3.

Every ranked candidate carries accuracy_group: the evidence group of its figure (measured, estimated or none), its cohort and cohort_basis — what that cohort was ranked on (measured, estimated_one_method, estimation_methods_differ or no_figures) — whether its term is min_max or the unknown_contribution and why, and on an estimate estimated_measurement_needed: true with the label "estimated — measurement needed" and the estimation method. Each estimate is also named in the result's gaps, and an estimate is never labelled most_accurate while a measured figure is ranked.

10. Winner, order, roles and ties

The ranked list is ordered by verdict tier first — every FIT candidate, then every FIT WITH RISKS one, then every NEEDS VALIDATION one — and by the published score inside a tier; a planning candidate (its miss rests on planning-only evidence, #263) follows every other candidate. Score is authoritative within a tier and never across tiers: a candidate with a material check left UNKNOWN never outranks one that fits, whatever its score. A preference moves candidates inside a tier, never across one. NO FIT candidates are excluded, not ranked.

The winner is ranked[0], full stop. There is no override, no re-pick and no input by which a caller can choose differently; a caller who dislikes the winner changes the preference or the constraints, not the result. The result's verdict is the winner's verdict, and the winner is never NO FIT.

When every evaluated model is excluded, the answer is NO FIT, not an error: the response is a success with verdict: NO_FIT, winner: null and an empty ranked list, every evaluated model in excluded with the constraint that excluded it, and questions naming the changes that can help — lower the load, relax a latency target, drop a precision or family filter, or ask Hardware Match which platform holds the load.

When the records hold no candidate at all — a platform with no benchmark row, a task or precision with no row on it, a family filter nothing on it matches — the answer is an evidence gap, not a NO FIT: nothing was shown not to fit. The response is a success with verdict: NEEDS_VALIDATION (insufficient evidence), winner: null, empty ranked and excluded lists, a refusal row whose binding_inputs names the field that left the universe empty, and questions that lead to a value with evidence — a platform with rows for the task, a task or precision on record for the platform, dropping the filter, Compute Fit for a named model, or Hardware Match. The page keeps offering those platforms and tasks and shows this answer. Only a malformed request, or a family or variant name no record carries anywhere (a 400 that names the field and the accepted values), is a request error.

Roles are labels on the ranked list, not a second selection pass. fastest and smallest mark the true maximum or minimum of the feasible set; most_accurate marks the highest measured deployment accuracy — the highest estimate only when no measured figure is ranked and the estimates share one method — and each may land on the winner itself. stretch marks a candidate that is at least a tenth more accurate than the winner but whose verdict is FIT WITH RISKS or NEEDS VALIDATION — the model worth measuring before you settle; on an estimate it sits beside estimated — measurement needed. A candidate may carry several roles or none.

The whole order is deterministic: verdict tier; inside a tier, coverage (thin candidates last — only the NEEDS VALIDATION tier can hold one); then score; then higher evidence, the accuracy standing (a measured figure before an estimate before none, then the higher figure inside the same group), lower total memory and the stable candidate id. A planning candidate (a miss that rests on planning-only evidence, #263) is listed after every other candidate — it can only be NEEDS VALIDATION — and anchors no cohort term. Two identical calls return byte-identical results.

11. What the registries actually contain

• A reference accuracy is on record for roughly half of the candidate universe. The model-facts registry carries the model-card figure for the detector families whose publishers print one and for YOLO11-seg and YOLO11-pose; classification, the other pose and segmentation models and the transformer tail mostly have none. The accuracy term reads the deployment accuracy at the precision asked for, never the reference, so a model with no quantised figure — every YOLO11-seg and YOLO11-pose candidate included, since their card does not state the precision it was validated at — takes the unknown contribution on accuracy and competes on margin, evidence and size.
• Licence is not on record for any model. A licence filter therefore has nothing to filter on, and the field is null everywhere rather than guessed.
• Speech-recognition word error rate and language-model quality scores do not exist here at all. A language candidate is ranked on evidence and size.
• Weight size is a parameter count times a byte width for every vision candidate — a model, not a published artefact size, and labelled class D on every row.
• The TensorRT memory split is published for five detector variants. Every other vision candidate's activation figure is that curve interpolated, class E.
• Per-platform accuracy figures often do not name their evaluation set. Those are recorded with dataset: unstated and flagged when they conflict with the model card by more than five points.

12. What invalidates a result

• Your own evaluation set. Every accuracy here is COCO or ImageNet. Relative ordering on a public benchmark is a weak predictor of relative ordering on your scenes, your classes and your lighting; if accuracy is the deciding term, evaluate the top two on your own data.
• Quantisation you have not done yourself. The published INT8 drops assume the publisher's calibration set and method. A different calibration set, or post-training quantisation where the publisher used quantisation-aware training, can move the figure by several points.
• The vision memory estimate. Outside the five variants with a published split it is a class E interpolation, so a TIGHT verdict there is a hypothesis, not a measurement.
• Everything Compute Fit's own caveats cover. Software versions, thermal state, input resolution, batch, the multi-stream efficiency curve and the fact that every benchmark row is inference-only.
• The weight table. It is judgement. A deployment with a hard accuracy floor or a hard memory ceiling should treat those as constraints on the question — narrow the precision, exclude the families that cannot meet them — rather than hoping a weighted sum expresses them.
• Coverage. A model that nobody has benchmarked on your silicon is absent, not rejected. Absence from this list is not evidence against a model.

13. Method changelog

Method changes bump the method version; the composite dataset version follows the newest of the registries this engine reads, and every result also carries the per-engine versions in sub_engines.