Will this model fit in memory?
Weights, KV cache, runtime overhead and OS headroom evaluated against a Jetson module's memory pool for an LLM, VLM, ASR model or vision pipeline. Weights come from a published GGUF/safetensors file size when one exists, otherwise from the model's own parameter count; the KV cache comes from its own layer, head and head-dimension architecture — nothing is guessed. The result says FITS, TIGHT or DOES NOT FIT, the largest context and concurrency that still fit, and how the next quantisation down compares.
Method v1.5.0 · data verified 2026-09-29 · registries: model-memory, hardware-capabilities · methodology
01 · Define workload
Advanced
03 · How this works
Weights are a published number, not a guess
When the registry has a published quantised file (a GGUF from bartowski/unsloth, or the vendor's own safetensors index) the exact artefact size is used — evidence class A. Otherwise weights are parameters × bytes-per-parameter for that quantisation: fixed widths (FP16, FP8) are class A by definition; block quants (INT8/Q8_0, Q4_K_M, NVFP4) use the median measured-artefact-to-parameter ratio across the registry, class D.
KV cache is the model's own architecture
KV cache = 2 × layers × KV heads × head dimension × context tokens × bytes-per-value × concurrent sequences, per NVIDIA's published inference-optimization formula. Layers, KV heads and head dimension are read from the model's own config.json — not estimated from parameter count — so a Grouped-Query Attention model with few KV heads gets a correspondingly small cache. A Custom model asks for the same four numbers directly.
Runtime overhead, OS headroom, reserve
Runtime overhead is an engineering heuristic per runtime — a fixed base plus a percentage of weights for the CUDA/TVM/PyTorch context and compute buffers — evidence class E, stated on every result. OS headroom and reserve use the same shared memory budget as Camera Stream Capacity and Model Match, so all three engines agree on what a module "actually" has available. Vision pipeline mode takes the model-side parts — the registry weight size and the published TensorRT activation and workspace split, or that curve interpolated and labelled class E — from the one shared vision-memory module Model Match also reads, adds the per-stream decode buffers, tracker and deployment overhead, and rebuilds the total on this same budget.
Full formulas, data tables and what invalidates a result: methodology. Static tables: model memory fit by module and quantisation. Raw dataset: datasets/model-memory.json.