Local AI memory budget worksheet: estimate first, then measure

An “8B at 4-bit” model has an idealised weight-only size of 4 GB. That does not mean it will run in 4 GB of system memory. The calculation excludes model-format overhead, working memory, context, concurrent requests, the operating system and other applications.

The decision is:

Use weight arithmetic as a lower bound, then measure the exact model, runtime and workload on the intended computer.

Do not add a universal overhead percentage. The additional memory depends on the architecture and how the system will be used.

1. Record the candidate

Field	Your entry
Workload
Exact model and version
Source and licence reviewer
Architecture
Total parameter count
Active parameters, if applicable
Quantisation and model format
Downloaded file size
Runtime and version
Computer and operating system
System RAM
GPU or accelerator memory
Free storage

Use total parameters for the basic weight calculation. For a mixture-of-experts model, active parameters may help describe computation, but they do not determine the storage required for all weights. For example, Mistral’s Mixtral description distinguishes total parameters from the subset used per token. The weight-only arithmetic here uses the total; it is not a measured download size.

Review the licence for the exact model version. A family name does not establish the terms for every release or derivative.

2. Calculate the weight-only lower bound

Use:

weight-only bytes = total parameters × bits per parameter ÷ 8

At an idealised four bits per parameter:

Parameters	Calculation	Result
8 billion	8B × 4 ÷ 8	4 billion bytes, or 4 GB
14 billion	14B × 4 ÷ 8	7 billion bytes, or 7 GB
32 billion	32B × 4 ÷ 8	16 billion bytes, or 16 GB

Complete the calculation:

Field	Your entry
Total parameters
Bits per parameter
Weight-only bytes
Decimal GB: bytes ÷ 1,000,000,000
GiB: bytes ÷ 1,073,741,824

GB and GiB are different units. Four billion bytes is 4 GB but about 3.73 GiB. Record the unit used by each file, operating-system reading and hardware specification.

This formula estimates raw weights only. It is not a prediction of downloaded file size, GPU-memory use or total system-memory demand.

3. Account for excluded memory

Component	Why it matters	How to check
Quantisation metadata	Format overhead is not captured by the idealised bit rate: for example, ggml’s Q4_0 definition stores a scale alongside packed values.	Compare the lower bound with the actual file.
Other-precision tensors	Some tensors may use more than the headline bit rate; llama.cpp’s quantisation implementation selects types per tensor.	Review model details and runtime reporting.
Key-value cache	Demand varies with architecture, context, batch or concurrency and cache representation.	Test the approved workload.
Runtime buffers	Inference requires working memory beyond weights.	Measure the running process.
CPU/GPU placement	Memory may be divided across different pools.	Record residency and both memory readings.
OS and applications	The computer must remain stable during normal work.	Compare idle and workload measurements.

The Ollama FAQ documents ollama ps for viewing model residency. It is one runtime-specific check, not a complete memory audit. Hugging Face’s cache guidance describes cache strategies and memory trade-offs. NVIDIA’s TensorRT-LLM memory documentation provides a runtime-specific example of weights, working buffers and KV-cache demand. Use these explanations to plan measurement; they do not establish memory demand for your candidate.

4. Define the workload

Field	Pilot value
Typical and maximum input
Maximum approved context
Expected output
Simultaneous users
Concurrent requests or batch size
Tools or retrieval enabled
Other applications kept open
Acceptable response time
Failure threshold
Acceptable headroom or resource thresholds

A short single-user chat does not establish capacity for longer contexts or concurrent requests. Test the workload you intend to operate, not the largest advertised configuration.

5. Measure representative tasks

Measurement	Idle	Model loaded	Typical task	Worst approved task
System RAM used
GPU or accelerator memory used
Storage used
Context and concurrency	N/A	N/A
Response time	N/A	N/A
Memory error or failure

Before testing, record the workload limits and acceptable headroom or resource thresholds. For each run, record the observation tool, units, peak readings and any measurement gaps. Do not substitute one universal spare-memory percentage.

Record the model file, quantisation, runtime version, operating-system version and test date. Repeat the same fixed tasks after material configuration changes; results from another model or quantisation may not transfer.

Decision record

Choose one:

• Memory fit demonstrated: carry the recorded configuration and workload limits into the parent decision; this is not approval to operate.
• Reduce and retest: lower context, concurrency or model size, then repeat memory and output-quality checks.
• Evaluate another candidate: the tested configuration does not meet the required memory or responsiveness limits.
• Hold: measurements or headroom evidence are incomplete.
• Return to the parent decision: compare hosted operation or retain the existing process if no tested local candidate fits.

A 4 GB weight-only result for an 8B four-bit model is the beginning of the worksheet, not a 4 GB system requirement.

Return the completed memory worksheet to the companion decision guide Running AI locally. Memory fit is one input to that workload decision, not a deployment approval. For later implementation sequencing, see An AI workforce orchestration implementation roadmap.

Source notes

Source material for this worksheet includes the Ollama FAQ and llama.cpp README, compared with the worksheet in the 9 October 2026 selected-section documentary review. These retained living-document sections are not evidence of the installed version; check current guidance for the exact runtime before testing. No model was installed or benchmarked. The calculations are illustrative arithmetic, not measured hardware requirements.