Focused guide / AI Knowledge Hub
What is AI model quantisation?
Quantisation represents an AI model’s numbers with reduced precision, often using fewer bits per value. A bit is a unit of digital information. Fewer distinct numerical values can be represented, so some original values are approximated. Memory, speed and output quality need separate checks.
Selected primary sections reviewed 10 October 2026
On this page
What changes
Quantisation can reduce storage for model weights—the learned numbers used in calculations.[Q1] Some methods also quantise activations, the intermediate values produced during calculations. Ask which values a “4-bit” label covers, and which method was used.
Quantisation itself does not add knowledge, verify answers or settle permissions.
A bounded analogy
Teaching analogy—not a description of model operation. Recording measurements to the nearest centimetre rather than millimetre loses detail: nearby measurements may become indistinguishable.
This illustrates coarser representation, not which answers will change. Actual methods use numerical mappings; some retain higher precision for selected calculations.[Q1, Q2]
Weights versus runtime memory
Illustrative arithmetic only—not measured file size, memory or performance. Suppose an imaginary model has eight billion weight values, represented uniformly:
| Representation | Weight-only calculation | Idealised size |
|---|---|---|
| 16 bits per value | 8 billion × 16 ÷ 8 | 16 GB |
| 4 bits per value | 8 billion × 4 ÷ 8 | 4 GB |
Eight bits make one byte; GB means one billion bytes.[Q5] This excludes metadata and higher-precision values. It does not establish that a model will run in 4 GB.
Runtime memory is more than weights. NVIDIA’s TensorRT-LLM documentation identifies activations, runtime buffers and cache allocations alongside weights—a runtime-specific example, not a universal allocation formula.[Q6]
A key–value cache stores intermediate information reused during text generation. Memory behaviour depends on the model and cache strategy; test longer contexts and simultaneous requests.[Q3, Q6] Context is the material available during a request; the runtime is the software executing the model. Measure system memory and GPU memory—memory on a graphics processor—separately where applicable.
Speed and quality
Quantisation can reduce memory transfers and enable efficient lower-precision calculations on compatible hardware. Speed depends on hardware support, runtime implementation, conversion overhead and workload.[Q1]
Numerical differences can affect quality; test the chosen model and method on relevant work.[Q1] The GPTQ version 2 abstract reports low accuracy degradation in its tested settings.[Q4] This is not a guarantee for another model, runtime or firm.
Readable output is insufficient evidence. Check qualifications, dates, amounts and references that matter to decisions.
Checks before adoption
Treat the model as a candidate configuration, not an automatic replacement.
| Check | Record or test |
|---|---|
| Identify files | Publisher, base model (starting model), revision (specific version), quantisation method and runtime version. |
| Compare work | Approved tasks or invented test cases without client information; expected answers; a higher-precision comparison version tested on the same work where available. |
| Check errors | Missing exceptions, changed figures, unsupported claims and incorrect citations. |
| Measure use | Peak memory, time to first output, completion time, failures, context length and simultaneous requests. |
| Set limits | Acceptance limits defining satisfactory results, human review, owner and a tested fallback—an alternative when the candidate fails. |
| Review responsibilities | Exact licences; lineage (upstream models and transformations); approved data, access, retention, logging and deployment arrangements.[Q8, Q9] |
Record the digest algorithm and value—a checksum for comparing file bytes, not proof of publisher authenticity or safety.[Q7]
Start with non-sensitive material. Fitting weights on a computer does not authorise client-data use. Quantisation does not clear licensing, privacy, security or deployment duties. OAIC guidance supports suitability checks and human oversight; applicability and exact licence permissions still require assessment.[Q8, Q9]
Related reading
Technical next steps:
Return to the broader decision: Open weights, closed weights and the rights your firm needs. For exact-file permissions, use Read the licence attached to the exact weights.
Sources and boundaries
[Q1] Hugging Face: Quantization concepts — “lower-precision data types” supports the definition; “on compatible hardware” qualifies speed.
[Q2] Hugging Face: Bitsandbytes — “dynamically preserves higher precision for critical computations” describes LLM.int8(), its named 8-bit method, not every method.
[Q3] Hugging Face: Cache strategies — “stores these calculations so they can be reused without recomputing them” supports cache reuse.
[Q4] Frantar and colleagues: GPTQ version 2 abstract — “reducing the bitwidth down to 3 or 4 bits per weight, with negligible accuracy degradation relative to the uncompressed baseline” is an attributed result, not independently verified here.
[Q5] NIST: Binary prefixes — “8 bit” and “1 000 000 000 B” support byte and decimal-GB units.
[Q6] NVIDIA: TensorRT-LLM memory — “weights, internal activation tensors, and I/O tensors” supports this runtime’s broader memory inventory.
[Q7] NIST: Hash value — “The result of applying a hash function to data” defines the digest, not trustworthiness.
[Q8] Hugging Face: Licences — “seek out and respect a project’s license” supports checking applicable terms.
[Q9] OAIC: Commercial AI privacy guidance — “conduct due diligence to ensure the product is suitable to its intended uses” supports adoption checks, not a firm-specific legal determination.
Q1–Q4 reuse QNT-1.1 writer captures from 10 October 2026: Q1–Q3 were partial/truncated; Q4 was abstract-only. Q5–Q9 were newly read for this revision. No complete-document review is implied. The arithmetic and analogy are teaching illustrations; no model was benchmarked.