{"sources":[{"id":"S1","url":"https://arxiv.org/pdf/2106.09685","title":"LoRA: Low-Rank Adaptation of Large Language Models","publisher":"arXiv; Microsoft authors; ICLR 2022","published_at":"2021-10-16","accessed_at":"2026-11-16","version":"arXiv:2106.09685v2","document_id":"","revision_id":"","limitations":"Original method and experiments; no universal rank or runtime guarantee. Full-text sections 4.1–4.2 read."},{"id":"S2","url":"https://aclanthology.org/2021.acl-long.568.pdf","title":"Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning","publisher":"Association for Computational Linguistics","published_at":"2021-08-01","accessed_at":"2026-11-16","version":"ACL-IJCNLP 2021 proceedings, pages 7319–7328","document_id":"","revision_id":"","limitations":"Proceedings date records the conference start. Intrinsic-dimension results do not establish universal low-rank layer updates."},{"id":"S3","url":"https://arxiv.org/pdf/2305.14314","title":"QLoRA: Efficient Finetuning of Quantized LLMs","publisher":"arXiv; University of Washington authors","published_at":"2023-05-23","accessed_at":"2026-11-16","version":"arXiv:2305.14314v1","document_id":"","revision_id":"","limitations":"Full-text sections 2–3 and Appendices E–G read. Memory and quality results depend on setup. Rounded lab constants do not reproduce production kernels."},{"id":"S4","url":"https://arxiv.org/html/2402.09353v3","title":"DoRA: Weight-Decomposed Low-Rank Adaptation","publisher":"arXiv; NVIDIA and HKUST authors","published_at":"2024-03-05","accessed_at":"2026-11-16","version":"arXiv:2402.09353v3","document_id":"","revision_id":"","limitations":"Sections 3.2 and 4 read. Column convention must be translated carefully to implementation storage layouts; learning-pattern findings are empirical."},{"id":"S5","url":"https://arxiv.org/html/2208.07339v2","title":"LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale","publisher":"arXiv; original research authors","published_at":"2022-11-10","accessed_at":"2026-11-16","version":"arXiv:2208.07339v2","document_id":"","revision_id":"","limitations":"Sections 3.1–3.3 read. Threshold and outlier prevalence concern studied models, not every architecture."},{"id":"S6","url":"https://arxiv.org/pdf/2210.17323","title":"GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers","publisher":"arXiv; ICLR 2023","published_at":"2023-03-22","accessed_at":"2026-11-16","version":"arXiv:2210.17323v2","document_id":"","revision_id":"","limitations":"Sections 3–4 and Algorithm 1 read. A guessed v4 HTML URL failed; evidence comes from the accessible v2 PDF and version metadata. No current kernel performance asserted."},{"id":"S7","url":"https://arxiv.org/html/2306.00978v5","title":"AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration","publisher":"arXiv; MLSys 2024","published_at":"2024-07-18","accessed_at":"2026-11-16","version":"arXiv:2306.00978v5","document_id":"","revision_id":"","limitations":"Sections 3.1–3.2 read. Saliency percentages and generalization findings are conditional experimental observations."},{"id":"S8","url":"https://huggingface.co/blog/4bit-transformers-bitsandbytes","title":"Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA","publisher":"Hugging Face; integration authors including QLoRA authors","published_at":"2023-05-24","accessed_at":"2026-11-16","version":"Dated integration article; saved revision and live article consulted","document_id":"fb4d661faf3878f40ddb941b0e1169e9","revision_id":"207cc8a4c1ef3bbf0a79bc23efa14c5a","limitations":"Used for frozen-base and storage-versus-compute explanation, not current installation commands or hardware compatibility. This overlaps authorship with QLoRA and is not independent corroboration."}],"claims":[{"id":"C1","text":"The explicitly assumed training-state layout totals 16 bytes per trainable parameter; 70 billion parameters give 1,120 GB or approximately 1,043 GiB. Frozen-base-plus-adapter subtotal is 2N + 16P under the stated assumptions.","location":"memory; memory table; Q8","evidence":[],"limitations":"Accounting scenario, not a universal implementation layout or measured device footprint; excludes activations and temporary allocations.","kind":"original_example","basis":"Inspectable arithmetic: 2+2+4+4+4=16; 70,000,000,000*16=1,120,000,000,000; divide by 10^9 or 2^30. For frozen BF16 values retain 2N and assign the assumed training states only to P adapters."},{"id":"C2","text":"Intrinsic-dimension experiments motivate restricted adaptation spaces without proving low rank for every layer or task.","location":"memory","evidence":[{"source_id":"S2","passage":"there exists a low dimension reparameterization","location":"Abstract, page 7319"}],"limitations":"Empirical results in studied parameterizations; matrix rank and intrinsic dimension are different concepts.","kind":"source","basis":"Support qualified to the paper's experimental scope."},{"id":"C3","text":"LoRA freezes the base, trains two compatible low-rank factors, uses Gaussian A and zero B in the original paper, scales by alpha/r, and permits materializing a merged dense weight.","location":"memory; lora; adapter-branches.drawio; Q1","evidence":[{"source_id":"S1","passage":"random Gaussian initialization for A and zero for B","location":"Section 4.1, page 4, equation 3 and following paragraph"},{"source_id":"S1","passage":"perform inference as usual","location":"Section 4.1, No Additional Inference Latency"}],"limitations":"Scaling is a heuristic rationale, not guaranteed rank-invariant optimization. Initialization describes the original paper, not every library option.","kind":"source","basis":"Section 4.1 defines shapes, frozen/trainable split, alpha/r and merge; section 4.2 discusses target subsets."},{"id":"C4","text":"The stated LoRA gradients, initial behavior, merge identity, diagram flow and selected-projection parameter counts follow from h=W0x+sBAx.","location":"lora; adapter-branches.drawio; lab.py; Q1–Q2","evidence":[],"limitations":"Differentiable dense projection with fixed scalar s; no dropout or bias in the example. Floating-point evaluation orders can differ.","kind":"original_example","basis":"Differentiate the C3 equation: dB=s*g*(Ax)^T; dA=s*B^T*g*x^T; dx=W0^T*g+s*A^T*B^T*g. Distribute multiplication for merge. Count four 4096^2 matrices versus four times 16 times 8192 adapter entries. lab.py parameter_counts and merge expose these operations."},{"id":"C5","text":"DoRA parameterizes column-wise magnitude and normalized direction, using a low-rank directional update and permitting a materialized effective matrix.","location":"dora; Q3","evidence":[{"source_id":"S4","passage":"each column of $V/||V||_{c}$ remains a unit vector","location":"Section 3.2, equation 2 explanation"},{"source_id":"S4","passage":"DoRA can be merged with the pre-trained weight before inference","location":"Section 4.1, following equation 5; typography normalized"}],"limitations":"Paper convention; other storage orientations require translation. Empirical learning-pattern similarity is not a trajectory guarantee.","kind":"source","basis":"Sections 3.2 and 4.1 define m, V, column norms, initialization and low-rank update."},{"id":"C6","text":"The synthetic DoRA column example reconstructs (3,4), permits magnitude-only or direction-only changes, and requires treatment of zero norms.","location":"dora; lab.py dora_columns; Q3","evidence":[],"limitations":"Illustrative column convention; rejection of zero columns is this lab's policy.","kind":"original_example","basis":"sqrt(3^2+4^2)=5; 5*(0.6,0.8)=(3,4); 10*(0.6,0.8)=(6,8); 5*(0,1)=(0,5). One m entry per d_in column follows from the declared shape. dora_columns rejects zero norms."},{"id":"C7","text":"QLoRA uses a frozen four-bit base and adapters; NF4 uses normal-distribution-informed nonuniform values including zero; the paper reports a 65B/48GB experiment.","location":"qlora; Q4","evidence":[{"source_id":"S3","passage":"a single 48GB GPU","location":"Abstract"},{"source_id":"S3","passage":"zero-centered normally distributed data","location":"Section 3, NormalFloat definition"}],"limitations":"Normality and quality conclusions are conditional; no arbitrary 70B capacity guarantee or universal optimal-task-error claim.","kind":"source","basis":"Abstract, section 3 and Appendix F support the qualified account."},{"id":"C8","text":"QLoRA double quantizes block scales using 64-weight and 256-scale blocks, and paged optimizer states move between host and GPU memory.","location":"qlora; Q5","evidence":[{"source_id":"S3","passage":"quantizing the quantization constants","location":"Section 3, Double Quantization"},{"source_id":"S3","passage":"automatically evicted to CPU RAM","location":"Section 3, Paged Optimizers"}],"limitations":"Does not imply unlimited capacity, free transfers, or a complete 4.127-bit model allocation.","kind":"source","basis":"Section 3 specifies FP8 scales, block sizes 64 and 256, mean centering and optimizer paging."},{"id":"C9","text":"QLoRA distinguishes base-weight storage precision from computation precision, dequantizes for computation and trains the adapters while keeping the base frozen.","location":"qlora","evidence":[{"source_id":"S8","passage":"The LoRA layers are the only parameters being updated during training.","location":"QLoRA paper section; saved original and live article, paragraph before storage/computation discussion"}],"limitations":"Historical integration explanation; not a universal current-kernel or register-placement description.","kind":"source","basis":"Saved original read_document and live article both contain the storage-versus-compute paragraphs."},{"id":"C10","text":"The ideal nested-scale payload totals 4.126953125 bits per weight, or 8,452 bytes for 16,384 weights under the specified accounting.","location":"qlora; lab.py main; Q5","evidence":[],"limitations":"Omits metadata, offsets, alignment, partial groups, adapters and other model components; lab does not implement double quantization.","kind":"original_example","basis":"Uses C8's cited configuration. 32/64=0.5; 8/64+32/(64*256)=0.126953125. For 16384 weights: 16384/2+256+4=8452 bytes."},{"id":"C11","text":"Merging a reconstructed base with an adapter preserves the projection algebra, but subsequent requantization generally changes the represented matrix.","location":"qlora; Q2","evidence":[],"limitations":"Mathematical distinction, not a measured error or quality bound.","kind":"original_example","basis":"Distributivity gives (What+D)x=What*x+D*x. A finite reconstruction set cannot represent arbitrary sums; lab.py quantize_block visibly maps continuous inputs to 16 possible normalized values."},{"id":"C12","text":"LLM.int8() combines vector-wise INT8 computation with higher-precision outlier contributions; its threshold and prevalence observations are model-dependent.","location":"ptq table and prose","evidence":[{"source_id":"S5","passage":"mixed-precision decomposition for matrix multiplication","location":"Section 3.2"},{"source_id":"S5","passage":"about 0.1% of all feature dimensions","location":"Section 3.2"}],"limitations":"The paper's alpha=6.0 and scale observations are not universal laws. No speedup or all-model quality guarantee is asserted.","kind":"source","basis":"Sections 3.1–3.3 explain vector-wise scales, outlier split and experimental scope."},{"id":"C13","text":"GPTQ minimizes calibration layer-output reconstruction error using inverse-Hessian information, damping, column quantization, compensation and a Cholesky/block formulation.","location":"ptq table and prose; Q6","evidence":[{"source_id":"S6","passage":"updating all not-yet-quantized weights","location":"Section 3, Optimal Brain Quantization background"},{"source_id":"S6","passage":"using a Cholesky decomposition","location":"Section 4, Cholesky Reformulation"}],"limitations":"Local calibration reconstruction is not a universal downstream accuracy or generalization guarantee.","kind":"source","basis":"Equation 1 defines the objective; Algorithm 1 specifies (2XX^T+lambda I)^-1, Cholesky, column quantization and updates."},{"id":"C14","text":"AWQ uses activation-informed channel scaling for weight-only quantization, motivated by saliency experiments; it does not require keeping exactly 1% in FP16.","location":"ptq table and prose; Q7","evidence":[{"source_id":"S7","passage":"per-channel scaling","location":"Section 3.2"},{"source_id":"S7","passage":"without actually keeping them as FP16","location":"Section 3.1, Limitations"},{"source_id":"S7","passage":"does not rely on any backpropagation or reconstruction","location":"Abstract"}],"limitations":"Saliency percentages and generalization results are empirical, not universal. No acceleration ratio is transferred to other hardware.","kind":"source","basis":"Sections 3.1–3.2 distinguish motivating mixed-precision experiments from the scaling method."},{"id":"C15","text":"Equal weight perturbations can have different output effects, and diagonal reciprocal scaling preserves an unquantized product without preserving quantized results.","location":"ptq; Q6–Q7","evidence":[],"limitations":"Original two-coordinate example and algebra; does not implement or benchmark GPTQ or AWQ.","kind":"original_example","basis":"(0.1,0) dot (10,1)=1; (0,0.1) dot (10,1)=0.1. For invertible S, (WS)(S^-1 X)=WX by associativity; finite-grid rounding is not generally equivariant to scaling."},{"id":"C16","text":"The original lab uses deterministic synthetic inputs and standard-library arithmetic, validates merging, DoRA reconstruction, zero blocks and nibble packing, and performs no file, shell, network or model calls.","location":"lab; merge-example; assets/lab.py including comments","evidence":[],"limitations":"Illustrative code; no actual execution result is claimed. Packing format and tie policy are local teaching choices.","kind":"process","basis":"Inspectable complete lab.py imports only math; uses fixed matrices and weights, explicit assertions, min-based nearest reconstruction, bytes/bytearray packing, and print. No source-supplied code was executed."},{"id":"C17","text":"The worked two-dimensional LoRA example yields [5,9.5] by both algebraic routes, with merged matrix [[2,1],[2.5,4.5]].","location":"lab prose, code excerpt and lab.py","evidence":[],"limitations":"Hand-derived prediction for synthetic inputs, not a reported execution trace.","kind":"original_example","basis":"A*x=1; W0*x=[4,10]; 2*B*[1]=[1,-0.5]; 2*BA=[[1,-1],[-0.5,0.5]]. Add to W0 and multiply by x."},{"id":"C18","text":"The teaching quantizer uses rounded NF4 constants and block absmax normalization; it is not exact bitsandbytes NF4 and does not implement double quantization. Index packing excludes scale and object overhead.","location":"lab; Q4; lab.py LEVELS, quantize_block, pack_indices and main","evidence":[{"source_id":"S3","passage":"The exact values of the NF4 data type","location":"Appendix E, page 24, immediately preceding the 16-value list"}],"limitations":"Constants rounded to six decimals; Python arithmetic, custom nibble order and metadata differ from production implementations. No RMSE value is asserted without execution.","kind":"original_example","basis":"S3 Appendix E supplies numerical premises; compare rounded LEVELS with its table. Inspect pack_indices for two indices per byte and records for separately retained scale/count. The lab prints computed RMSE rather than hardcoding the supplied notebook's output."},{"id":"C19","text":"The notebook's scope is mechanism explanation and synthetic arithmetic, not reproduced training or deployment benchmarks.","location":"book; limits; diagram and lab scope statements","evidence":[],"limitations":"Does not establish production quality, memory fit or latency.","kind":"process","basis":"Inspectable article, adapter-branches.drawio and lab.py contain one projection diagram, arithmetic, mechanisms and self-assessment, with no model execution or benchmark apparatus."},{"id":"C20","text":"The package separates cited primary-source method descriptions from synthetic calculations and records source versions, dates, saved identifiers and evidence locations.","location":"limits; references; sources metadata","evidence":[],"limitations":"These inspectable evidence records are not independent expert verification; isolated teaching-code execution is not a model benchmark.","kind":"process","basis":"Inspect sources S1–S8 for publication/access dates, versions and the saved Hugging Face document/revision identifiers; inspect the claim ledger for exact passage locations and separate original-example derivation bases. Inspect the limits disclosure for these boundaries. This claim makes no assertion about historical author tool activity."}]}