Training, Fine-Tuning and Retrieval: When Not to Train
Chapter 13 designed the serving of a model that already exists. This chapter concerns the step before it: whether a model, or a change to a model, has to be trained at all. It belongs to E3 Design, the function that shapes the solution for less, and it applies the first three levels of the optimisation hierarchy of Chapter 4 to a decision that organisations take more often than they record: necessity (do not run what is not needed), sufficiency of model (the smallest approach that meets the requirement) and efficient serving (retrieval instead of training). The approaches range from a fixed rule to training from scratch, and their footprints differ in kind: some are paid once, some on every use, some on every change of the documents.
The chapter defines the approaches, counts what training and retrieval cost at the boundaries the published figures use, and gives a rule for deciding between them. In the evidence read for this book (November 2026) there is no measured, like-for-like comparison of the energy of retrieval against fine-tuning for the same task at the same quality. The decision rule follows the method of Chapter 4: Count the training once, allocate it to the uses and the period it serves, and Compare the sum with the alternative. The chapter ends in a training budget set by someone other than the proposer, measured outside the job and backed by a stop path someone can use (Cap).
The chapter builds on AI Platform and Security, Chapters 7 and 10, on the model lifecycle as code and on retrieval and grounding (27, Nordqvist, 2026), and on AI Governance, Chapter 5, on readiness for enterprise AI (26, Nordqvist, 2026). It adds the environmental cost to those decisions and does not repeat them.

Figure 14.1: The ladder of approaches. Each arrow down the ladder is taken only when the rung above fails the acceptance test on its own data. The one-off cost rises down the ladder; the cost per use may rise or fall. The decision record and the budget with its stop path sit beside the ladder. No figure is carried by the diagram.
A training run nobody needed
The organisation and all figures of this scene are illustrative and belong to the fictional Nordvik Region. On a Friday in October 2032 the platform team at Nordvik Region started a fine-tuning job on the reserved accelerators at the hosting provider (NR-2026-031). The casework unit had asked for the casework variant of the staff assistant to "know the Region's rules", and the quickest route was to adapt the open-weight model on the regulations and guidance. The job took eight accelerators for 14 hours. Nobody had to approve it: the pool was reserved and, at an average utilisation of 0.58 in the third quarter, its accelerators looked free.
On Monday the evaluation of the alternative came back. The same model, given the relevant paragraphs of the rules at question time by a retrieval step and told to cite them, answered 93 per cent of 300 caseworker questions correctly with a citation, against a threshold of 90 per cent set by the caseworker lead. The unadapted model without retrieval reached 71 per cent. The fine-tuned model, evaluated later, reached 84 per cent and could not say which paragraph an answer rested on. The adapted weights were not used.
The energy of the run is small and is stated at the boundary of the Region's own method: eight accelerators for 14 hours is 112 accelerator-hours; at the 0.52 kW per accelerator that Chapter 8 assumed for the pool and the provider's power usage effectiveness (PUE) of 1.21, the operational energy at the facility boundary is about 70 kWh, or about 2.3 kg CO2e at the Swedish location-based factor of 32 g per kWh (illustrative, 2032). That counts the accelerators at full draw; the increment over idle draw is smaller and was not measured. The size is not the lesson. No gate stood between a wish and a job, the cost was invisible because the capacity was reserved, and a test costing a few hours of one accelerator would have shown the job to be unnecessary. The rest of the chapter builds that gate.
The ladder of approaches
An approach is the way a use gets the behaviour it needs from an AI model. Six rungs cover the choices.
No AI, or rules. A fixed rule, a template, a search or a form does the work. It is the necessity level of the optimisation hierarchy.
Prompting an existing model. The model is used as it is. The task is described in the request, perhaps with a few worked examples. Nothing is trained, and the cost is entirely per use: every request carries the instructions and examples again.
Retrieval. In retrieval-augmented generation, a retrieval step searches the organisation's own documents for the passages relevant to a question and puts them into the request, so that the model answers from them (141, Lewis et al., 2020). An embedding is a list of numbers that represents the meaning of a passage, so that passages with similar meaning lie close together; an index is the store in which the embeddings are kept and searched. The documents stay outside the model and can be changed, cited and deleted without touching it.
Adapter fine-tuning (parameter-efficient). Fine-tuning continues the training of an existing model on the organisation's examples. A parameter-efficient method freezes the original weights, the numbers that make up the model, and trains a small added set. Low-rank adaptation (LoRA) is the best known: for a model of 175 billion parameters its authors report that it cut the trainable parameters by 10,000 times and the GPU memory needed by three times against full fine-tuning with the Adam optimiser, with model quality on par or better and no added latency at serving (139, Hu et al., 2021). A later method, QLoRA, combines it with a compressed (quantised) copy of the model and fine-tunes a 65-billion-parameter model on a single GPU with 48 GB of memory (140, Dettmers et al., 2023). Both papers report parameters and memory, not energy.
Full fine-tuning and continued pre-training. All weights are updated, on the organisation's examples (fine-tuning) or on a large body of its text (continued pre-training). The memory, the hours and the energy are those of training a model of that size on that data.
Training from scratch. The organisation trains a model of its own from random weights. It is the largest one-off cost on the ladder, and few European enterprises or public authorities need it: a new modality, a language or domain no available model covers, or a requirement none can be trusted to meet.
Table 14.1 sets the rungs side by side, as kinds of cost and not as figures.
Table 14.1: The six approaches and where their cost falls.
| Rung | What changes | One-off cost | Cost per use | Choose when |
|---|---|---|---|---|
| 0 No AI or rules | Nothing is learned | None | Small and fixed | The task is regular and the rule can be written |
| 1 Prompting | The request | None | Longer requests | The model already meets the acceptance test |
| 2 Retrieval | The context, per request | Embedding and indexing | Query and retrieved text | The knowledge is documents that change or must be cited |
| 3 Adapter | A small set of weights | A short training run | May fall, with a smaller model or shorter prompt | Behaviour, format or style, not facts |
| 4 and 5 Full training | All weights, or a new model | Large | May fall further | No cheaper rung meets the test |
The rungs are an order of testing, not a ranking of quality. A use starts at the top, runs the acceptance test on its own data, and moves down only when the test fails. The test is the one the benefit record of Chapter 9 already requires: a measure of whether the use does what it is for, on the organisation's own cases, with a threshold set by the person who owns the outcome before the test is run. A threshold set after seeing the results is a choice of the answer.
The ladder orders the one-off cost, not the total: a fine-tuned small model can cost much less per use than a large prompted one, as the measurements below show. The rungs also combine; each combination has its one-off and per-use terms counted separately.
What training costs, counted
The cost of training is the best documented cost on the ladder, but the published figures differ in what they count. Table 14.2 gives the figures this chapter uses, each at its boundary.
Table 14.2: Published and measured training figures (retrieved November 2026).
| Figure | Boundary | Year | Source | Type |
|---|---|---|---|---|
| Llama 3.1 8B and 70B: 1.46 and 7.0 million GPU hours; 420 and 2,040 t CO2e | Training only; H100 at 700 W thermal design power; location-based; market-based total 0 t | 2024 | Meta model card, source 144 | Vendor-reported |
| Llama 3.1 405B: 30.84 million GPU hours (emissions in Chapter 5); all three: 39.3 million, 11,390 t | Training only; H100 at 700 W thermal design power; location-based; market-based total 0 t | 2024 | Meta model card, source 144 | Vendor-reported |
| BLOOMz-7B: training 51,686 kWh; fine-tuning 7,571 kWh | Training from BLOOM footprint paper; fine-tuning from logs at design power, an upper bound | 2023 | Luccioni et al., source 131 | Independent |
| A research model: final run 120 GPU hours; all 4,789 development jobs 239,942 GPU hours | GPU hours from one project's logs; 2018 hardware (Titan X and M40 GPUs) | 2019 | Strubell et al., source 143 | Independent |
Three readings matter for a deployer.
The same run has two honest carbon figures. The Llama 3.1 card reports 11,390 tonnes of CO2e on the location-based method and zero on the market-based method for the same 39.3 million GPU hours (143, Meta, 2024). Chapter 5 explains the two bases and Chapter 22 the reporting rule; a training figure without its basis cannot be used. The card does not say whether experiments, failed runs and earlier versions are inside the figure, so it is read as the cost of the released runs and not of developing the model.
Fine-tuning can be a small part of the total, and the total is counted once. For BLOOMz-7B the independent estimate of the fine-tuning energy is 7,571 kWh against 51,686 kWh for the training, about 15 per cent (130, Luccioni et al., 2024). The authors state that neither model family reported its own total training energy and that the fine-tuning figure assumes full utilisation. The reading for a deployer is that the footprint of a model is the sum of its training and its adaptation, each known at a different quality.
Development is a multiple of the final run. Strubell and colleagues logged the development of one research model: 123 small hyperparameter searches, 4,789 jobs, 172 days. The GPU hours summed to about 240,000, against 120 hours for training the final model once, a ratio of about 2,000 (142, Strubell et al., 2019). The hardware is of 2018 and the project was a research one, so the ratio is no forecast; it is a reason to count the experiments, since a figure that omits failed runs, searches and reruns understates what the model cost.
The rule that follows is the allocation rule of Chapter 5 applied to the deployer's own training: the footprint of a training cycle, with its experiments, is counted once, at a stated boundary, and allocated to the uses and the period that benefit from it. A cycle serving one classifier for twelve months is allocated to that classifier's volume over twelve months; an adapter shared by three uses is divided among them by a stated rule, written in the record.
When fine-tuning pays
Fine-tuning has a one-off cost and may lower the cost of every later use, so whether it pays depends on volume and time. The decision rule needs three quantities.
- E_train: the energy of the whole training cycle, runs, searches and failures included, at the facility boundary of the site that ran it.
- Δe: the energy per use saved against the rung that would otherwise serve the use, at the same boundary and on the same functional unit (Chapter 6).
- N*: the number of uses at which the saving equals the cost, E_train divided by Δe. If Δe is zero or negative the tuned model never pays back, and the question is closed.
The pay-back period is N* divided by the monthly volume, set against the retraining cadence: a model retrained every twelve months that pays back in fifteen never pays back. This is the Compare test of Chapter 4 in arithmetic form; the rebound that a cheaper use invites (Chapter 10) is a separate measurement this calculation does not contain.
What is measured about the saving
The per-use saving is the one input for which there is independent measurement, and it is large for narrow tasks. Luccioni and colleagues ran 1,000 inferences per task on one node of eight NVIDIA A100 GPUs, with the CodeCarbon tool measuring GPU, CPU and memory, in a region with an average of 297.6 g CO2e per kWh, in 2023, with requests run one at a time and not batched. Across the ten tasks examined, the mean energy of models trained for the task ranged from 0.002 kWh per 1,000 inferences for text classification to 2.9 kWh for image generation. Against zero-shot multi-purpose models that were prompted for the same tasks, the most efficient task-specific models emitted 0.3 g CO2e per 1,000 inferences for extractive question answering where the multi-purpose models emitted about 10 g, and for summarisation 4 to 10 g against 20 to 30 g (130, Luccioni et al., 2024).
Three limits apply. The comparison is of small trained models against prompted multi-purpose models, not of fine-tuning against retrieval. The hardware is of 2023 and the requests were not batched, so the absolute figures are not those of a production service (Chapter 13). And the two kinds of model are hard to compare on quality with one metric, as the authors say. The study supports the direction and order of magnitude for narrow tasks such as classification: a model trained for the task can use a small fraction of the energy per use of a general model prompted to do it.
What is not measured: retrieval against fine-tuning
For the comparison this chapter is named after, the evidence is thinner. A search for a measured comparison of retrieval and fine-tuning on the same task at the same quality found none among the sources verified for this book. The closest sources measure one side: Guo and colleagues measured retrieval variants (below) and did not fine-tune, and Luccioni and colleagues compared trained models with prompted ones and did not use retrieval. The reductions reported for adapter methods are in parameters and memory, not energy, and single reports of adapter training energy could not be verified and are not used. This is a gap, not a finding that the two cost the same, and the decision rule must work without it.
The decision rule
The rule has four steps and uses the quantities above, each measured or estimated for the organisation's own use.
- Test the cheaper rungs first, on the use's own data. If prompting or retrieval meets the acceptance threshold, stop: training is not necessary.
- If only training meets the threshold, count the cycle. E_train includes the searches and the failures, at the facility boundary, with the carbon basis stated.
- Compare the totals over the period of use. For each candidate, the total is the one-off term, plus the refresh and retraining terms over the period, plus uses times the per-use energy. The candidate with the lowest total that meets the threshold is the design. For retrieval the one-off term is the embedding and indexing, the refresh term is each update of the index, and the per-use term includes the retrieved text.
- Record the uncertainty, not a single number. Give the pay-back in months with a range, and the retraining cadence next to it. If the range crosses the life of the model, the evidence does not support training and the use stays one rung up until volume or accuracy changes.
The rule as code is a few lines; the work is in measuring the inputs.
"""Pay-back volume of a training cycle against a per-use saving (illustrative inputs)."""
from dataclasses import dataclass
@dataclass(frozen=True)
class Case:
train_kwh_it: float # all runs of the development cycle, IT energy
pue: float # facility overhead of the site that runs the job
baseline_wh: float # per use, rung above, facility boundary
tuned_wh: float # per use, tuned model, facility boundary
uses_per_month: int
retrain_months: int # months between retraining cycles
def train_kwh(self) -> float:
return self.train_kwh_it * self.pue
def payback_uses(self) -> float | None:
saving = self.baseline_wh - self.tuned_wh
return None if saving <= 0 else self.train_kwh() * 1000 / saving
def payback_months(self) -> float | None:
n = self.payback_uses()
return None if n is None else n / self.uses_per_month
def net_kwh_per_year(self) -> float:
saving_wh = (self.baseline_wh - self.tuned_wh) * self.uses_per_month * 12
return saving_wh / 1000 - (12 / self.retrain_months) * self.train_kwh()
A worked example with ranges
All figures of this example are illustrative and belong to the fictional Nordvik Region. The correspondence classifier of FPR-NR-0005 routes about 120,000 items a month (the profile of September 2032). New categories after a reorganisation raise the question of whether to fine-tune a new version or to prompt a general model with the category descriptions. The training cycle in the central case is 20 runs of eight accelerators for six hours at 0.6 kW: 576 kWh of IT energy, and at the PUE of 1.55 of nordvik-dc (illustrative) 893 kWh at the facility boundary, about 29 kg CO2e at the Swedish location-based factor of 32 g per kWh and about 312 kg at 350 g per kWh. The prompted general model costs 0.50 Wh per item at the facility boundary, the tuned small model 0.01 Wh. The saving per item is 0.49 Wh. These per-item figures are at the facility boundary, for a prompted model carrying the category descriptions in every request; the lower node-boundary figures of Chapter 12 are never added to or compared with them.
Table 14.3 shows the result and its range; the range is wide because each input is uncertain, and the width is the finding.
Table 14.3: Pay-back of a classifier fine-tune (illustrative, Nordvik Region, 2032; facility boundary, operational, location-based carbon).
| Case | Cycle (kWh) | Saving per item (Wh) | Pay-back | Net a year, retrained every 12 months (kWh) |
|---|---|---|---|---|
| Best (8 runs, large saving) | 357 | 0.995 | 3.0 months | +1,076 |
| Central | 893 | 0.49 | 15.2 months | -187 |
| Worst (40 runs, small saving) | 1,786 | 0.17 | 87.5 months | -1,541 |
In the central case the fine-tune pays back in about 15 months. Retrained every twelve months it saves 706 kWh a year and costs 893, a net loss of 187 kWh; retrained every 24 months, on drift, the net is a gain of 259 kWh a year. The decision therefore rests less on the training than on the retraining policy, and the Region's record fixes retraining on a measured trigger and not on the calendar (below). On a grid of 32 g per kWh the carbon of the cycle is small and the argument for care is the energy and the accelerator time; at 350 g per kWh the same cycle emits about ten times the carbon. If the prompted alternative fails the acceptance test, none of this arithmetic applies.
Retrieval's footprint
Retrieval is not free. It moves cost from a one-off training to four recurring items.
- Embedding and indexing. Every passage is embedded and stored in an index, once for the corpus and again for what changes.
- Storage. The embeddings and the index occupy memory and disk while the service runs.
- The query. Each question is embedded, the index is searched and the passages are ranked.
- The retrieved text. The passages are added to the request, so the model reads more than the question. It is paid on every use, grows with the number and length of passages, and in most services is the largest of the four.
The only controlled measurement that could be verified is on one system. Guo and colleagues (January 2026, accepted at ICSE-SEIS 2026) tested energy-saving techniques on a retrieval-augmented system on one workstation with an NVIDIA RTX 4090: an embedding model of 1,024 dimensions, a frozen Llama 3.1 8B Instruct model, 501,916 chunks of 320 tokens and a baseline of 150 retrieved documents. Energy was attributed per container with the Kepler tool (144, Guo et al., 2026). Table 14.4 gives the changes against that baseline.
Table 14.4: Energy changes of retrieval techniques in one controlled experiment (2026; one RTX 4090 workstation; relative to the baseline; independent).
| Technique | Energy against baseline | Accuracy |
|---|---|---|
| Stricter similarity threshold, 0.78 and 0.88 against 0.58 | -20.7% and -51.0% | No loss reported at an optimal threshold; value not stated in the text read |
| Smaller embeddings, 768 and 384 dimensions against 1,024 | -20.0% and -38.1% | No loss reported by the authors |
| Two index types (IVFFlat, HNSW) | -59.8% and -58.6% | Losses of about 32% and 22% (IVFFlat, HNSW) |
| Reranker (BM25S) | -32.6% | Not stated in the text read |
The study measures relative changes, and the text read does not say whether embedding the corpus and building the index were inside the metered energy, so it is evidence about the per-query side. The two techniques the authors call truly energy-efficient, a tuned threshold and smaller embeddings, cut energy without an accuracy loss. That is consistent with the retrieved text being where the energy goes, since a stricter threshold passes fewer documents into the request; the reading is the book's interpretation, not the paper's, and the set-up is a single workstation with a small open model. The lesson is the order of the levers: set the number and relevance of retrieved passages before any other change.
No verified figure for the energy of embedding a corpus or building an index was found, so the book estimates it from measurable inputs and says so.
A worked example: the index against the query
All figures of this example are illustrative and belong to the fictional Hansa Mutual. The claims assistant (FPR-HM-0001) answers about 1.1 million queries a month over policy wordings on the Frankfurt platform (PUE 1.38, illustrative). Suppose the corpus is 400,000 passages, embedded at 200 passages a second on one accelerator drawing 650 W. A full re-embedding takes about 33 minutes and about 0.36 kWh of IT energy, or about 0.50 kWh at the facility boundary; refreshed monthly it is about 6 kWh a year. A query with retrieved context takes 1.2 accelerator-seconds at 650 W, about 0.30 Wh at the facility boundary, and 1.1 million queries a month are about 330 kWh. This simplified query shows the size of the index term; it is not the metered serving figure of FPR-HM-0001 in Chapter 13 (about 2.4 to 3.9 Wh a query at the rack boundary), and the two are never added or compared. The monthly full refresh is about 0.15 per cent of the month's query energy, and a tenfold error in the estimate would leave it under 2 per cent. The footprint of this use sits in the query and the retrieved text, where the techniques of Table 14.4 act, and not in the index.
The same inputs give the fine-tuning route. A monthly adapter fine-tune on four accelerators for three hours at 650 W is about 7.8 kWh of IT energy, or about 10.8 kWh at the facility boundary, about 22 times a full re-embedding, and it must be repeated for every change in a policy wording. These are estimates, and they show the shape: where knowledge changes, retrieval turns a retraining into a re-indexing, and the one-off terms differ by an order of magnitude or more.
At Nordvik Region the same logic applies to the casework retrieval assistant: a corpus of 2.0 million passages in nordvik-dc, of which about 8 per cent change in a month. An incremental refresh of 160,000 passages at the same rate takes about 0.22 kWh at the facility boundary (PUE 1.55, illustrative), about 2.7 kWh a year. The classifier cycle of Table 14.3, 893 kWh, is about 4,000 times a monthly refresh; the comparison is of one-off terms only.
One design consequence is not a footprint at all. A fact that must be deleted, such as a document withdrawn under a data-protection request or a regulation repealed, is removed from an index by deleting its passages; removing it from the weights of a fine-tuned model is not a routine operation. This is the book's engineering conclusion, not a cited finding, and one reason why retrieval is the default rung for knowledge.
Governing experiments
Training is governed by a budget like any other use, and the experiments are part of it: the energy of a training project is decided in the searches, retries and reruns, which no one approves one by one.
A training budget has the four properties that Chapter 4 gave every environmental budget.
- Set by someone who is not the one spending it. The sustainability lead and the platform lead agree an energy budget for the training cycle, in kWh at the facility boundary, before it starts. The person who proposes the training does not set it.
- Measured outside the job. The meter is the scheduler's accounting of accelerator-hours per project, or the provider's metering, not the training script's own log. The job cannot raise its own allowance by under-reporting.
- Enforced, not requested. A project quota in the job scheduler stops new jobs at the budget. A hyperparameter search runs inside the quota, with early stopping of runs that are not improving, and each run is logged, including the failed ones.
- A stop path outside the system stopped. A named person, the platform lead at Nordvik Region and the head of the group AI platform at Hansa Mutual, can revoke the quota. The revocation is made in the scheduler and not in the training code, and it is rehearsed: a stop that has not been drilled in the last 90 days is not counted as a cap (Chapter 4).
A rerun needs a recorded reason (a data error, a bug, a drift trigger), and a calendar is not a reason. A search is sized before it starts. The Cap test is the same as in serving: the budget exists, someone else set it, the meter is outside, and someone can stop the work.
An agent that launches training or fine-tuning jobs has the power to spend: a pipeline that retrains on a schedule, an automated search, a coding agent asked to improve a model. The ceilings of Chapter 4 apply. Within the budget it may schedule and scale down at Level 3 Bounded and provision at Level 2 Supervised. It never provisions or scales its own compute, or starts work, beyond a budget that someone else set, and it never raises its own quota or disables the meter or the stop. A person who is not the agent reviews any change to the quota (Chapter 15).
Reuse and sharing
The cheapest training is the one already done. Three kinds of reuse apply, each recorded so that a footprint is not counted twice and a model is not trained twice.
Reuse of released models and checkpoints. The Llama 3.1 model card makes the point itself. It states that, because the models are released openly, the training energy and emissions "will not be incurred by others" (143, Meta, 2024). A deployer that adapts a released model starts from training that has been paid for; the question is only whether the adaptation is necessary.
Shared adapters. One small adapter can serve several uses that need the same behaviour, such as a house style or terminology; the allocation rule divides its cycle among them.
A central index. Several services that retrieve over the same documents can share one index and one refresh. The saving is the duplicate embedding and storage; the cost is a shared dependency that needs an owner, an access model respecting who may see which document, and one refresh schedule. For a public authority the access model is also a legal question, since documents may sit under different rules of public access and secrecy (Chapter 24).
Two legal points belong beside this, framed as implications and not as advice. A provider of a general-purpose AI model keeps technical documentation with the minimum content of Annex XI (AI Act, Article 53(1)(a)), which includes the known or estimated energy consumption of the model; it goes to the AI Office and the national authorities on request, not to the public (1, European Union, 2024). And a deployer that adapts a model may become a provider of the result: Article 25(1)(b) and (c) treat a third party that substantially modifies a high-risk AI system, or changes the intended purpose of an AI system so that it becomes high-risk, as its provider. Whether an adaptation crosses either line is a question for legal counsel. Whoever adapts a model may inherit the duty to know what the adaptation cost; retrieval leaves the model, and the question of who is its provider, as it was. Chapter 21 treats the regulation in full.
Data and its footprint
Training and retrieval both rest on data, whose footprint is seldom counted: the storage of the corpus, its copies and derived embeddings, and the energy of preparing it. No measured figure was found among the verified sources, and the book supplies none. The rules below are its own and do not depend on one.
Curate before training. A smaller, cleaner corpus shortens a run before any other lever acts. Lee and colleagues found that removing duplicated text from training data made language models emit memorised text about ten times less often and reach the same or better accuracy in fewer training steps (145, Lee et al., 2022). The paper reports steps and accuracy, not energy, and its models are of 2021; the reading for a deployer is that better data can mean fewer steps and so less energy for the same run. A duplicated or leaked test set gives a result that is too good, and the decision is then taken on false evidence.
Count the copies. The training corpus, its copy in the retrieval store, the embeddings and each environment's copy are separate storage items; a record that lists them makes duplicates visible and owners accountable.
Set a retention period. A training set kept after the model is retired, and an index nobody queries, still occupy storage and carry the data-protection duties of their data. A retention period with a deletion date closes both; for personal data, the principles of data minimisation and storage limitation point the same way (3, European Union, 2016).
The decision record
A decision between the rungs is recorded, so that the next person can see what was tested, what it cost and when the result expires. The form is an AI Decision Record in the style of an Architecture Decision Record: context, options with their results, decision, consequences and a review date. It carries the three tests of Chapter 4: Count is the estimate of the cycle, with boundary and year; Compare is the per-use energy of each option, the pay-back and the rebound check; Cap is the training budget, who set it, how it is measured and who can stop it. The record below belongs to the fictional Nordvik Region and is illustrative.
# register/decisions/AIDR-1314.yaml (illustrative, Nordvik Region)
id: AIDR-1314
title: Retraining policy for the correspondence classifier after the reorganisation
status: accepted
date: 2032-11-15
organisation: Nordvik Region
use: FPR-NR-0005
owner: henrik.dahl # owner of the environmental register
reviewer: astrid.nyberg # sustainability lead; not the proposer
context: >
New correspondence categories after the reorganisation. The existing
classifier (AIDR-0042) does not know them. About 120000 items a month.
acceptance_test:
data: 2000 labelled items, set before testing, owner caseworker-lead
threshold: {metric: macro_f1, value: 0.92}
options:
- {rung: 1, name: prompt general model with category descriptions, macro_f1: 0.86, wh_per_item: 0.50, passes: false}
- {rung: 2, name: prompt with retrieved examples, macro_f1: 0.88, wh_per_item: 0.62, passes: false}
- {rung: 3, name: adapter fine-tune of the small model, macro_f1: 0.94, wh_per_item: 0.01, passes: true}
decision: Rung 3, adapter fine-tune; retrain on a measured trigger, not on the calendar.
count:
boundary: operational, facility, location-based
cycle_kwh: {low: 357, central: 893, high: 1786} # runs and failed runs included; PUE 1.55
carbon_g_per_kwh: {value: 32, year: 2032, basis: location-based}
compare:
baseline: rung 1 prompt
saving_wh_per_item: {low: 0.17, central: 0.49, high: 0.995}
payback_months: {low: 3.0, central: 15.2, high: 87.5}
rebound_check: report items routed a month; review if above 1.5 times 120000
cap:
training_budget_kwh: 900
set_by: henrik.dahl
measured_by: scheduler project accounting (platform team), reconciled with provider invoice
stop_holder: maja.lindqvist # platform lead; revokes the scheduler quota
last_stop_drill: null # no drill yet, so not counted as a cap
retrain_trigger: macro_f1 on a monthly sample below 0.90, or a new category
reuse: adapter registered in the model register; test set kept with this record
evidence:
estimate_made: 2032-11-15
measured_after_run: pending # replace the estimate within 90 days
valid_days: 90
review_by: 2033-02-13
labels: [illustrative]
The record is a file in version control, next to the footprint record of the use. The measured_after_run: pending field is deliberate: the estimate stands for 90 days and the measured energy of the finished cycle replaces it, with the date, as Chapter 6 requires. An estimate never replaced is an assumption promoted to a fact.
Practical guide: the when-not-to-train checklist
Purpose. To stop an unnecessary training run before it starts, and to make a necessary one countable.
Roles. The AI product owner (proposes), the platform engineer (runs the test and, if needed, the training), the sustainability lead (reviews, not the proposer), the owner of the outcome (sets the threshold) and the head of cloud or platform lead (holds the stop path).
Inputs. The requirement in words; 200 to 2,000 labelled cases of the use's own data; the threshold, set beforehand; the footprint record of the use; the training budget form.
Steps. Answer the five questions in order and stop at the first that settles the matter.
- Is a model needed? Could a rule, a template or a search do the work at the required quality? If yes, use it (rung 0).
- Does an existing model meet the threshold with a prompt, or with retrieval? Run the test on the use's own data. If yes, stop at rung 1 or 2. Reduce the retrieved text first (Table 14.4).
- Is the gap one of knowledge or one of behaviour? Facts that change, must be cited or must be deleted belong in retrieval; format, style and a narrow decision belong in an adapter.
- Does training pay back, with the range? Count the cycle with its experiments, estimate the saving per use, compute N* and the pay-back period, and compare with the retraining cadence. If the range crosses the life of the model, stay one rung up.
- Is the cap in place? The budget is set by someone else, measured outside the job, enforced in the scheduler, and a named person can revoke it. If not, do not start.
Artefact. The decision record above, with the test set and the footprint record, and an entry in the environmental register.
Across the modes and the organisations
Across the usage modes and degrees of delegation
Table 14.5 sets out what each degree of delegation means for training and retrieval jobs. The usage modes differ in how much of the ladder the organisation can reach. A user of embedded or ambient AI (Modes 0 to 2) cannot choose a rung. The builder lane and the platform (Modes 3 and 4) can climb it as set out here. A production agent (Mode 5) that launches work needs the budget most, because it can spend without a person in the loop.
Table 14.5: What each degree of delegation means for training and retrieval jobs.
| Level | What an agent may do | Condition |
|---|---|---|
| 0 Advisory | Recommend a rung and estimate the cycle | A person decides and runs the test |
| 1 Assisted | Draft the decision record and the test set | A person reviews and signs |
| 2 Supervised | Provision a job within the training budget | A person approves each cycle; stop path outside the agent |
| 3 Bounded | Schedule retraining on the trigger; scale down | Inside the scheduler quota; reports each run |
| 4 Autonomous | Never for provisioning or scaling its own compute or starting work beyond a budget set by someone else; never for disabling the meter, the budget or the stop path; never for publishing an environmental claim | Prohibited by Chapter 4 |
Nordvik Region and Hansa Mutual
All figures of this section are illustrative.
Nordvik Region. The scene at the head of the chapter led to a rule that no training or fine-tuning job starts on the reserved pool without a decision record and a training budget. For the classifier, run in nordvik-dc, AIDR-1314 of 15 November 2032 chose an adapter fine-tune of the small model on the evidence of Table 14.3, with a budget of 900 kWh at the facility boundary, set by Henrik Dahl and reviewed by Astrid Nyberg, with the platform lead Maja Lindqvist holding the stop path as the revoker of the scheduler quota. The Region replaces the estimate with the measured energy of the finished cycle within 90 days. The casework retrieval assistant refreshes its index weekly for changed documents and re-embeds the whole corpus once a quarter, a cadence that costs a small fraction of any retraining and keeps a repealed rule from being cited the next day.
Hansa Mutual. The group's claims assistant already runs on retrieval over policy wordings (FPR-HM-0001). In November 2032 the claims-handling unit proposed to fine-tune the self-hosted model on the policy wordings of its seven markets to shorten the retrieved context. AIDR-1364 (22 November 2032, illustrative; owner Anja Weiß, method reviewed by Dr. Verena Lindner) decided to stay at the retrieval rung, tighten the threshold and the number of passages first, measure the effect, and re-examine the fine-tune when two months of measured volume exist. The sensitivity classifier of the classification-routed set-up (a small fine-tuned model in the set-up whose exit plan is EXIT-004; new and illustrative) is retrained on a trigger and not on a calendar, within a training budget that the group AI governance board, chaired by Dr. Annette Krämer, approves above its threshold. The Frankfurt training runs are metered at rack level, which Nordvik Region lacks on its reserved pool, and the carbon of a cycle is recorded location-based and market-based side by side under Dr. Lindner's method (AIDR-1350).
Observable, expected, speculation
Observable. Parameter-efficient adapters (139, Hu et al., 2021) and retrieval (141, Lewis et al., 2020) are documented methods, and some providers of open models publish training hours and emissions with their model cards, as the Llama 3.1 card does. Published energy figures for adaptation are few, and a measured comparison of retrieval against fine-tuning on the same task was not found.
Expected. More reuse of shared adapters and released checkpoints, and more disclosure of training energy by providers of general-purpose AI models, since the AI Act requires documentation of it to the AI Office and the authorities (Article 53(1)(a), Annex XI), though not to the public.
Speculation. The future share of training in the total energy of AI; the sources read give no verified split, and the book offers none.
Summary
This chapter concerned E3 Design, applied to the question of when not to train. The approaches form a ladder from no AI to training from scratch, and a use moves down it only when the rung above fails an acceptance test on its own data, with a threshold set before the test. The ladder orders the one-off cost and not the total: a model trained for a narrow task can use a small fraction of the energy per use of a general model prompted to do it, as an independent measurement of 2023 found, and retrieval trades a training for recurring costs, of which the query and the retrieved text, not the index, are the larger part.
The evidence has a visible gap: no measured like-for-like comparison of retrieval and fine-tuning was found, and the adapter methods are documented in parameters and memory, not energy. The decision rule does not wait for one. Count the training cycle once, with its experiments and failed runs, at a stated boundary and basis; allocate it to the uses and the period that benefit; Compare the totals over the period, including refresh and retraining, and set the pay-back against the retraining cadence; and Cap the cycle with a budget set by someone else, measured outside the job, with a stop path someone can use. Training budgets bind agents that launch jobs; Level 4 Autonomous is never used for them to provision beyond the budget.
Nordvik Region's AIDR-1314 on retraining its classifier and Hansa Mutual's AIDR-1364 to stay at the retrieval rung show the practice. The reader can start with the five-question checklist before the next training job is approved. In each case the stop path is the revocation of the scheduler quota, held by the platform lead or the head of the group AI platform and not by the job. Chapter 15 turns to agents, whose power to start work makes that budget and stop path more urgent, and Chapter 16 to the hardware on which every rung runs.
Sources
- Nordqvist, G. (2026). AI Governance: Governing Artificial Intelligence in European Enterprises and Public Authorities. The Transformed Architecture, Book 2. (26, Nordqvist, 2026)
- Nordqvist, G. (2026). AI Platform and Security: Building and Securing AI Platforms in European Enterprises and Public Authorities. The Transformed Architecture, Book 3. (27, Nordqvist, 2026)
- European Parliament and Council of the European Union. Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union, 12 July 2024. Articles 25 and 53 and Annex XI read in the consolidated text of 27 July 2026. (1, European Union, 2024)
- European Parliament and Council of the European Union. Regulation (EU) 2016/679 (General Data Protection Regulation). Article 5(1)(c) and (e). (3, European Union, 2016)
- Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L. and Chen, W. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv 2106.09685. Available at https://arxiv.org/abs/2106.09685 (accessed November 2026). (139, Hu et al., 2021)
- Dettmers, T., Pagnoni, A., Holtzman, A. and Zettlemoyer, L. (2023). QLoRA: Efficient Finetuning of Quantized LLMs. arXiv 2305.14314. Available at https://arxiv.org/abs/2305.14314 (accessed November 2026). (140, Dettmers et al., 2023)
- Lewis, P. et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020. Available at https://arxiv.org/abs/2005.11401 (accessed November 2026). (141, Lewis et al., 2020)
- Luccioni, A. S., Jernite, Y. and Strubell, E. (2024). Power Hungry Processing: Watts Driving the Cost of AI Deployment? ACM FAccT 2024. Available at https://arxiv.org/abs/2311.16863 (accessed November 2026). (130, Luccioni et al., 2024)
- Strubell, E., Ganesh, A. and McCallum, A. (2019). Energy and Policy Considerations for Deep Learning in NLP. ACL 2019. Available at https://arxiv.org/abs/1906.02243 (accessed November 2026). (142, Strubell et al., 2019)
- Meta (2024). Llama 3.1 model card: Training Greenhouse Gas Emissions. Hugging Face. Available at https://huggingface.co/meta-llama/Llama-3.1-8B (accessed November 2026; vendor-reported). (143, Meta, 2024)
- Guo, Z., Gao, C. and Bogner, J. (2026). On the Effectiveness of Proposed Techniques to Reduce Energy Consumption in RAG Systems: A Controlled Experiment. arXiv 2601.02522. Available at https://arxiv.org/abs/2601.02522 (accessed November 2026). (144, Guo et al., 2026)
- Lee, K. et al. (2022). Deduplicating Training Data Makes Language Models Better. ACL 2022. Available at https://arxiv.org/abs/2107.06499 (accessed November 2026). (145, Lee et al., 2022)