00 · IN THREE MINUTES
The answer in three steps
- 1Training and serving models require chip operations, memory movement, networking and cooling.
- 2Water use can occur on site in cooling and off site in electricity generation and manufacturing.
- 3A single universal footprint per prompt is misleading without model, hardware, location and utilization data.
01 · THE MODEL RUNS IN A MACHINE
The model runs in a machine
Matrix operations are performed by processors that draw electricity and move data through memory. Servers, storage, network equipment and power conversion add overhead beyond the accelerator itself.
02 · TRAINING AND INFERENCE DIFFER
Training and inference differ
Training may concentrate enormous computation into a limited run; inference repeats whenever users request outputs. A widely used model can accumulate more lifetime energy in serving than in its initial training.
The model runs in a machine
Matrix operations are performed by processors that draw electricity and move data through memory. Servers, storage, network equipment and power conversion add overhead beyond the accelerator itself.
computeTraining and inference differ
Training may concentrate enormous computation into a limited run; inference repeats whenever users request outputs. A widely used model can accumulate more lifetime energy in serving than in its initial training.
memory + networkHeat must leave the building
Fans, chillers, evaporative systems or outside air transfer waste heat. Water consumption depends on cooling design and climate; electricity generation may add an indirect water footprint even where the facility itself uses little.
coolingEfficiency can lower or raise totals
Better chips and algorithms reduce energy per computation, but cheaper use can increase demand. The relevant measure combines efficiency, total workload, grid carbon intensity and the timing and location of operation.
location matters03 · HEAT MUST LEAVE THE BUILDING
Heat must leave the building
Fans, chillers, evaporative systems or outside air transfer waste heat. depends on cooling design and climate; electricity generation may add an indirect water footprint even where the facility itself uses little.
04 · EFFICIENCY CAN LOWER OR RAISE TOTALS
Efficiency can lower or raise totals
Better chips and algorithms reduce energy per computation, but cheaper use can increase demand. The relevant measure combines efficiency, total workload, grid carbon intensity and the timing and location of operation.
05 · TRANSPARENT ACCOUNTING NEEDS BOUNDARIES
Transparent accounting needs boundaries
Reports should separate operational electricity, embodied hardware emissions and water withdrawal from water consumption. Estimates without system boundaries can differ while all appear numerically precise.
06 · SOURCES AND EVIDENCE
Sources and evidence
Claims are linked to foundational papers, standards or the primary study behind the update.
- 01Carbon Emissions and Large Neural Network TrainingPRIMARY STUDY ↗
Supports a defined mechanism, measurement or evidence boundary in this article.
- 02Making AI Less ThirstyPRIMARY STUDY ↗
Supports a defined mechanism, measurement or evidence boundary in this article.
- 03Energy and AIINSTITUTIONAL REPORT ↗
Supports a defined mechanism, measurement or evidence boundary in this article.
