liberlume-content-v3
lang: en
title: The energy cost of a language model
summary: Where the energy cost of a language model really comes from when it answers
btc-anchor: 957159,0000000000000000000088f211f8fa52b0353e897ee2415983b142bb63b556f6
prev: sha256:3c42a6a7481e8b297a33801c14c4b9ad029dfcf00fde7ae018310bb3201026d4
--- body ---
### Where the energy cost of a language model really comes from

## A bill that looks small

When we ask an assistant like ChatGPT, Gemini, or Claude a question, how much energy do we use?
The most recent and best-documented estimates say: **around a quarter of a watt-hour** for a typical text prompt.
On this point, Google has published 0.24 Wh for the median Gemini prompt; independent estimates for ChatGPT put it at ~0.3 Wh. That is *little*: the equivalent of leaving a TV on for less than ten seconds.
So why is there so much talk about it? For two reasons. The first is **scale**: those few watt-hours, multiplied by billions of requests a day, become a level of consumption that, according to the International Energy Agency, will lead data centers to double their hunger for electricity by 2030. The second reason, perhaps more interesting, is the theme of these reflections: **trying to understand where that energy is physically held (or rather, where it is channeled).** The answer to this last question perhaps contradicts the most natural intuition.
The wrong intuition is this: "the model *thinks*, that is, it does an avalanche of calculations, and calculating costs energy." True, but only in part. To understand where every Joule really ends up, it helps to take a journey, following the energy from the electricity meter all the way down to the single little word the model produces.
Before setting off, one point about scope: the figures just seen and the whole journey ahead concern the cost of **answering**, that is, inference; training the model, an enormous but one-off expense, is a different bill, and will return only in passing towards the end.

## The floor of physics (and why it's so far away)

Let's start from the bottom, asking: *is there a minimum cost, imposed by the laws of physics, to process information?* The answer is yes, and it has a formulation in the **Landauer principle**, stated in 1961 and confirmed in the lab in 2012.
It says something elegant and counter-intuitive: **erasing** a bit of information (forgetting it, zeroing it out) inevitably produces a crumb of heat. Not because of a technological imperfection, but because of a deep law: thermodynamics does not let you "throw away" information for free. That minimal crumb is worth about **three thousandths of a billionth of a billionth of a Joule** per bit, in figures, `3×10⁻²¹` Joule, that is, three *zeptojoules*. A number so small that, even if a language model paid it for every bit, the total energy would be negligible.
And that is exactly the point: **we are extremely far from this limit**. Today's chips spend, for each operation, something like *a billion times* (and more) that physical minimum. In other words: the energy an AI consumes is not "inevitable entropy" imposed by the universe. It is almost all **engineering waste**: margins, voltages, and above all, as we will see, the cost of moving data. Fundamental physics, here, is *not* the bottleneck. There is enormous room for improvement before thermodynamics becomes a problem.

There is also a missing symmetry worth telling. Landauer talks about *erasing* information. But **creating** information is another story: copying a piece of data (onto blank memory) is in principle free, and even generating new data (in the physical, random sense) can actually *release* energy rather than consume it. Destroying costs, creating does not: an asymmetry that has a surprising consequence for AIs.

> **The information you read in the answer, its *content*, is not what you pay for: translated into physical terms it would be worth energetically *nothing*. What you pay for is the *process* of producing it, and that cost is almost all data movement: an engineering cost (charging wires and memory) that is *not* a Landauer cost and has no physical floor.**

The real bill, in short, is run up by everything happening behind the scenes. But to understand *why* information in itself "doesn't weigh anything," and where the heat hides instead, it helps to open a parenthesis on the word that gives the title to these reflections: **entropy**.

## The two entropies: the same ledger

There are two "entropies" that are usually encountered in very distant contexts, one in physics, one in information theory, and that seem to have nothing to do with each other. In reality they are the *same thing*, written in two different units of measurement. And the Landauer principle is exactly the **exchange rate** between the two.

- **Thermodynamic entropy** (Boltzmann, late 1800s): measures how many different microscopic ways (which molecules, where, at what speed) produce the same *macroscopic* state we observe. More possible ways = more entropy = the less we know about the exact state. It is measured in Joules per Kelvin.
- **Shannon entropy** (1948): measures how much *information we are missing* to guess a message, that is, how much uncertainty there is. It is measured in **bits**.

Put this way they are already suspiciously similar: **both measure "how much information we don't have."** And indeed the link is a single formula, thermodynamic entropy = (a constant of nature) × Shannon entropy:

> `S` (thermodynamic) `= k · ln2 ·` `H` (Shannon, in bits)

where `k` is the Boltzmann constant. Translated: **one bit of information equals `k·ln2` of thermodynamic entropy**, and, at room temperature, about **`k·T·ln2 ≈ 3×10⁻²¹` Joules** of energy. That number sounds familiar: it *is* the Landauer limit. It is no coincidence. Erasing a bit means making a bit of Shannon entropy disappear from memory; but *total* entropy cannot decrease (this is the second law of thermodynamics), so that entropy must **reappear elsewhere**, as heat dispersed into the environment. The information you erase from the register reappears as disorder in the air around the chip. The ledger is always balanced.

To understand how *literal* this exchange is, there is a famous thought experiment (the "[Maxwell's demon](/en/maxwells-demon/)," made precise by Leó Szilárd in 1929):

```
1. A box (at constant temperature, in thermal contact with the surroundings)
   with ONE single gas molecule.
2. A demon looks and notes in its notebook: "it's in the left half".
   → it has acquired 1 bit.
3. Knowing which side it's on, it slips a piston in from the empty side and lets
   the gas push it as it expands: it extracts work = k·T·ln2.
        ← the energy comes from the surroundings as heat;
          it is the bit that makes the extraction possible.
4. But the note still reads "left". The next measurement needs a blank page:
   looking again won't do, it would write over a full memory. It must first be
   ERASED, and erasing a bit costs k·T·ln2 (Landauer).
        ← bill back to zero.
```

The "free lunch" does not exist: **that single bit of information was worth exactly `k·T·ln2` of energy.** It is the cleanest demonstration that *information is physical*, and that the two entropies are the same coin. A coin, however, worth very little: `k·T·ln2`, and nothing more. An aspect to keep in view, because shortly the *content* of an answer, converted at this rate, will turn out to weigh almost nothing.

**What does this have to do with an AI?** Everything, because it clarifies *where* entropy (that is, heat) is produced inside a chip. There are two sources, and they are profoundly different:

| Source of heat (entropy produced) | Is it tied to information? | Does it have a physical floor? | How much it weighs |
|---|---|---|---|
| **Erasing logical bits** (Landauer) | Yes: it is Shannon entropy becoming thermodynamic | Yes: `k·T·ln2` per bit | negligible |
| **Moving charges in the wires** (dissipation) | No: only heat from resistance and switching | No: in theory recoverable | **dominates everything** |

The first is the entropy *of information*: mandatory, but tiny. The second is *wasted* entropy: enormous, and it has nothing to do with the information contained in the answer, it is just the price of *how we move electrons today*.

Here, then, is what "the content of the answer is energetically free" really means: the information you read, converted at the Landauer exchange rate, is worth barely `10⁻¹⁸` Joules, invisible next to the heat we waste pushing it out. And "we are far from Landauer" can now be said precisely: **almost all the entropy our chips pour into the environment is of the second type, wasted dissipation, not of the first.** All the rest of the article is, at bottom, the story of that second entropy.

## Moving costs more than computing

Let's come, then, to the source that dominates. Inside a chip, doing a calculation (a multiplication, an addition) costs very little energy. **Going to fetch** the number on which to do that calculation, if it sits in main memory, costs **a hundred to seven hundred times more** (note: here the comparison is no longer with the Landauer limit, but with the cost of a calculation). Why does this matter so much for an LLM? Because of how it generates text: **one word at a time**. To produce each little piece of a word (a "token"), the model must reread *all* of its billions of parameters from memory. It does very few calculations on an avalanche of data just transported. Transporting costs.
The cost does not depend on the "meaning" of the data, only on the physical act of moving charge along a conductor. It is like filling a pipe with water to signal "1" and emptying it for "0": you pay for the pumping, not the message. And the longer the pipe (reaching all the way to far-off main memory) the more charge it takes each time: hence the hundred-to-seven-hundred factor.
The insiders call this being *"memory-bandwidth limited"*: the brake is not computing power, it is the speed at which the model's weights can be moved.
This is precisely the **second entropy** of the table above: *wasted* heat, not a "Landauer cost." It has no physical floor, in principle one could charge the wires slowly and recover the charge (adiabatic/reversible charging), and the energy would tend toward zero. Saying "we are far from Landauer" is a different way of saying "the cost is the transport": there is overhead due to how we build chips today, it is not "mandatory" entropy.
This single idea reorganizes everything. It explains why many of the optimizations that work do not serve to "make it do fewer calculations," but to **move less data** or to **move it once for many users**.

## The engineers' levers (and the open challenges)
From here arise the great efficiency battles underway today:

- **Quantization: using "coarser" numbers.** If you represent each parameter with 8 bits instead of 16, you *halve the data to transport*. Real measurements show consumption drops of around 40% with contained quality losses. It is the lever with the best effort/result ratio. The challenge: pushing to 4 bits or fewer without the model becoming less accurate.
- **Serving many users together (batching).** If the dominant cost is going to fetch the weights from memory, then fetching them *once only* to respond simultaneously to a hundred users spreads that cost over a hundred. It is the reason an AI in a large data center costs, per answer, far less than the same AI running just for you on your own computer.

- **Smaller but better-trained models.** An important discovery (the "Chinchilla laws") says that often a more compact model, but trained on more data, is just as good as a giant one. And a smaller model costs less *for every single answer*, forever, that is, a saving that multiplies across billions of uses.
- **Switching on only a part of the brain (Mixture-of-Experts).** Instead of using the whole model for
 each word, only the useful portion of it is activated. Calculations are saved, but, careful, sometimes memory traffic is *increased*. It is a perfect example of how "fewer calculations" does not automatically mean "less energy."

- **The far frontier: reversible computing.** If the fundamental cost comes from *erasing* information, there exist in theory ways of computing that erase nothing and that might one day drop below the Landauer limit. Today it is basic research, far from practice, but it is the direction that says "how low, in principle, one could go." (It is, not by chance, the only lever that acts on the *first* entropy, the informational one, instead of the second.)

## Data centers and accounting

Above the chip there is the building. Cooling the servers, distributing the power: all of this adds about **50% more** to the energy of the actual calculations (it is the famous "PUE" indicator, stuck at ~1.5 for years). Then there is the water consumption for cooling, and the carbon footprint of building the hardware itself.
But the important point, so as not to be fooled, is another:
**the numbers on "an AI's consumption" are often not comparable.** They vary by more than tenfold not because one model is ten times more voracious, but because each one **measures different things**: only the accelerator chip? also the computer hosting it? also the machines switched on but idle? also the cooling? is the initial training counted, or only the answers? Without declaring *what* goes into the bill, any figure is useless for the purpose of a comparison. It is also the reason standardization efforts exist (such as the MLPerf Power benchmark, which measures the current "at the socket" of the whole system).
And there is a fundamental question that still does not have a simple answer: for a heavily-used model, does it weigh more to **train it once** (an enormous but one-off cost) or to **answer** its questions (a tiny cost, multiplied by billions)? The answer changes depending on what you want to know, and it is one of the open questions.

## Efficient for whom, and for what?

Two aspects to explore further.

**Efficiency does not mean less consumption.** In 1865 the economist William Stanley Jevons noticed that more efficient steam engines, instead of making coal consumption fall, made it *rise*: since it cost less to run things, many more of them were run. This is the **Jevons paradox**, and it is exactly what we see today with AI: the energy per single answer has collapsed (about tenfold in a few years), and yet the *total* consumption of data centers is set to double. The cheaper it becomes, the more it is used everywhere, and the total rises all the same. Efficiency is *necessary*, but on its own it is not enough to reduce the overall footprint.

**We are counting the wrong thing.** A model is not more efficient because it *churns out more text* for the same energy: it is more efficient if it *reaches more correct goals* for the same energy. The difference is enormous with the new models that "reason": they produce far more words, they seem wasteful, but they get right answers that others get wrong. Counted by the word, they are the worst; counted by *problem solved*, they can be the best.
The metric that turns out to be most adequate is not energy per token: it is **energy per correct result**. Measuring it is difficult (what is a "right" answer, for a poem?), and this is precisely one of the most interesting open challenges.

## Trying to draw a synthesis

The efficiency of a language model is played out on planes and on aspects that are not necessarily intuitively obvious. It is not a battle against the laws of physics (we are extremely far from that wall), and it is not even mainly a matter of "doing fewer calculations."
It is played out at the center: **reducing data movement** and **amortizing the dominant cost** over more useful work. And when we measure, let's remember to count the **goals reached**, not the words, and to look at the **total**, not just the single answer.
The two entropies help to stay on course: the *informational* one (Landauer) is an extremely distant and almost free floor; the *dissipative* one (transport) is where the heat really goes, and it is there, not in fundamental physics, that one wins or loses. Thermodynamics, here, does not constrain us; accounting, if we do not set up the problem properly, could fool us.

## References

- Google, *Measuring the environmental impact of delivering AI at Google Scale*, 2025 (0.24 Wh for the median Gemini prompt; the drop in cost per answer over time): [arxiv.org/abs/2508.15734](https://arxiv.org/abs/2508.15734)
- Epoch AI, *How much energy does ChatGPT use?*, 2025 (the independent ~0.3 Wh estimate): [epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use](https://epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use)
- IEA, *Energy and AI*, 2025 (data center consumption doubling by 2030): [iea.org/reports/energy-and-ai](https://www.iea.org/reports/energy-and-ai)
- R. Landauer, *Irreversibility and Heat Generation in the Computing Process*, IBM Journal of Research and Development, 5(3), 1961: [doi.org/10.1147/rd.53.0183](https://doi.org/10.1147/rd.53.0183)
- A. Bérut et al., *Experimental verification of Landauer's principle linking information and thermodynamics*, Nature 483, 2012 (the laboratory confirmation): [doi.org/10.1038/nature10872](https://doi.org/10.1038/nature10872)
- L. Szilárd, *Über die Entropieverminderung in einem thermodynamischen System bei Eingriffen intelligenter Wesen*, Zeitschrift für Physik 53, 1929 (the demon made precise): [doi.org/10.1007/BF01341281](https://doi.org/10.1007/BF01341281)
- C. E. Shannon, *A Mathematical Theory of Communication*, Bell System Technical Journal 27, 1948: [doi.org/10.1002/j.1538-7305.1948.tb01338.x](https://doi.org/10.1002/j.1538-7305.1948.tb01338.x)
- M. Horowitz, *Computing's energy problem (and what we can do about it)*, ISSCC, 2014 (the cost of memory access versus computation): [doi.org/10.1109/ISSCC.2014.6757323](https://doi.org/10.1109/ISSCC.2014.6757323)
- J. Hoffmann et al., *Training Compute-Optimal Large Language Models*, 2022 (the "Chinchilla laws"): [arxiv.org/abs/2203.15556](https://arxiv.org/abs/2203.15556)
- ML.ENERGY, *LLM Inference Energy: A Longitudinal Analysis*, 2026 (measurements of quantization savings): [ml.energy/blog](https://ml.energy/blog/measurement/energy/llm-inference-energy-a-longitudinal-analysis/)
- Uptime Institute, *Global Data Center Survey*, 2025 (average PUE stuck around 1.5): [intelligence.uptimeinstitute.com](https://intelligence.uptimeinstitute.com/resource/uptime-institute-global-data-center-survey-2025)
- MLCommons, *MLPerf Power: Benchmarking the Energy Efficiency of Machine Learning Systems*, 2024 (the "at the socket" measurement of the whole system): [arxiv.org/abs/2410.12032](https://arxiv.org/abs/2410.12032)
- W. S. Jevons, *The Coal Question*, Macmillan, London, 1865 (the Jevons paradox).