The Hidden Carbon Cost of AI

Every time you ask an AI assistant a question, send an image through a generation model, or let an algorithm recommend your next purchase, you’re contributing to a growing energy demand that most users never see.

Training vs Inference

The energy cost of AI comes in two phases:

Training

Training a large language model from scratch requires enormous computational resources. Recent estimates suggest:

• GPT-4 class models: 50+ GWh for training (~$100M in compute costs)
• Frontier models (2026): 100-500 GWh per training run
• Carbon equivalent: 500-2000 tons CO2 per model

Inference

But training is a one-time cost. Inference—running the model to generate responses—happens billions of times:

• Per query: 0.001-0.01 kWh (roughly 10x a Google search)
• Daily global AI queries: Estimated 10+ billion
• Annual inference energy: 50-100 TWh globally

The Efficiency Paradox

As AI models become more capable, they also become more efficient per parameter. But this efficiency gain is overwhelmed by the exponential growth in model size and usage:

Year	Typical Model Size	Energy/Query	Total Queries	Net Energy
2020	175B params	0.005 kWh	100M/day	Low
2023	1T params	0.003 kWh	1B/day	Medium
2026	10T params	0.002 kWh	10B/day	Very High

What Can Be Done?

1. Efficient Architectures: Mixture-of-experts, sparse attention, and other techniques can reduce compute by 10-100x

2. Hardware Optimization: Custom AI accelerators (TPUs, NPUs) are 10-50x more efficient than general-purpose GPUs

3. Carbon-Aware Scheduling: Running training jobs when renewable energy is abundant

4. Energy-Aware Inference: Joule’s approach—setting energy budgets at the application level—can limit runaway consumption

The Joule Approach

Joule treats energy as a first-class constraint. Instead of asking “how fast can this run?”, we ask “how efficiently can this run within an energy budget?”

#[energy_budget(max_joules = 0.01)]
fn summarize(text: &str) -> String {
    // Compiler enforces this function stays within budget
    ai::summarize(text, max_tokens: 100)
}

This isn’t about limiting capability—it’s about being intentional with resources.

---

Energy data compiled from academic research, industry reports, and our own measurements.