How Sustainable is AI?

How Sustainable is AI? Build • James Martin • 15/02/24 • 6 min read

Just over a year after the ChatGPT-fuelled generative AI explosion, it’s hard to remember a time without these groundbreaking tools. However, it remains to be seen if the breakneck speed of change has given us enough time to fully assess generative AI’s true impact on the planet. So let’s take a look.

The impact, in figures

First and foremost, it’s now well established that generative AI requires considerably more computing power than standard calculations. A key reason for this is that generative AI model training calls for GPUs rather than CPUs. The former generally requires around four times more energy than the latter (case in point: Ampere’s CPUs for AI consume 3-5 times less energy than the equivalent NVIDIA machines ).

Furthermore, as AI GPUs tend to generate 2.5x more heat than CPUs (standard CPUs used in cloud computing are in the range of 250-350W TDP, whereas GPUs are in the 750-800W range, cf. Intel , AMD x2 , & NVIDIA ), they require that much extra cooling power. So the processors needed for generative AI training and inference are considerably more power-hungry than pre-generative AI models.

Then there’s the difference between training and inference. Looking at the former, or the process required to ‘educate’ a generative AI model by feeding it as much data as possible, the emissions generated by training vary hugely depending on the model:

552 tCO2e - GPT3.5, 1.3, 6 & 175bn parameters ( source )

284 tCO2e - a medium-size LLM, 213m parameters ( source )

30 tCO2e - BLOOM, a frugal LLM, 175bn parameters ( source )

(tCO2e = tons of CO2 equivalent, namely CO2 + the 3 other most potent greenhouse gasses)

This means that training a generative AI model can generate anything from the equivalent of three French people’s annual emissions (10 tCO2e), to 50.

But of course, training is a one-off occurrence. Inference, or the everyday usage of a model, has its own impact, which has been estimated at 200 times higher than that of training . According to French tech association Data for Good , considering ChatGPT has 100m weekly users , that’s 100,000 tCO2e/year for GPT-3.5.

To put it another way, generating one image with generative AI can use as much energy as that required to fully recharge a smartphone , according to the latest white paper co-authored by Sasha Luccioni, Climate Lead and AI Researcher at Hugging Face. "Can" is the operative word here, however, as The Verge points out, given the huge variety of GenAI models already available.

Then there’s water . Also linked to inference, it’s been established that one conversation with ChatGPT uses half a liter of water in terms of the data center cooling resources required (cf. the considerable heat generated by GPUs, above). Not to mention GPT-3’s training, which required 5.4 million liters of water ( same source ). That’s a bit more than one liter per training hour (training GPT-3 took 4.6 million GPU hours, according to… ChatGPT !)

Given these elements, it’s not surprising that AI energy demand is set to outpace supply.

If Google were to use AI for its around 9 billion daily searches - which it most likely will - it would need 29.2 terawatt hours (TWh) of power each year, according to researcher Alex de Vries. As such, as de Vries told Euronews last year , by 2027, AI could consume as much electricity as a medium-sized country like the Netherlands .

The IEA (International Energy Association) recently issued a similar warning : data centers’ energy consumption could more than double by 2026, to 1,000TWh, driven by AI and cryptocurrency.

One of AI’s most influential leaders naturally saw this coming: at Davos in January 2024, OpenAI CEO Sam Altman said AI will definitely need much more energy than initially thought . “There’s no way to get there without a[n energy] breakthrough [like nuclear fusion]”, Reuters reported him saying on a panel. This could well be why OpenAI’s most famous investor, Microsoft, just hired a new Director of Nuclear Development Acceleration: to “help power its own AI revolution”, according to TechRadar Pro .

Whilst we’re a long way off nuclear fusion - versus current fission methods - a trend of nuclear-powered data centers is definitely bubbling up.

According to AMD CEO Lisa Su , in around ten years’ time we may see zettaflop-class supercomputers, whose requirement for 500MW facilities will far outstrip todays’ 20-50MW facilities. Such needs can only be powered by local, dedicated sources like nuclear SMRs (small modular reactors).

This is why The Register reports that last year, Cumulus Data opened a 65MW nuclear data center, which it claims will ultimately reach a capacity of 950MW. In addition, SMR-powered facilities are currently being investigated by Green Energy Partners/IP3 (Virginia, USA) and Bahnhof (Sweden).

Given our current reliance on fossil fuels (e.g. with the US still dependent on them for 80% of its energy), could nuclear-powered emission-free data centers be a better option for the planet than current solutions? Time will tell, especially for future generations…

How to reduce that impact

The first rule of any sustainability strategy, especially in tech, should be to ask “do I really need this?”

Indeed, generative AI is neither inevitable, nor adapted to all use cases. As we’ve already explained here , symbolic, or “good old-fashioned” AI, can do a lot more than what many of us expect, and with considerably less impact . French startup Golem.ai has notably established that one of their email-sorting symbolic AI models emits 1000 less CO2eq than GPT-3 .

That said, if you do decide you absolutely must use generative AI, does it have to be on the scale of ChatGPT? Must it hoover up all of the world’s data, or can it just focus on a specialized dataset, like legal documents, for example?

Do you have to use a supercomputer for training, or would a smaller, single H100 GPU do the trick? Could you simultaneously prolong the life of old hardware and save money by using older generation GPUs?

For inference, could a less energy-hungry CPU, like Ampere’s, meet your needs (cf. above)?

Next, it can be inspiring to look into the many ways generative AI is being used today to actively further sustainability; potentially, to an extent that may far outweigh its impact.

Indeed, a McKinsey report once estimated AI-based technologies could help companies to reduce their emissions by up to 10%, and their…