Research Summary: Greenpixie's AI Token Methodology: Assessing the Energy, Water and CO2-eq Impact of AI Tokens for Open and Closed Weight Models

The research describes a novel methodology for assessing the per-token energy, water, and CO2-equivalent impact of cloud-hosted Large Language Model (LLM) inference. This approach differentiates between input (prefill) and output (decode) tokens, primarily focusing on Graphics Processing Unit (GPU) energy usage during inference benchmarking. It incorporates a model that relates energy per token to LLM size, traffic, and hardware configuration, extending its application to proprietary models through size binning.

Why it matters

This methodology provides a standardized and granular approach to quantify the environmental footprint of AI model usage, offering critical data for sustainability initiatives and resource optimization. Understanding the energy costs at a per-token level enables more informed decision-making regarding AI deployment, hardware selection, and operational efficiency across various sectors reliant on LLMs.

Key insights

• A methodology has been developed to estimate the per-token energy cost for cloud-hosted LLM inference.
• The methodology distinguishes between energy costs for input (prefill) and output (decode) tokens.
• GPU energy consumption is directly measured during inference benchmarking for open-weight models across various text-based tasks.
• Non-GPU server energy contributions are estimated based on inference wall time.
• Bayesian linear regression is employed to model the relationship between energy per token and LLM size, request traffic, and hardware deployment.
• Proprietary frontier LLMs are categorized into size buckets using naming conventions and performance indicators for impact assessment.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.33965