r/EnergyAndPower • • 3h ago

Why vLLM burns peak 700W during autoregressive decode (and our open-source C++ NVML pacer to fix it)

1 Upvotes

Hey everyone,

If you run vLLM or Triton on modern accelerators (H100, A100, L40S, etc.), you probably noticed that GPUs stay locked at maximum core clock frequencies (e.g., 1980 MHz on H100) through the entire generation step.

### The Problem: Silicon Asymmetry in Serving

- **Prompt Prefill:** Compute-bound (GEMM). Arithmetic intensity is > 100 FLOPs/Byte. You *need* peak SM clocks to keep TTFT low.

- **Autoregressive Decode:** Memory-bound (GEMV). Arithmetic intensity drops to ≤ 4 FLOPs/Byte. Over 85% of step time is spent stalling on High-Bandwidth Memory (HBM) reads.

Because dynamic CMOS power dissipation scales quadratically with voltage ($P_{dyn} \propto V^2 f$), sustaining peak core clocks while memory stalls burns massive amounts of electricity without improving token latency.

### What we built: TensorPackAI (Community Edition)

We just open-sourced **TensorPackAI**: a zero-overhead C++17 NVML runtime layer with Python bindings:

  1. **Fast NVML Direct Loader:** Direct `dlopen` of `libnvidia-ml.so` (eliminates standard python ctypes latency).
  2. **Prometheus Telemetry:** Exposes windowed `tensorpack_joules_per_token` and `tensorpack_tokens_per_watt` on port `:9100/metrics`.
  3. **CLI Tuner (`tensorpack-tune`):** Inspect real-time per-GPU power and recommend memory-paced decode clocks.

### Quick Test:

```bash

pip install tensorpackai

tensorpack-tune status

```

In our Chatbot Arena benchmark runs, pacing core clocks to memory speeds during decode drops dynamic power by up to **58.1%** while keeping P90 Inter-Token Latency (ITL) strictly under human-perceptible thresholds (<20ms).

- **GitHub (Apache 2.0):** https://github.com/tensorpackai/tensorpackai-community

- **Docs:** https://tensorpackai.com/docs

Would love to hear your thoughts, feedback, and edge cases on different architectures (especially A100/L40S)!


r/EnergyAndPower • • 7h ago

Looking for an article

Thumbnail sciencedirect.com
0 Upvotes

[Article] Artificial intelligence based forecasting and optimization model for concentrated solar power system with thermal energy storage


r/EnergyAndPower • • 8h ago

Taking on ASEAN EV programmes while still supplying ICE: what changes for automotive suppliers?

Thumbnail reddit.com
1 Upvotes

r/EnergyAndPower • • 20h ago

Trump Orders to Keep Florida Coal Plant Running Cost Ratepayers Millions

Thumbnail
insideclimatenews.org
1 Upvotes

r/EnergyAndPower • • 1d ago

Electricity Bill

Post image
2 Upvotes

r/EnergyAndPower • • 1d ago

Emerging economies skipped fixed-line telephone networks to go straight to mobile. Now the exact same leapfrog is happening in clean energy.

Post image
9 Upvotes

This analysis is produced by Ember in partnership with the Climate Vulnerable Forum and V20 Finance Ministers (CVF-V20), a coalition of 74 countries across Africa, Asia, the Caribbean, Latin America, and the Pacific. It examines how the electrotech fast-track offers these nations a faster, cheaper route to energy access and development than the conventional fossil path. Most CVF countries are low or lower-middle-income economies, nearly all are net fossil fuel importers, and all but two are located in the sunbelt.

https://ember-energy.org/latest-insights/the-electric-fast-track-for-emerging-markets/


r/EnergyAndPower • • 1d ago

Beyond Petrol: How Dangote’s Lamu Project Could Build Kenya’s Petrochemical, Manufacturing, Power and Logistics Hub

Thumbnail
reddit.com
0 Upvotes

r/EnergyAndPower • • 2d ago

Could Quebec's hydroelectricity give Canada a real advantage in advanced manufacturing?

Thumbnail reddit.com
2 Upvotes

r/EnergyAndPower • • 2d ago

Nb power

1 Upvotes

Has anyone had any experience with this program (Enhanced Energy Saving Program)with nb power?


r/EnergyAndPower • • 2d ago

CBS News: Largest U.S. data centers use as much electricity as some states and countries, study finds

Thumbnail
cbsnews.com
36 Upvotes

Holy Fucking Shit! Just Holy Fucking Shit!


r/EnergyAndPower • • 3d ago

The development in Ann Arbor will share its solar and battery power with the grid, as part of a pilot that aims to lower energy costs and provide reliability.

Post image
10 Upvotes

In the coming weeks, Veridian households can enroll in Michigan’s first residential pilot of a virtual power plant; Detroit-based utility DTE Energy will then be able to remotely control the network of residents’ home batteries to inject their stored solar power onto the grid when operators need it most. ... It'll benefit both Veridian residents, who DTE will pay to be a part of the project, and DTE customers at large, as it'll help reduce the use of expensive fossil-fueled peaker plants. https://www.canarymedia.com/articles/virtual-power-plants/new-net-zero-community-michigan-vpp

The zero-bills home: how one family is beating the energy price hikes | Renewable energy
"The Naylors’ four-bedroom home in Milton Keynes acts like a mini-power station after retrofitting solar panels, a battery and a heat pump" ... ..."While millions of British households face the highest energy charges in three years from this week after a 4% rise in the price cap, and predictions of a 16% increase in January, upgrades to Naylor’s home mean she will not have to pay a thing." https://www.theguardian.com/environment/2026/oct/01/the-zero-bills-home-how-one-family-is-beating-the-energy-price-hikes


r/EnergyAndPower • • 3d ago

Plug Symposium Makes the Case for Hydrogen as a Path to Energy Security and Independence

Thumbnail ir.plugpower.com
0 Upvotes

r/EnergyAndPower • • 3d ago

Energy costs/ Energy suppliers.

Thumbnail reddit.com
1 Upvotes

r/EnergyAndPower • • 3d ago

Alfa Energy Tech

Thumbnail
youtube.com
0 Upvotes

r/EnergyAndPower • • 3d ago

UNION OF USA DECLARING A NATIONALEMERGENCYTO SECURETHE UNITED STATES BULK-POWER SYSTEM

0 Upvotes

After this executive order, how can a foreign-controlled electrical equipment company avoid sanctions, participate in U.S. grid bidding, design its corporate equity structure, and meet qualification requirements?


r/EnergyAndPower • • 4d ago

Governments turn to renewables for relief from the energy crisis

Thumbnail
ft.com
24 Upvotes

r/EnergyAndPower • • 4d ago

Report: Virtual Power Plant Capacity Increases 25.9% Year-Over-Year to 47.2 GW

Thumbnail
tdworld.com
2 Upvotes

PJM and ERCOT continue to have the largest utility commitments to data center capacity and the greatest disclosed VPP offtake capacity.


r/EnergyAndPower • • 4d ago

Nuclear Energy is actually chill.

28 Upvotes

Nuclear energy is absolutely the solution to the havoc fossil fuels have caused the earth as well as the growing energy demand. Nuclear power is unmatched in power output (averaging a whopping 1 gigawatt per hour, especially compared to other green energy sources. So naturally I believe this is the way to go, as disasters like Chernobyl are incredibly uncommon when plants are operated correctly. Nuclear energy certainly has its cons and it seems many feel very strongly that it is simply too dangerous, expensive or time consuming to utilize. I can see where one might be coming from with these concerns because they are valid. Although, in general all forms of energy that we use for our grid come with a wide list of downsides and when comparing nuclear power’s downsides to the rest of the options, I believe that it is the most efficient way to satisfy the growing energy demand. While it is very expensive and the time between planning is long, it’s better than relying on other countries' oil and natural gas, especially in the long term. 


r/EnergyAndPower • • 4d ago

Crude Up, Emissions Down

Post image
2 Upvotes

SciAm: "Oil prices are so high that CO2 emissions from fossil fuels are dropping," by Jackie Flynn Mogensen."

The American-Israeli war in Iran has been the ongoing greatest cause of a drop in annual fossil-fuel carbon emissions. "Between 2019 + 2025, carbon dioxide emissions from fossil fuels grew by about 0.8% annually, according to a new study published in Nature Climate Change." But 2026 is set to buck that trend: high oil prices, as well as the continued rise of green technology—solar power, wind + electric cars—will cause a “short-term” 0.4% drop, according to the study. 

“The high petrol prices following the war in Iran are reducing oil use + keeping emissions low,” says the study’s lead author Corinne Le Quéré, a professor of climate change science at the University of East Anglia in England. “But that’s not the whole story: The expansion of renewable energy has progressed a lot in recent years, led by China." 

The positive trend is that electrified transportation + rising renewable energy including geothermal are offsetting two-thirds of the growing demand for energy. The less positive trend is that "all sources of carbon considered, including deforestation, total global CO2 emissions are set to continue increasing." The encouraging fall in emissions is remarkable because "it is the first annual reduction since the COVID pandemic, when emissions dropped by an estimated 5% or so, Le Quéré says."

Researchers also measured a small decrease in emissions after the 2008 recession. “However, in both cases, emissions rebounded the year after." 

"If oil prices return to their prewar levels, Le Quéré + her colleagues expect to see emissions rise once more." But the “rebound” could + should be countered by further [addition of] renewable energy to the world’s portfolios. But these researchers have quantitatively established that "to tackle climate change + guarantee that global fossil emissions decrease durably, the expansion of renewable energy + electric vehicles needs to accelerate further,"

I would not just support + second this declaration—if this were a poker game—I would be all in.


r/EnergyAndPower • • 4d ago

Wood Mackenzie puts the 2025 US power transformer supply deficit at 30% and says imports are about 80% of supply. How should planners treat a 128-week lead time in a project schedule?

Post image
3 Upvotes

Ive been reading Wood Mackenzie's transformer numbers and the import share is what stuck with me. their Aug 2025 release puts the US supply shortage in 2025 at 30% for power transformers and 10% for distribution, and estimates imports at about 80% of power transformer supply (50% for distribution). the Sept 2025 report with American Clean Power has average power transformer lead time at 128 weeks in Q2 2025, after easing by 10 weeks, and says power transformers and generator step-up units are seeing sustained lead times of 2 to 3 years.

what that does to a schedule: in impact assessment work you build the programme backwards from permits and construction, and the transformer was always a line item that quietly showed up on time. at 2 to 3 years it's the critical path. a plant can be built, tested and permitted and still be waiting on a factory slot, and the slot belongs to whoever ordered earliest, which is a funny thing to discover in month 30..

what i can't tell from the headline numbers is how much of the 128 weeks is standard-design manufacturing versus custom engineering, and how much of the 30% is real missing capacity versus developers reserving slots for projects that may never get built. the 30% is a supply gap estimate, not a count of units. wood mac also says OEMs have announced about $1.8bn of North American capacity since 2023, which helps, but it won't land in a 2027 schedule.

so for people who actually schedule these: when do you place the transformer order, at financial close, at permit, or earlier? and would you pay for a slot reservation before the project is certain?

sources:
https://www.woodmac.com/press-releases/power-transformers-and-distribution-transformers-will-face-supply-deficits-of-30-and-10-in-2025/
https://www.powermag.com/wp-content/uploads/2026/01/making-the-connection-executive-summary-september-2025.pdf
https://www.powermag.com/transformers-in-2026-shortage-scramble-or-self-inflicted-crisis/


r/EnergyAndPower • • 4d ago

How Snowy Hydro’s Costs Snowballed by Billions

Thumbnail newsweekly.com.au
0 Upvotes

r/EnergyAndPower • • 5d ago

US data centers are burning through as much electricity as entire states

Thumbnail
newatlas.com
1 Upvotes

r/EnergyAndPower • • 5d ago

Study estimates magnitude of US data center pollution as climate talks unveil clean AI pledge: COMPARISON w/ future resource use of Carbon Direct Removal

Thumbnail apnews.com
0 Upvotes

A Comparison between data center resource usage and future carbon direct removal facilities. Imagine the public pushback on these facilities when you can't even point to a direct economic advantage people can wrap their heads around


r/EnergyAndPower • • 5d ago

Lags in Utility Mapping

Thumbnail
reddit.com
1 Upvotes

Energy Central: "Closing the Utility Field-to-Office Gap."

"GIS-powered [Geographic Information Systems] enterprise mobility drives safety, reliability, + resilience, with new Energy Central audience data on what's behind field safety incidents." 

So—imagine a crew swaps out a pole [power or telecommunications] on Tuesday. "Two weeks later a planner pulls up the map, and the old pole is still there." When we asked utility pros how long it takes for a field change to show up in their records, only 16% said the same day, + more than 25% said a month or longer.

That lag has real costs. "Crews end up working from outdated or incorrect info, planners budget around a network that has already changed, + many safety incidents + near-misses trace back to information or communication breakdowns."

ESRI [Environmental Systems Research Institute] is an American GIS corporation founded in 1969 + headquartered in Redlands, California, which specializes in geographic information system (GIS) software, location intelligence, + mapping. ESRI + Energy Central have collaborated in generating a report, though I think you may have to request joining Energy Central to get access.

This report, a collaboration with ESRI, shows how utilities are closing the field-to-office gap with GIS + mobile tools, and what changed when they did, as exemplified here:

—Puget Sound Energy went from 120 days to 24 hours for new gas assets to show up in its maps.

—Great River Energy saved 450 hours of work after linking its maps to its work-order system.

-—Denmark's N1 [largest electricity system operator in Denmark, subsidiary of Norlys, a major energy + telecommunications 

holding company] kept power on for 99.6% of customers through an historic storm surge.

I think everybody can see the utility in this, pun sort of intended.