How Much Does It Cost To Build A Data Center?

Data centers are facing a reckoning. Between pushback from local communities and tech enthusiasts who have had their hobbies upended by ever-increasing prices due to component shortages, the data center is now public enemy number one. Of course, there's a massive difference between an AI data center and a regular one, in that the AI data center costs are exorbitant and now require even more complicated hardware to ensure efficiency.

Breaking down the costs, companies building out AI data centers estimate the cost to be around $10 million and $20 million per megawatt for "AI-optimized" data centers. Meanwhile, the "hyperscalers," companies that are building out data centers to then rent out the compute housed inside to other AI companies, are looking at $45 to $55 billion per gigawatt. It's not just the building that eats costs, but the intense setup that's inside, including cooling, water, plus whatever the land underneath the data center costs. Maintaining that power has become a problem, and even getting the data center hooked up to the wider grid is now being met with delays, as there is quite literally not enough juice for everyone.

These are the rough costs for what an AI data center costs, with it looking unlikely that any element of it will come down. RAM, graphics cards, processors, and storage have been impacted by shortages caused by data centers hoarding kit for build-outs. These components by themselves will now run hundreds of dollars for just the correct RAM to insert in the machines, while Nvidia's top-grade GPUs used within the AI workflows will cost into the hundreds of thousands of dollars.

Trying to figure out the direct costs of AI data centers themselves can be a little tricky, especially when it comes to Elon Musk's Colossus sites in Memphis and Mississippi, since he installed illegal gas turbines to get around the lack of power. Epoch.ai has kept a detailed tracker of various data centers across the U.S., with calculations based on purchases and news sources, to estimate the total cost.

Musk's Colossus 1, with around 340MW of power, is estimated to cost around $12.9 billion with all the hardware inside. Colossus 2, however, is using newer hardware and, as such, is estimated to be around $35.8 billion with all its hardware included. Microsoft's data center, Fairwater, which is the current enemy of those living around it in Milwaukee, Wisconsin, is also reported to have cost around $7.3 billion, but Epoch does estimate that everything tallied together brings that to $24.1 billion.

Sadly, the costs are likely to jump again in the future. Data center GPUs and hardware that are running all day, every day, doing intense computational tasks, will inevitably need replacing. In the case of the graphics cards, this isn't a drop-in replacement. As of right now, the latest in its lineup, Vera Rubin NVL72, can cost as much as $9 million for a rack. Now what if we told you that in a couple of years, after the next server-grade GPU announcement, all of them will need to be replaced to keep the data center up to scratch?