Inside AI Data Centers and Their Growing Bills

By Sheng Pang · Published · 11 min read · Fictional story with reported financial figures

AI in practice ai data centers ai infrastructure hardware cloud computing ai costs

Ben paused an old game he liked when another data center announcement appeared on his phone. The photo showed a building large enough to make his apartment look like a cupboard. He had seen several announcements like it lately, each attached to an amount of money he could barely picture. He looked at the save files on his own computer. Were people really producing so much data that storing it required all this concrete?

He meant to search for a quick answer before returning to the game. Soon he had a photograph of server cabinets on one screen and the pause screen on the other. The cabinets looked a little like expensive lockers. Ben opened the AWS and IBM introductions to data centers and began drawing what was inside them. His first correction came before he had finished the outline. Those lockers contained working computers, not just shelves of saved files.

A server was a computer providing a service to other computers. In a data center, many could sit in tall frames called racks, with cables connecting them. Ben recognized the CPU from his own PC. It ran the server software and helped handle requests and prepare work for the AI chips. He put it in his drawing beside the part he had spent far more time shopping for.

His own graphics card had been chosen with spaceships in mind. Its GPU could perform many similar calculations in parallel, which also suited the large arrays of numbers used by neural networks. In an AI server, GPUs used that parallel processing to calculate with the model's numbers. Other kinds of chips could do this work too.

Ben followed a model file through a server diagram to find out where the calculations happened. Storage drives kept files even when they were not being actively used. System RAM held active programs and data, while GPU memory kept numbers close to the chip doing the work. Model weights, the learned numbers used in predictions, needed space along with working data. Ben had read that enough disk space did not guarantee enough working memory to run a local model. How the work was split across chips affected which memory held it.

The storage was doing more than keeping the original model file safe. Training produced checkpoints, saved states that let a run continue after an interruption. Fast storage also helped feed training material into the computers, while other storage could keep archives.

Moving data between the machines required the cables in the photograph. Switches connected machines, and routers connected networks so requests could reach the service. Inside a large AI job, chips could need to exchange results as they worked on a model together. Slow connections could leave expensive processors waiting for one another.

A photo of power equipment made him scroll back. Power had to reach the racks safely through cables and equipment that split the supply between them. An uninterruptible power supply, usually shortened to UPS, could keep things running through a brief outage. Backup generators could support a longer outage, depending on the design. Ben had lost game progress to a power cut before. He had never thought much about how an online service stayed available while the lights were out.

Under his desk, the fan beside his knee was still running. His room could get warm during a long game, and he looked for where the server room's heat went. Some designs moved it with air, while others used liquid cooling near hot components. Dense AI equipment could require changes to both the cooling system and the electrical supply. There were whole parts of the building devoted to moving heat away from the computers he had been looking at.

Ben was about to close the introduction when he reached the maintenance work. Software assigned jobs to available machines and watched whether they were healthy. People maintained equipment and dealt with faults. Physical security protected the building, while digital controls limited access to systems and data.

He could see why training needed machines. But what happened after the model was trained? Training adjusted a model's weights through repeated calculations, as he had read in an explanation of how a chatbot is trained. Answering a fresh prompt still required calculations using those weights. That stage was called inference. Ben thought about how often he asked for another version of the same email. Each fresh draft needed a computer to do more work.

Now he pictured the service answering requests while another team trained or tested a model. More people using it at once could need more machines, and harder requests could keep those machines busy for longer. A company also needed enough power and room to expand. Sites in different places could serve different regions and reduce the risk of relying on one building. Ben could see reasons to build beyond storage space. He still wanted to know whether each new site had enough work to justify it.

Then the game on his other screen caught his eye. His computer was near the age when he usually began browsing new parts, somewhere around four or five years. Trying a newer game lately had meant lowering the settings to make it run well. If an AI company faced the same problem across whole buildings, did it have to buy everything again just to stay in business? Ben sat up. He was beginning to wonder whether the shopping ever stopped.

The old game he had paused still ran well enough for him. It asked less of the computer than the newer one, so he looked up whether the card itself would lose speed with age. Heat could reduce a processor's speed, and components could develop faults. A heavier game asking more of working hardware was a different problem. He had mixed together equipment that needed repair and equipment that could no longer do the job he wanted.

Searching for server replacement took him to a Microsoft Azure article about the life of its hardware. It described keeping parts for reuse and repairs as well as retiring old equipment. For a particular AI workload, older chips might still be useful if they met its memory needs and supported the software. Newer chips could finish the work with less energy or make a demanding model practical. Ben had been shopping for a newer card without checking whether the games he wanted needed one. A server operator had a similar reason to check the work before replacing the machine.

He had also given every part of the building the same expiry date. Replacing computing equipment did not automatically require replacing the roof or buying the land again. Power and cooling equipment had their own limits, and fitting a denser generation of servers could still require expensive changes. A research paper from Microsoft and the University of Texas at Austin studied these linked decisions across a data center's life. Ben could see why no single replacement age answered his question. He erased the large arrow he had drawn from old GPU to new building.

Replacing a failed server raised another question. Did customers lose access while someone changed it? An operator could monitor faults and arrange spare capacity so a service could send work elsewhere. That required a system designed for it. AWS documentation warned that even automatic recovery could fail and require manual help. Ben went back to the page about the power supplies. Keeping a service available clearly added work beyond buying the chips.

By then Ben wanted to know who was paying for all of this. He opened a spreadsheet and headed the first column AI revenue. Beside it he put infrastructure spending, expecting a quick comparison. He wrote the reporting period beside each figure. Almost immediately he had to change his headings. The companies did not all publish a separate total for AI sales, and the spending figures covered different things.

Microsoft gave him an example. Its July 29 report for the quarter ending June 30 put Microsoft Cloud revenue at $59.3 billion. It reported about $41 billion in capital expenditures, investment in assets such as servers and buildings. This included new assets added through finance leases, where payment was spread over time. The measure therefore differed from cash paid in the quarter. Since Microsoft Cloud also included much more than AI, Ben left the AI-only quarterly revenue cell empty.

Ben found a more direct AI number in Microsoft's earlier April 29 call. The company said its AI business had passed a $37 billion annualized revenue run rate. That described revenue at an annual pace, not $37 billion already collected during a completed year. It also came from an earlier disclosure than the June quarter. He could not use it to fill his quarterly revenue cell. He added an extra column for the annual pace instead.

Alphabet's July 22 results covered the quarter ending June 30 too. Google Cloud revenue was about $24.8 billion, while Alphabet spent about $44.9 billion in cash buying property and equipment. This spending figure counted cash purchases, unlike the Microsoft measure that included newly added finance leases. Cloud sales included services beyond AI, and the spending supported a wider business than Google Cloud. Ben put the figures on separate lines so he could still see what each meant.

He opened Meta's October 21 release next. It reported $64.6 billion in revenue for the quarter ending September 30, mostly from advertising. AI could support that business through recommendations and advertising tools, but the release gave no separate AI revenue total. Capital spending was about $34.91 billion. This included cash paid toward the amount owed on finance leases, excluding interest. Meta counted these principal payments rather than the value of new leased assets included in Microsoft's measure.

Meta also reported a $9.86 billion accounting profit, called net income. Its operations brought in about $32.79 billion of cash, reported as operating cash flow. But after capital spending, its free cash flow was negative $2.12 billion. Ben subtracted $34.91 billion from $32.79 billion and got the reported shortfall. He looked up depreciation to understand why buying equipment did not also turn the profit into a loss. Buying long-lived equipment used cash, while depreciation spread its cost through accounting expenses over its estimated useful life.

A search for OpenAI revenue brought him to its March 31 announcement. It said the company was generating $2 billion per month, a statement about that time rather than a total for 2026. He found a much larger number in the January 2025 Stargate announcement. It described a new company intending to invest $500 billion over four years in infrastructure for OpenAI, with several partners involved. Ben checked who was making the investment before putting it in the sheet. This was a project intention, not $500 billion OpenAI had already spent from its own bank account.

Ben leaned back from the spreadsheet. He had wanted a quick sum that would tell him whether AI was a brilliant business or an expensive mistake. Instead he had monthly sales beside quarterly accounts and a project plan stretching across years. The lease figures did not measure the same thing either. Adding the numbers into a grand total would have hidden those differences and could count related deals more than once. He deleted the total row.

Ben crossed out his idea that old chips meant endless whole-building replacements. There would still be repeated bills, and he saw a real risk of buying more capacity than paid work could support. Keeping machines busy would not be enough if the prices were too low to cover their costs. Better software or newer chips could reduce the cost of a task, while maintenance and power still had to be paid for. His company figures could not settle whether the building in the photo would earn its cost back. He still lacked its own costs and a way to measure the business it supported.

Ben finally returned to his game. He left the graphics settings alone and decided not to order a new card that night. On his phone, under the data center announcement, he saved a question. Would the business this data center supported earn enough to cover its costs?

← More AI in practice articles  ·  All articles

↑ Top