Local LLM vs Cloud: Idle RAM Is Not Waste

Sam Altman, on the Cheeky Pint podcast, as reported by The FPS Review on 2 November 2026: "From a global allocation perspective, actually, it's quite terrible to have everybody have a bunch of the world's limited supply of DRAM sitting in their own computer and being used 5% of the time."

The same day, at Adipec in Abu Dhabi, he said AI "was mostly not as smart as our smartest scientists, and now it's mostly smarter than them."

I disagree with both statements, and I will argue it from the work I do: running a local LLM for clients whose data cannot leave their own hardware. In the local LLM vs cloud argument, "maximum utilization" answers the wrong question.

Short answer. Utilization is the number a landlord watches. An owner buys other things with idle memory: answers without a network round trip, data that stays in the building, and a bill that does not move when someone else's finances do. And a model that is "mostly smarter than scientists" still needed a source list and an evaluation engine every time I put one into production.

Is RAM that sits idle 95% of the time actually wasted?

No. Idle capacity is what an owner pays for so that the machine answers immediately, offline, and without sending anything anywhere. Average utilization measures how busy a machine is over a month. Ownership buys the moment you need it.

A fire extinguisher sits at zero percent utilization for years. A central depot across the city would raise their utilization and defeat their purpose.

Altman's accounting counts the other 95% as loss. Mine counts it as the reason a document I load into a local model is never copied to anyone's server. Those are different ledgers, and he picked the one where my choice looks like a mistake.

Local LLM vs cloud: who does maximum utilization actually serve?

The provider. His second sentence was "You want maximum utilization in the cloud if you're trying to deal with this very supply-constrained environment." A provider earns more per machine the fuller its machines run, so utilization is the landlord's score. A tenant keeps a different one: what it costs, who can read the data, and what the bill does next year.

What did leaving the cloud for self-hosted infrastructure actually save?

My tenant-side numbers are specific. I moved an EdTech platform off AWS to a self-hosted managed cloud with zero downtime and about 70% lower infrastructure cost. I wrote up the order the services moved in and what the move cost. In my current role I cut monthly cloud spend by 45% across production environments on hybrid on-prem and cloud infrastructure, at 99.95% uptime.

The reason clients give me is not price. The companies I work with in health, education, medical and research want sensitive data on their own servers, because on AWS it sits on Amazon's. Self-hosted retrieval and inference follow from that rule. The retrieval layer I run for them, described in Qdrant and BGE-M3 hybrid search for RAG, exists because the documents have to stay on hardware the client controls.

OpenAI sells access to its models through its own cloud, so a world where my RAM is "wasted" is a world where I rent. That is my reading of his motive, and FPS Review makes a similar one.

He called the arrangement quite terrible. I will borrow his words: Sam Altman is a "quite terrible" and "wasteful" human being. That is my opinion, and the argument in this essay does not rest on it.

What does owning the machine cost, and what has broken?

Owning the machine costs operations: backups, patching, monitoring and on-call, all of it yours. Self-hosting moves risk rather than removing it, as the AWS write-up says, and a team with nobody to own those duties should not attempt it.

On the clients' infrastructure I run self-hosted Sentry, Prometheus and exporters for services and for hardware, with Grafana dashboards and alerting. A failing disk is my problem, not a status page. Before any Terraform or Ansible is written there is procurement, KYC and NDA paperwork, and that comes first every time.

Altman has a fair point hiding here. Most people do not want to run a database failover. For them the cloud is the right trade, and I would not tell a founder with no operations hire to rack servers. My argument is narrower: the cost of ownership is real, I pay it knowingly, and calling the unused memory the waste ignores everything I spend to keep it useful.

Is AI now smarter than our smartest scientists?

Not on anything I have measured. On a small CPU test, retrieval answered 8 of 8 reworded questions correctly and a LoRA adapter trained on the same facts answered 5 of 8. The sentence also carries its own hedge: "mostly". In my work a model's answers were only as good as the sources I could put beside them, and the platforms I shipped needed an evaluation engine because a model's answer is a claim that has to be checked.

Here is the measurement. On 19 October 2026 I ran a small test of retrieval against fine-tuning on Qwen3-0.6B, on a CPU, with invented labelled data, and published it as fine-tuning vs RAG for a self-hosted LLM. Retrieval answered all 8 reworded questions correctly. A LoRA adapter trained on the same facts answered 5 of 8.

Then I changed one fact, a retention period from 5 years to 7. Re-indexing took 0.014 seconds and the retrieval answer was right. The adapter gave a value that was neither the old one nor the new one, and fixing it would have meant another 600-second training run.

A scientist told that a retention period moved from five years to seven does not need ten minutes of retraining, and does not answer with a number that is neither. Citing the source, updating when the fact changes and saying where an answer came from are most of what the word "scientist" means. None of that held for the adapter.

The same test also showed what tuning is good for. On a strict four-field JSON output, a careful prompt produced fully correct notes 1 time in 10, and an adapter trained on 60 labelled notes got 10 of 10 (on test notes drawn from the same templates as the training notes, so the real gain is smaller). Models are good at behaviour. They are a poor record.

The pharma data platform I led had the same shape at larger scale. It ran fine-tuned self-hosted GPT-OSS, Llama and Qwen models, plus a custom chat and evaluation engine. Nobody builds a grading harness for a colleague they believe outranks their scientists.

What am I not saying?

I am not saying the shortage is imaginary. FPS Review describes the DRAM shortage as created by the AI industry; I have not audited the memory supply chain and take no position on its cause. Altman named energy as his biggest challenge at Adipec, and I believe him.

I am also not saying the cloud is wrong for bursty workloads, or that every team should self-host. The 70% figure comes from one platform, and the 45% from one engagement. They are what I measured, not a law.

The 8-of-8 and 5-of-8 results come from a 0.6B model, 8 questions and invented data on a shared CPU host. They show what a retrieval layer does for a small model. They do not show what a frontier model does, and I have not tested one here.

I read the Cheeky Pint remarks as printed by The FPS Review, not from the recording or a transcript, so the exact wording above is theirs.

And I am not arguing against human control of AI. Altman said there must be no room for debate on that point, and I agree.

Who should hold the control that humans are meant to keep?

The person who holds the machine. Control is a property of where the weights run, who can switch them off and who can read the prompts. Control of a system that lives in someone else's data center is a permission, and the owner of the data center can revoke it.

I run models on hardware I or my client holds, on a bill I can predict, with an evaluation engine to check the answers. None of that shows up as utilization, and all of it is why the memory can sit there at 5%, if his figure is right.

Maximum utilization is how a landlord keeps score. I would rather own the house and leave some rooms empty.

If you have measured real utilization on a local inference box, I would like to see the number, and what you were paying for with the other 95%.

References

• The FPS Review, "Sam Altman ridiculed by Elon Musk following a comment that it's quite terrible that ordinary users are consuming DRAM during this global shortage", 2 November 2026: https://www.thefpsreview.com/2026/11/02/sam-altman-ridiculed-by-elon-musk-following-a-comment-that-its-quite-terrible-that-ordinary-users-are-consuming-dram-during-this-global-shortage/
• The National, "No room for debate over human control of AI, says Sam Altman", 2 November 2026: https://www.thenationalnews.com/future/technology/2026/11/02/no-room-for-debate-over-human-control-of-ai-says-sam-altman/
• Abdullah Chaudary, "Fine-tuning vs RAG for a self-hosted LLM" (test run 19 October 2026): https://abchaudary.me/blog/fine-tuning-vs-rag-self-hosted-llm
• Abdullah Chaudary, "What leaving AWS for self-hosted costs, in order": https://abchaudary.me/blog/aws-self-hosted-migration-cost
• Abdullah Chaudary, "Qdrant and BGE-M3: hybrid search without a second model": https://abchaudary.me/blog/qdrant-bge-m3-hybrid-search-rag