Outlook theme 1 of 4
Inference everywhere
Training builds the brain. Inference is the brain at work - every second of every day.
The big picture
There are two phases in the life of an AI model. Training is when a model learns - a huge, one-off effort run in giant data centres. Inference is when the trained model is actually used: answering a question, writing an email, reviewing a contract, spotting a defect on a production line.
The first wave of AI spending was dominated by training. The next wave belongs to inference. Every new user, every new app and every AI agent working in the background creates more inference demand - and unlike training, it never stops.
Why now
AI has moved from experiment to everyday tool. Assistants are built into phones, office software and search; businesses are deploying agents that run tasks around the clock. Usage, not model-building, is now the main driver of compute demand.
Where the opportunity sits
The layers I’m researching
Efficient inference chips
Accelerators and custom silicon built to run models cheaply at scale, not just train them.
Distributed data centres
Smaller facilities closer to users, cutting delay for real-time AI services.
Memory & networking
Running models fast depends on high-bandwidth memory and low-latency connections.
On-device AI
Phones, PCs and cars running models locally, driving a new hardware upgrade cycle.
Signals I watch
How I’ll know it’s working
- ↗Growth in daily active users and queries across major AI platforms
- ↗Cloud providers reporting inference as a rising share of AI revenue
- ↗Falling cost per AI query - which historically expands total usage
- ↗Launches of chips designed specifically for efficient inference
Risks
What could go wrong
- !Rapid efficiency gains could lower spending per query faster than usage grows
- !Price competition between AI providers could squeeze margins
- !Today’s leaders in training chips may not lead in inference
How I’m approaching it
I’m tilting compute exposure from training-heavy names toward businesses that benefit from usage volume: inference-focused silicon, memory, networking and the platforms that sell AI by the query.
“Training happens once. Inference happens billions of times a day - and that’s where the next decade of demand lives.”
These are personal, forward-looking views, not predictions or recommendations. They may turn out to be wrong.