Ahmadreza Samadi / HomeTechnology and proptech, explained
Why does moving data matter for AI chips?
Microsoft’s 12 October Maia announcement highlights data movement in AI infrastructure. Here is the idea, without repeating its performance forecasts.
Training learns. Inference uses.
Training adjusts a model. Inference uses a trained model to produce a result from an input. Today’s chip story focuses on inference.
Ask three workload questions.
What model and input size? How long until a useful reply? How much complete-system energy and cost? Keep the output quality requirement the same.
A faster number needs a context.
Our original framework: identify the workload, locate the bottleneck, then test the complete result. No chip test, measured saving or vendor ranking is claimed.
AI needs data. Not just arithmetic.
Microsoft’s 12 October Maia announcement highlights data movement in AI infrastructure. Here is the idea, without repeating its performance forecasts.
Training learns. Inference uses.
Training adjusts a model. Inference uses a trained model to produce a result from an input. Today’s chip story focuses on inference.
Source: IBM: What is AI inference?
The chip needs the next pieces.
A computing unit performs operations. Model data and working information must reach it from memory. Waiting for those pieces can limit useful work.
Source: IBM: What is AI inference?
A nearby shelf is a useful analogy.
Imagine a cook and ingredients. A fast cook can still wait on deliveries. Nearby storage can reduce some trips. This is an analogy, not a chip diagram to scale.
Source: IBM: What is AI inference?
More requests change the problem.
Serving many users also involves memory, communication and scheduling. The whole system matters; one headline chip number does not describe every workload.
Source: IBM: What is AI inference?
Today and 2027 are separate.
Microsoft says Maia 200 serves production workloads. It plans Maia 300 production deployments in 2027. A plan is not evidence of deployment today.
Ask three workload questions.
What model and input size? How long until a useful reply? How much complete-system energy and cost? Keep the output quality requirement the same.
Source: IBM: What is AI inference?
A faster number needs a context.
Our original framework: identify the workload, locate the bottleneck, then test the complete result. No chip test, measured saving or vendor ranking is claimed.
Source: IBM: What is AI inference?
See the explanation in slides
These original diagrams explain the mechanism. The carousel is in English.
View all 8 English slides
Download the English carousel PDF
Sources
Primary sources checked on 2026-10-13. Dated announcements and operator reports are attributed in the explanation; availability and terms can change.