CPU vs GPU vs TPU vs NPU vs LPU 可视化详解：
如今驱动 AI 的 5 种硬件架构。
每种架构都在灵活性、并行性和内存访问之间做了截然不同的取舍。
> CPU
专为通用计算而生。几个强大的核心处理复杂逻辑、分支和系统级任务。
拥有深层的缓存层级和片外主存 (DRAM)。非常适合操作系统、数据库和逻辑密集型代码，但不擅长矩阵乘法这类重复性数学运算。
> GPU
GPU 不像 CPU 只有几个强力核心，而是将工作分发给数千个较小的核心，这些核心在不同数据上执行相同的指令。
这就是为什么 GPU 主宰了 AI 训练。这种并行处理模式完美契合神经网络所需的数学运算。
> TPU
术业有专攻，TPU 走得更远。
其计算核心是一个乘累加 (MAC) 单元网格，数据像波浪一样流动。
权重从一侧进入，激活值从另一侧进入，中间结果直接传播，无需每次都写回内存。
整个执行过程完全由编译器控制，而非硬件调度。Google 专门为神经网络工作负载设计了 TPU。
> NPU
这是边缘侧优化的变体。
架构围绕“神经网络计算引擎”构建，集成了大量的 MAC 阵列和片上 SRAM。不同于高带宽内存 (HBM)，NPU 使用低功耗系统内存。
其设计目标是在个位数瓦特的功耗预算内运行推理，比如智能手机、穿戴设备和物联网设备。
Apple Neural Engine 和 Intel 的 NPU 都属于这一类。
> LPU (语言处理单元)
Groq 推出的新角色。
该架构完全摒弃了关键路径上的片外内存，所有权重都存储在片上 SRAM 中。
执行过程完全确定且由编译器调度，这意味着零缓存未命中和零运行时调度开销。
取舍在于单颗芯片的内存容量有限，需要数百颗芯片互联才能跑起一个大模型。但延迟优势确实非常明显。
AI 算力已经从通用的灵活性 (CPU) 进化到极致的专业化 (LPU)。每一步演进都是在用通用性换取效率。
下图并排展示了这五种架构的内部设计。
注意连接这五者的线索：每一代架构的出现都是为了减少数据搬运。因为数学运算从来不是难点，如何够快地把数据喂给计算单元才是。
软件层面的博弈同样激烈。在 LLM 推理过程中，单个 GPU 每天会产生数 TB 的 KV 缓存，且几乎全部被丢弃并重新计算，这也是 Agent 工作负载昂贵的主要原因之一。
我写了一篇深度解析，介绍了解耦缓存层如何解决这一问题，并将首字生成速度提升了 14 倍。文章见下文。
同时也推荐关注 @lmcache 的 GitHub 仓库：https://github.com/LMCache/LMCache
(别忘了点个 star 🌟)
👉 轮到你了：这 5 种硬件中，你实际使用或部署过哪几种？
如今驱动 AI 的 5 种硬件架构。
每种架构都在灵活性、并行性和内存访问之间做了截然不同的取舍。
> CPU
专为通用计算而生。几个强大的核心处理复杂逻辑、分支和系统级任务。
拥有深层的缓存层级和片外主存 (DRAM)。非常适合操作系统、数据库和逻辑密集型代码，但不擅长矩阵乘法这类重复性数学运算。
> GPU
GPU 不像 CPU 只有几个强力核心，而是将工作分发给数千个较小的核心，这些核心在不同数据上执行相同的指令。
这就是为什么 GPU 主宰了 AI 训练。这种并行处理模式完美契合神经网络所需的数学运算。
> TPU
术业有专攻，TPU 走得更远。
其计算核心是一个乘累加 (MAC) 单元网格，数据像波浪一样流动。
权重从一侧进入，激活值从另一侧进入，中间结果直接传播，无需每次都写回内存。
整个执行过程完全由编译器控制，而非硬件调度。Google 专门为神经网络工作负载设计了 TPU。
> NPU
这是边缘侧优化的变体。
架构围绕“神经网络计算引擎”构建，集成了大量的 MAC 阵列和片上 SRAM。不同于高带宽内存 (HBM)，NPU 使用低功耗系统内存。
其设计目标是在个位数瓦特的功耗预算内运行推理，比如智能手机、穿戴设备和物联网设备。
Apple Neural Engine 和 Intel 的 NPU 都属于这一类。
> LPU (语言处理单元)
Groq 推出的新角色。
该架构完全摒弃了关键路径上的片外内存，所有权重都存储在片上 SRAM 中。
执行过程完全确定且由编译器调度，这意味着零缓存未命中和零运行时调度开销。
取舍在于单颗芯片的内存容量有限，需要数百颗芯片互联才能跑起一个大模型。但延迟优势确实非常明显。
AI 算力已经从通用的灵活性 (CPU) 进化到极致的专业化 (LPU)。每一步演进都是在用通用性换取效率。
下图并排展示了这五种架构的内部设计。
注意连接这五者的线索：每一代架构的出现都是为了减少数据搬运。因为数学运算从来不是难点，如何够快地把数据喂给计算单元才是。
软件层面的博弈同样激烈。在 LLM 推理过程中，单个 GPU 每天会产生数 TB 的 KV 缓存，且几乎全部被丢弃并重新计算，这也是 Agent 工作负载昂贵的主要原因之一。
我写了一篇深度解析，介绍了解耦缓存层如何解决这一问题，并将首字生成速度提升了 14 倍。文章见下文。
同时也推荐关注 @lmcache 的 GitHub 仓库：https://github.com/LMCache/LMCache
(别忘了点个 star 🌟)
👉 轮到你了：这 5 种硬件中，你实际使用或部署过哪几种？
CPU vs GPU vs TPU vs NPU vs LPU, explained visually:
5 hardware architectures power AI today.
Each one makes a fundamentally different tradeoff between flexibility, parallelism, and memory access.
> CPU
It is built for general-purpose computing. A few powerful cores handle complex logic, branching, and system-level tasks.
It has deep cache hierarchies and off-chip main memory (DRAM). It's great for operating systems, databases, and decision-heavy code, but not that great for repetitive math like matrix multiplications.
> GPU
Instead of a few powerful cores, GPUs spread work across thousands of smaller cores that all execute the same instruction on different data.
This is why GPUs dominate AI training. The parallelism maps directly to the kind of math neural networks need.
> TPU
They go one step further with specialization.
The core compute unit is a grid of multiply-accumulate (MAC) units where data flows through in a wave pattern.
Weights enter from one side, activations from the other, and partial results propagate without going back to memory each time.
The entire execution is compiler-controlled, not hardware-scheduled. Google designed TPUs specifically for neural network workloads.
> NPU
This is an edge-optimized variant.
The architecture is built around a Neural Compute Engine packed with MAC arrays and on-chip SRAM, but instead of high-bandwidth memory (HBM), NPUs use low-power system memory.
The design goal is to run inference at single-digit watt power budgets, like smartphones, wearables, and IoT devices.
Apple Neural Engine and Intel's NPU follow this pattern.
> LPU (Language Processing Unit)
This is the newest entrant, by Groq.
The architecture removes off-chip memory from the critical path entirely. All weight storage lives in on-chip SRAM.
Execution is fully deterministic and compiler-scheduled, which means zero cache misses and zero runtime scheduling overhead.
The tradeoff is that it provides limited memory per chip, which means you need hundreds of chips linked together to serve a single large model. But the latency advantage is real.
AI compute has evolved from general-purpose flexibility (CPU) to extreme specialization (LPU). Each step trades some level of generality for efficiency.
The visual below maps the internal architecture of all five side by side.
Notice the thread connecting all five. Every generation exists to move data less, because the math was never the hard part. Feeding the math units fast enough is.
The same battle plays out one layer up in software. During LLM inference, a single GPU produces terabytes of KV cache per day, and nearly all of it gets thrown away and recomputed, which is a big reason agent workloads cost what they do.
I wrote a full breakdown of how a disaggregated caching layer fixes this, with up to 14x faster time-to-first-token. The article is quoted below.
You should also check the @lmcache GitHub repo: https://github.com/LMCache/LMCache
(don't forget to star 🌟)
👉 Over to you: Which of these 5 have you actually worked with or deployed on?
5 hardware architectures power AI today.
Each one makes a fundamentally different tradeoff between flexibility, parallelism, and memory access.
> CPU
It is built for general-purpose computing. A few powerful cores handle complex logic, branching, and system-level tasks.
It has deep cache hierarchies and off-chip main memory (DRAM). It's great for operating systems, databases, and decision-heavy code, but not that great for repetitive math like matrix multiplications.
> GPU
Instead of a few powerful cores, GPUs spread work across thousands of smaller cores that all execute the same instruction on different data.
This is why GPUs dominate AI training. The parallelism maps directly to the kind of math neural networks need.
> TPU
They go one step further with specialization.
The core compute unit is a grid of multiply-accumulate (MAC) units where data flows through in a wave pattern.
Weights enter from one side, activations from the other, and partial results propagate without going back to memory each time.
The entire execution is compiler-controlled, not hardware-scheduled. Google designed TPUs specifically for neural network workloads.
> NPU
This is an edge-optimized variant.
The architecture is built around a Neural Compute Engine packed with MAC arrays and on-chip SRAM, but instead of high-bandwidth memory (HBM), NPUs use low-power system memory.
The design goal is to run inference at single-digit watt power budgets, like smartphones, wearables, and IoT devices.
Apple Neural Engine and Intel's NPU follow this pattern.
> LPU (Language Processing Unit)
This is the newest entrant, by Groq.
The architecture removes off-chip memory from the critical path entirely. All weight storage lives in on-chip SRAM.
Execution is fully deterministic and compiler-scheduled, which means zero cache misses and zero runtime scheduling overhead.
The tradeoff is that it provides limited memory per chip, which means you need hundreds of chips linked together to serve a single large model. But the latency advantage is real.
AI compute has evolved from general-purpose flexibility (CPU) to extreme specialization (LPU). Each step trades some level of generality for efficiency.
The visual below maps the internal architecture of all five side by side.
Notice the thread connecting all five. Every generation exists to move data less, because the math was never the hard part. Feeding the math units fast enough is.
The same battle plays out one layer up in software. During LLM inference, a single GPU produces terabytes of KV cache per day, and nearly all of it gets thrown away and recomputed, which is a big reason agent workloads cost what they do.
I wrote a full breakdown of how a disaggregated caching layer fixes this, with up to 14x faster time-to-first-token. The article is quoted below.
You should also check the @lmcache GitHub repo: https://github.com/LMCache/LMCache
(don't forget to star 🌟)
👉 Over to you: Which of these 5 have you actually worked with or deployed on?
7
44
278
22.8K
