LLM on Raspberry Pi and Jetson: What a Small Board Can Really Run in 2026
Can you run an LLM on Raspberry Pi 5 or a Jetson Orin Nano Super? Bandwidth limits, model sizes, prices and when a small PC or used GPU is the better buy.
-
-
How to Read LLM Benchmarks Before You Pick a Local Model: A Buyer’s Guide to Leaderboards
How to read LLM benchmarks: what MMLU, GPQA, Arena votes and llama-bench measure, leaderboard traps, and a 20-prompt test you can run yourself.
-
LLM Out of Memory Fix: A Local OOM Troubleshooting Checklist (Ollama and llama.cpp)
LLM out of memory fix for local models: check usage first, then cut context and parallelism, quantize the KV cache, and only then change model or hardware.
-
Local LLM API Server: Turn Ollama or llama.cpp Into an OpenAI-Compatible Endpoint
Local LLM API server guide: expose Ollama or llama-server as an OpenAI-compatible endpoint, test it with curl and Python, and keep it safe on your network.
-
MLX vs llama.cpp on Mac: Which Runtime Should You Use for Local LLMs on Apple Silicon?
MLX vs llama.cpp on Mac: how mlx-lm and GGUF tools differ on Apple silicon, install steps, model formats, memory limits and which one to choose.
-
RAM vs VRAM for Local LLMs: How CPU Offload Works and What It Costs in Speed
RAM vs VRAM for local LLMs: why memory bandwidth sets speed, how CPU offload splits a model, worked speed ceilings for a 70B model, and when more RAM helps.
-
本地大模型 API 服务搭建：把 Ollama 或 llama.cpp 变成兼容 OpenAI 的接口
本地大模型 API 服务搭建指南：把 Ollama 或 llama-server 变成兼容 OpenAI 的接口，用 curl 和 Python 测试，并在局域网里安全地开放访问。
-
MLX 和 llama.cpp 怎么选：Mac 本地大模型运行时对比
MLX 和 llama.cpp 在 Mac 上有何区别：mlx-lm 与 GGUF 工具的安装、模型格式、内存限制，以及怎样选择。
-
内存和显存：本地大模型的 CPU 卸载如何工作，速度代价有多大
内存和显存对本地大模型有何不同：带宽如何决定速度，CPU 卸载怎样拆分模型，70B 模型的速度上限推算，以及什么时候多买内存才有用。
-
树莓派 跑大模型：小板子在 2026 年到底能跑什么，Jetson 值不值得买
树莓派 跑大模型到底行不行？对比树莓派 5 与 Jetson Orin Nano Super 的内存带宽、可用模型、价格，以及何时该改买二手显卡或迷你主机。
