Tag
A plain explanation of the LLM KV cache, prefill versus decode, paging, and how a sizing mismatch let one user's data reach another in a recent incident.
audiotale.net team6 min read
Cerebras's worked example shows how a roughly 3-trillion-parameter open model is split across wafers with all weights kept in on-chip memory; we recheck the sums.
audiotale.net team7 min read
DeepInfer ships under Apache 2.0 with support for DeepSeek-V4.1-Flash on NVIDIA SM100 and Huawei Ascend 950, but leaves out the parts needed for a complete serving stack.
audiotale.net team6 min read