"What are llama.cpp and vLLM, and how do they relate?"	"llama.cpp = a single C/C++ binary with max hardware breadth (CPU/Metal/CUDA/ROCm/Vulkan), the air-gap default. vLLM = a Python GPU serving engine for max production throughput. They are COMPLEMENTARY, not competing — the deployment context decides which you reach for."	c3::ftdd09::recall
"What is llama.cpp's defining property, and why does it matter?"	"HARDWARE BREADTH. The same binary runs on CPU, Apple Metal, NVIDIA CUDA, AMD ROCm, and Vulkan. If hardware exists, llama.cpp can probably run a model on it. This makes it the tool for the long tail: laptops, edge boxes, air-gapped servers, M-series Macs. It's the engine under Ollama and LM Studio."	c3::ftdd09::recall
"What is vLLM's defining property, and when do you reach for it?"	"MAXIMUM THROUGHPUT ON GPUs under concurrent load. Built for requests/sec in a datacenter serving context. Reach for it when you have a GPU datacenter with concurrent users, throughput is the constraint, and you want the OpenAI-compatible serving API. The default for cloud-deployed open-model services."	c3::ftdd09::application
"What are the two innovations that make vLLM win production throughput?"	"(1) PagedAttention — manages the KV cache like VIRTUAL MEMORY (fixed-size blocks/pages allocated on demand), eliminating fragmentation so many more concurrent requests fit per GPU. (2) Continuous batching — admits new requests at the ITERATION/TOKEN level as slots free, keeping the GPU saturated. (Kwon et al., SOSP 2023.)"	c3::ftdd09::recall
"Explain PagedAttention and the problem it solves."	"During generation, a transformer stores keys/values per token (the KV cache). Naive serving pre-allocates a contiguous block per request sized for max sequence length — most requests don't use it all, causing severe fragmentation/waste. PagedAttention manages the KV cache like OS virtual memory: fixed-size blocks allocated on demand. Eliminates fragmentation; memory only used for tokens actually generated."	c3::ftdd09::analysis
"Explain continuous batching and how it differs from static batching."	"Static batching waits to fill a batch, processes it, then starts the next — if one request is longer, short ones finish early and slots sit idle. Continuous batching admits new requests at the ITERATION/TOKEN level as slots free. The GPU stays saturated. Combined with PagedAttention, there's always work and always memory for new requests."	c3::ftdd09::analysis
"What quant format does llama.cpp use, and what are its quant schemes called?"	"GGUF (GPT-Generated Unified Format) — a single container bundling weights, tokenizer, metadata. Its quantization produces k-quants: Q4_K_M, Q5_K_M, Q8_0. These are block-wise schemes that adapt precision per weight block, outperforming naive round-to-nearest at the same bit width. Q4_K_M is the widely-recommended default."	c3::ftdd09::recall
"What quant formats does vLLM serve?"	"GPU-native formats: AWQ (Activation-aware Weight Quantization), GPTQ (post-training quantization), and FP8 (native 8-bit float on Hopper/Blackwell). Also FP16/BF16 if VRAM allows and you want zero quantization loss. All are optimized for CUDA/ROCm memory bandwidth and compute patterns."	c3::ftdd09::recall
"Can you serve a GGUF in vLLM? Can you serve an AWQ in llama.cpp?"	"NO to both. The formats are tied to their engines' kernels — you CANNOT cross-serve. GGUF runs in llama.cpp only. AWQ/GPTQ/FP8 run in vLLM only. This means the quant choice is DOWNSTREAM of the serving choice: decide the server first (by deployment context), then export to its format."	c3::ftdd09::application
"A team ships a model to both an air-gapped CPU box and a cloud GPU service. What two export artifacts do they maintain?"	"Two exports from the same trained weights: a GGUF (e.g., Q4_K_M) for the air-gapped CPU box (llama.cpp), and an AWQ or FP16 for the cloud GPU service (vLLM). The deployment context dictates the server, and the server dictates the format. Decide server first, then quantize."	c3::ftdd09::application
"What is the k-quant Q4_K_M, and why is it the recommended default for llama.cpp?"	"It's llama.cpp's block-wise 4-bit quantization scheme (the _K_ variants adapt precision per weight block). It's the recommended default because it's a good quality-to-size trade-off for most models — the block-wise scaling preserves quality at low bit-widths better than naive quantization, which is why llama.cpp can run a usable 7B on a consumer laptop."	c3::ftdd09::analysis
"Why does vLLM's OpenAI-compatible API matter for adoption?"	"Any client written against the OpenAI SDK works against vLLM with only a base-URL change. You don't rewrite your application to switch from an OpenAI model to a self-hosted open model. This is a significant adoption advantage — the entire ecosystem of OpenAI-SDK tooling works unchanged."	c3::ftdd09::analysis
"What is llama.cpp's telemetry posture, and why is it load-bearing for sensitive domains?"	"Makes NO network calls, collects NO telemetry. A single static binary you can drop on a machine that has never touched the internet. Load-bearing because for HIPAA, government, and classified environments (Pillar 7), a server that phones home is a non-starter. The single binary = minimal assurance surface for security review."	c3::ftdd09::recall
"How does vLLM's assurance surface differ from llama.cpp's for air-gap requirements?"	"vLLM is clean for self-hosting (doesn't force telemetry), but it's a Python application with a dependency tree — a heavier assurance surface than llama.cpp's single static binary. For the STRICTEST air-gap requirements, the single binary wins on auditability. vLLM is also GPU-oriented, so it's not the tool for CPU-only air-gapped boxes anyway."	c3::ftdd09::analysis
"What is tensor parallelism in vLLM, and when is it needed?"	"Splitting a model's matrices across multiple GPUs that cooperate on each forward pass. Needed for models too large for one GPU (e.g., 70B in FP16 needs ~140GB, beyond one A100/H100). Lets vLLM serve large models on multi-GPU nodes. llama.cpp supports multi-GPU too, but vLLM's is tuned for the production serving regime."	c3::ftdd09::application
"What are SGLang and TensorRT-LLM, and when do you reach for them?"	"The high-end tier above vLLM. TensorRT-LLM (NVIDIA): lowest latency/highest throughput on NVIDIA hardware, extra compilation work — reach for it when on NVIDIA and you need the absolute margins. SGLang: RadixAttention prefix caching + structured generation — excels on shared-prefix or constrained-output workloads. Most teams start at vLLM and move up only with measured reason."	c3::ftdd09::analysis
"What does the Red Hat llama.cpp-vs-vLLM benchmark conclude?"	"llama.cpp offers broad deployability and lower-overhead single-stream performance; vLLM delivers substantially higher aggregate throughput under concurrent multi-user load on GPUs. Neither dominates — each wins in its regime. This matches 'two tools, two jobs.'"	c3::ftdd09::recall
"Why is choosing the quant before the server an anti-pattern?"	"Because the format is tied to the engine's kernels: GGUF runs in llama.cpp, AWQ/GPTQ/FP8 run in vLLM, and you cannot cross-serve. A team that quantizes to AWQ and then needs to deploy to an air-gapped CPU box has to re-export. Decide the server first (by deployment context), then export to its format."	c3::ftdd09::analysis
"Why is using llama.cpp for high-concurrency cloud serving an anti-pattern?"	"llama.cpp can serve concurrent requests but is not optimized for the PagedAttention + continuous-batching regime vLLM is built for. If you have GPUs and real concurrent traffic, vLLM's throughput advantage is large. Reaching for llama.cpp out of familiarity caps your capacity. Use the tool designed for the job."	c3::ftdd09::application
"For a new production GPU serving workload, where should you start, and when should you move up the tier?"	"Start at vLLM — it's the production GPU default (PagedAttention, continuous batching, OpenAI-compatible API). Move up to SGLang or TensorRT-LLM only when you have MEASURED data showing vLLM is your bottleneck: SGLang for shared-prefix/structured-output workloads, TensorRT-LLM for the lowest NVIDIA latency. Don't add complexity without data."	c3::ftdd09::application