A new guide from KDnuggets walks through running Qwythos-9B-Claude-Mythos-5-1M, a compact 9-billion-parameter reasoning and coding model, entirely on local hardware using llama.cpp, then wiring it into the Pi coding agent for practical use.
Qwythos is built on Qwen3.5 and distinguishes itself from typical small open models by combining strong reasoning and tool-use ability with a one-million-token context window and native function calling, according to its documentation. Reports describe the model as trained on reasoning traces derived from Anthropic's Claude Mythos and Fable model family, which is the source of its name and its notably strong coding performance for a model small enough to run on consumer hardware.
The guide's author tested the setup on an RTX 4070 Ti Super with 16GB of VRAM, running the Q6_K MTP quantized version comfortably at that size. Getting started involves setting a Hugging Face cache directory with sufficient free storage, particularly useful advice for cloud machines where the default home directory tends to be tightly constrained, before launching the model through llama.cpp's server command with GPU layers fully offloaded, a 100,000-token context size, flash attention enabled, and multi-token prediction speculative decoding turned on to speed up generation.
The MTP designation refers to multi-token prediction, a technique supported in recent llama.cpp builds that lets the model draft several tokens ahead speculatively before verifying them, which can meaningfully improve throughput for structured outputs like code without changing the underlying model weights.
Once the local server is running, the guide connects it to Pi, a coding agent, through a dedicated plugin built specifically to bridge Pi with a locally running llama.cpp server, without requiring an external API key. That plugin design matters for anyone trying to build a fully offline coding workflow, since it removes cloud API calls from the loop entirely once the model is downloaded and running.
A parallel writeup from DataCamp covering the same model reached a similar conclusion using a different front end, Hermes Agent, which adds browser chat and Discord automation on top of the same local llama.cpp endpoint. That author's takeaway was that Qwythos performed well and generation speed was strong, particularly for structured code output, though for day-to-day coding they still preferred lighter-weight agents like Pi or OpenCode for their simplicity and faster startup, reserving the more full-featured Hermes setup for cases needing messaging integrations or longer-running project support.
Taken together, both guides point to the same underlying shift: models distilled from frontier reasoning traces are increasingly small enough to run comfortably on a single consumer GPU, while still being capable enough for real agentic coding work, narrowing the gap between what used to require a hosted API call and what can now run entirely on a desk.