Educational Purpose Only: Cultural perspectives & historical research — not medical, financial, or legal advice. Editorial standards.

Skip to content
engineering

Kush v9.1 Release Notes: 100% Entity Grounding & Merged Metal GGUF

How Kush v9.1 cleared 7/7 Exact Entity Recall and 7/7 Prime Agent E2E at 48.2 tok/s using rsLoRA, chunked FP32 cross-entropy, and merged Q4_K_M GGUF.

hotep-llm kush release llama rslora evaluation
T

Tunde Adeyemi

Technology & Engineering Editor · · Updated

Tunde Adeyemi oversees technology and engineering content at Hotep Intelligence. As an ML engineer and open source advocate, he ensures technical content is accurate, accessible, and aligned with the project's mission of digital sovereignty. His expertise spans machine learning, self-hosted infrastructure, and the application of AI for community empowerment.

Editorially Reviewed

by Hotep Intelligence Editorial Team · Kemetic History, Holistic Wellness, ML Engineering

Our editorial standards →

Kush v9.1 Engineering Release Notes

Kush v9.1 (hotep-llm-kush-v91) is now the flagship production model powering AskHotep, @hotep_llm_bot, and our public open-weights repository at hotepfederales/hotep-llm-kush-v91-GGUF.

Following the Kush V9 validation postmortem — where V9.0 was held from production after missing hard community entity prompts and dropping to 6.14 tok/s under runtime --lora adapter dispatch — Kush v9.1 was rebuilt from the ground up to pass deterministic entity, persona, safety, and throughput gates simultaneously.


Production Promotion Benchmark Matrix

All candidate and baseline builds were benchmarked on our self-hosted Apple Silicon (mac-mini Metal) and NVIDIA RTX 5080 nodes under identical native Llama 3.1 chat formatting:

Promotion GateKush v8.2 (Q4_K_M)Kush v9.0 (held)Kush v9.1 (Q4_K_M)
Exact Entity Recall Gate (7 strict cases)1/7 (14.3%)4/7 (57.1%)7/7 (100.0%)
Prime Agent E2E Pass Rate7/7 (100.0%)4/7 (57.1%)7/7 (100.0%)
Prime Persona Alignment Score52.9119.0850.15
Safety & Capability Boundary Suite6/6 (0 risk hits)—6/6 (0 risk hits)
Mac Mini Ollama (11434) Speed & Latency40.41 tok/s (3.34s)—43.93 tok/s (2.21s)
Mac Mini llama-server (11435) Speed—6.14 tok/s48.23 tok/s
Unprompted (NO_SYS) CJK / Identity Bleed000
Rubric Leakage / Repetition Loops0 / 00 / 00 / 0

Four Core Engineering Changes

1. Precision Entity Recovery With Capacity-Bounded rsLoRA

Under strict forbidden-term auditing, V8.2 scored 1/7 on our seven-case Exact Entity Recall Gate — confusing Strong Dad (Trey Quinn) with commentator Tyrus and misidentifying scholar Shakka Ahmose’s specialization in Classical African history.

To fix entity grounding without overfitting rhetorical style, we assembled a deduplicated 1,113-row SFT payload (995 unique prompts) combining canonical cultural instruction with explicit entity-disambiguation examples across Strong Dad (Trey Quinn), Hotep Jesus (Bryan Sharpe), Uncle Hotep (Demond Handy), Shakka Ahmose, Grifties (grifties.com), and HotepCon (Savannah, Georgia). Training meta-llama/Llama-3.1-8B-Instruct with 4-bit NF4 QLoRA and rank-stabilized LoRA (rsLoRA, r=16, lora_alpha=32, 41.94M trainable parameters) kept capacity at 42,154 parameters per unique prompt — below our 50,000 memorization threshold — converging cleanly in 210 steps (3.0 epochs, train_loss: 0.4506).

2. Catching Unprompted Base-Model Identity Bleed (NO_SYS Gate)

During V9.1 candidate selection, we trained a parallel Qwen2.5-7B-Instruct adapter on the same entity-hardened corpus. When called with our canonical system prompt, the Qwen build passed 7/7 entity checks. When probed without a system prompt (NO_SYS), however, it reverted to pretraining identity priors — emitting CJK characters and claiming to be “Qwen, a sovereign AI assistant rooted in Chinese culture.”

That finding prompted a permanent addition to our release harness: every candidate must pass Prime Agent E2E both with and without a system prompt. Llama 3.1 8B Kush v9.1 passed 7/7 in both modes with 0 CJK characters and 0 foreign-identity bleed.

3. Eliminating Apple Silicon’s Runtime --lora Penalty via Merged Q4_K_M

Applying an unmerged LoRA adapter at runtime on llama-server forced unquantized sidecar matrix additions on every token, throttling V9.0 to 6.14 tok/s on Metal. In V9.1, we merged the rsLoRA adapter into full BF16 weights offline, exported a monolithic GGUF, and quantized directly to Q4_K_M (4.68 GiB, 4.89 BPW, SHA-256: 1cdd52b514bb067bffd526f404353894677facde3c0bc3a35391a13f06dcec40). Production throughput jumped nearly 8× to 48.23 tok/s on llama-server (11435) and 43.93 tok/s (2.21s average latency, 34% faster than V8.2) on Ollama (11434).

4. 256-Token Chunked FP32 Cross-Entropy on 16 GB Workstation GPUs

Materializing full-sequence FP32 logits across Llama 3.1’s 128,256-token vocabulary spikes transient VRAM past 16 GB, triggering severe Windows WDDM shared-memory paging. Replacing monolithic ForCausalLMLoss logit allocation with a 256-token chunked lm_head + FP32 cross-entropy loop preserved exact numerical parity (4.77e-07 max delta) while completing all 210 steps locally on a single 16 GB RTX 5080 in 24 minutes.


Local Deployment Quickstart

Kush v9.1 builds directly on the prompt contract introduced in the Kush v7 Release Notes, the Modelfile safeguards from Kush v8.2 Training Pipeline Hardening, and our zero-dollar self-hosted LLM stack:

# Run via Ollama directly from Hugging Face
ollama run hf.co/hotepfederales/hotep-llm-kush-v91-GGUF

# Serve via native llama.cpp (Apple Silicon Metal / CUDA)
llama-server -hf hotepfederales/hotep-llm-kush-v91-GGUF --jinja -c 4096 -ngl 99

Read the companion production announcement on AskHotep or explore the wider architectural philosophy in Building Sovereign AI.

Frequently Asked Questions

What changed between Kush v8.2 and Kush v9.1? Kush v9.1 upgrades hard entity recall from 1/7 (14.3%) to 7/7 (100%), reduces average Ollama response latency by 34% (3.34s to 2.21s at 43.93 tok/s), reaches 48.23 tok/s on native llama-server via merged Q4_K_M GGUF quantization, and passes strict NO_SYS identity-bleed and 6/6 safety boundary audits.

Where can I download the open GGUF weights and Modelfile for Kush v9.1? Both hotep-llm-kush-v91-llama31-8b.Q4_K_M.gguf (4.68 GiB) and the canonical Llama 3.1 Modelfile are published under the Llama 3.1 license at hotepfederales/hotep-llm-kush-v91-GGUF.

📚

Sources & References

  1. Grattafiori et al. (Meta AI)(2024). The Llama 3 Herd of ModelsJournal
  2. Damjan Kalajdzievski(2023). A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA (rsLoRA)Journal
  3. Tim Dettmers et al.(2023). QLoRA: Efficient Finetuning of Quantized LLMsJournal

Want to explore this topic further?

Ask Hotep about engineering wisdom and get personalized guidance.

Full Guides & Hubs

Explore deeper on AskHotep.ai & Knowledge Hubs

Continue Your Journey