troed/inferno
Watch
1
0
Fork
You've already forked inferno
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ABANDONED: A fork of NInfer with an initial focus on making Qwen 3.8 27B as viable as possible to run on a single 16GB 5060Ti GPU
918 commits 5 branches 0 tags 13 MiB
  • C++ 51.6%
  • Cuda 33.9%
  • Python 14%
  • CMake 0.5%
Find a file
Exact
Repository files (latest commit first)
Filename Latest commit message Latest commit date
troed af9293c03a Update README.md
2026-08-24 17:07:03 +02:00
.codex chore(format): establish clang-format baseline 2026-07-15 15:36:34 +08:00
apps feat(engine): map fp6 kv storage to packed u8 cache planes 2026-08-16 12:23:13 +02:00
bench feat(engine): register model-aware sampling defaults 2026-08-15 01:47:42 +08:00
docs docs: mark rotation spec implemented and clarify inverse-transform comment 2026-08-17 10:06:16 +02:00
eval chore(eval): add reasoning/gpqa configs and ignore request jsonl 2026-07-20 20:00:04 +08:00
examples/cli docs(qwen3.8): document public support 2026-08-15 00:41:15 +08:00
include/ninfer feat(engine): thread kv rotation flag from config to cache views 2026-08-16 21:51:17 +02:00
model-cards docs(model-card): publish qwen3.8-27b artifact 2026-08-15 00:26:56 +08:00
src feat(engine): guard rotation flag and finalize activation rotation 2026-08-17 09:46:54 +02:00
tests docs: mark rotation spec implemented and clarify inverse-transform comment 2026-08-17 10:06:16 +02:00
third_party feat(engine): add ordinary concurrent inference 2026-08-10 02:25:01 +08:00
tools fix(serving): unify multimodal context admission 2026-08-16 14:19:48 +08:00
.clang-format chore: add clang-format config 2026-06-26 00:53:21 +08:00
.clangd chore: configure clangd for cuda analysis 2026-07-07 15:28:56 +08:00
.dockerignore feat(docker): add runtime image 2026-07-28 15:18:41 +08:00
.gitignore chore: ignore project-local .worktrees directory 2026-08-15 22:58:43 +02:00
AGENTS.md Fork info updates 2026-08-16 21:22:45 +02:00
CMakeLists.txt refactor(qwen3.6): remove 27b activation dump 2026-08-09 02:25:03 +08:00
CONTRIBUTING.md docs: define contribution quality requirements 2026-07-27 00:49:12 +08:00
Dockerfile refactor(qwen3.6): remove 27b activation dump 2026-08-09 02:25:03 +08:00
LICENSE build: define public source build defaults 2026-07-19 01:15:33 +08:00
README.md Update README.md 2026-08-24 17:07:03 +02:00

README.md

This project was aborted after testing. The ffn layer offloading puts a hard cap on execution speed where going Blackwell-only doesn't gain anything meaningfully back.