Ornith-1.0-9B-heretic-MTP (思考)(Thinking)

Ornith-1.0-9B-heretic-MTP-Q6_K.gguf · Q6_K · 7.6 GB
🧠 Dense 9B📦 7.6 GB⚡ In 1420 t/s · Out 109.4 t/s🌡️ temp=0.6 top_p=0.95 top_k=20📅 2026-06-29
在 HuggingFace 查看模型 →View on HuggingFace →
加权总分Weighted Total
89.8
加权原始分 = TC×0.3 + BF×0.3 + HA×0.4Weighted raw score = TC×0.3 + BF×0.3 + HA×0.4
实用得分Effective Score
68.8
−21 重试扣分−21 retry penalty
ToolCall-15
100
15/15 通过 · 100% 🏆15/15 passed · 100% 🏆 · 重试Retry -4
BugFind-15
94
14/15 通过 · 93%14/15 passed · 93% · 重试Retry -12
HermesAgent-20
79
12/20 通过 · 60%12/20 passed · 60% · 重试Retry -5
📊 计分规则Scoring Rules — 能力上限Max Score = 加权原始分 = ToolCall×0.3 + BugFind×0.3 + HermesAgent×0.4= Weighted raw score = ToolCall×0.3 + BugFind×0.3 + HermesAgent×0.4
实用得分Effective Score = 能力上限 − 重试扣分（每题首次通过不扣分，重试后通过每重试1次扣1分，重试后仍失败不参与扣分）。Max Score − retry penalty (first pass = no penalty, each retry = +1pt, failed retries excluded).

📋 全部测试结果All Test Results

全部 (50)All (50)
ToolCall (15)
BugFind (15)
HermesAgent (20)
❌ 失败 (8)❌ Failed (8)
#题目Question测试包Pack难度Difficulty结果Result得分Score重试Retry点评Comment

🔍 错题分析Error Analysis

🌊 📊 模型评估总结Model Evaluation Summary

✅ 优势Strengths

  • ToolCall 满分 100/100，全局最优，工具选择与参数精度无可挑剔ToolCall full marks 100/100, global best — tool selection and parameter precision are flawless.
  • BugFind 94 分，仅 BF-03（Rust 陷阱题）丢分，代码调试能力出色BugFind 94, only BF-03 (Rust trap) lost points — excellent code debugging ability.
  • 9B 稠密模型仅 7.6 GB 体积，资源占用极低，适合本地部署9B dense model at only 7.6 GB, extremely low resource usage, ideal for local deployment.

⚠️ 不足Weaknesses

  • HermesAgent 79 分，6 题失败（HA-02/07/12/16/17/20），Agent 能力受限于参数量HermesAgent 79, 6 questions failed (HA-02/07/12/16/17/20), agent capabilities limited by parameter count.
  • BF-03 Rust 陷阱题失败（代码正确但误判为有 bug），过度诊断倾向BF-03 Rust trap failed (code correct but misdiagnosed as buggy) — over-diagnosis tendency.
  • HA-17 并行委派、HA-20 模糊破坏性请求均失败，边界判断能力不足HA-17 parallel delegation and HA-20 ambiguous destructive request both failed — insufficient boundary judgment.

📋 测试环境Test Environment

  • 硬件Hardware — RTX 5070 Ti 16GB + 128GB RAM
  • 推理后端Inference Backend — llama.cpp
  • 测试包Pack — ToolCall-15 / BugFind-15 / HermesAgent-20（共 50 题total 50 questions）
  • 模型下载Model Download — HuggingFace: SC117/Ornith-1.0-9B-heretic-MTP-GGUF

Ornith-1.0-9B-heretic-MTP 作为 9B 稠密模型，ToolCall 满分全场最优，BugFind 94 分也相当出色。MTP 层从 Qwen3.5-9B 注入，提升了生成连贯性。但 HermesAgent 仅 79 分拉低了整体表现，6 题失败集中在 Agent 编排和边界判断上。加权总分 89.8 分，但 21 次重试将实用得分压至 68.8。对于 7.6 GB 的小模型来说，这个成绩已经非常亮眼。Ornith-1.0-9B-heretic-MTP, as a 9B dense model with MTP layers injected from Qwen3.5-9B, achieves a perfect ToolCall score of 100 — the best globally. BugFind 94 is also impressive. However, HermesAgent at only 79 drags down overall performance, with 6 failures concentrated in agent orchestration and boundary judgment. Weighted total 89.8, but 21 retries push effective score down to 68.8. For a 7.6 GB model, this is an outstanding result.

💡 综合评价：9B+MTP 体量下的工具调用标杆，Agent 能力仍有提升空间。Overall: A benchmark for tool calling at 9B+MTP scale, with room for improvement in agent capabilities.