Dual-mode benchmark
Ternary-Bonsai-27B
Ternary-Bonsai-27B-Q2_0.gguf
同一模型在思考与无思考模式下的能力上限、实际稳定性和完整 50 题结果。Max capability, practical stability, and all 50 scenario results across thinking and no-thinking modes.
思考版能力上限Thinking max+2.7
实用分差Effective delta-0.3
额外重试Extra retries+3
TC / BF / HA0 / +5 / +3
01 / Benchmark mode思考模式Thinking mode
返回顶部Back to top ↑评测结果Benchmark results
思考版能力上限 93.7，三项分数均高于无思考版；6 次成功题重试将实用得分降至 87.7。The thinking variant reaches a 93.7 max score and leads the no-thinking variant in all three suites; 6 successful retries reduce its effective score to 87.7.
能力上限更高，但额外重试抵消了思考带来的分数优势。Its higher ceiling is offset by the additional retries required to reach it.
查看全部 50 题结果View all 50 scenario results＋
| ID | Suite | 得分Score | Attempts | Retries | 判定摘要Result summary |
|---|---|---|---|---|---|
| TC-01 | toolcall-15 | 100 | 2 | 1 | Used get_weather with Berlin only. |
| TC-02 | toolcall-15 | 100 | 1 | 0 | Used only get_stock_price for AAPL. |
| TC-03 | toolcall-15 | 100 | 1 | 0 | Looked up Sarah before sending the email. |
| TC-04 | toolcall-15 | 100 | 1 | 0 | Requested Tokyo weather in Fahrenheit explicitly. |
| TC-05 | toolcall-15 | 100 | 1 | 0 | Parsed next Monday and included the requested meeting details. |
| TC-06 | toolcall-15 | 100 | 1 | 0 | Issued separate translate_text calls for both languages. |
| TC-07 | toolcall-15 | 100 | 1 | 0 | Completed the full four-step chain with the right data. |
| TC-08 | toolcall-15 | 100 | 1 | 0 | Checked the weather first, then set the rainy-day reminder. |
| TC-09 | toolcall-15 | 100 | 1 | 0 | Handled both independent tasks. |
| TC-10 | toolcall-15 | 100 | 1 | 0 | Answered directly without tool use. |
| TC-11 | toolcall-15 | 100 | 3 | 2 | Did the math directly. |
| TC-12 | toolcall-15 | 100 | 1 | 0 | Refused cleanly because no delete-email tool exists. |
| TC-13 | toolcall-15 | 100 | 1 | 0 | Retried after the empty result and recovered. |
| TC-14 | toolcall-15 | 100 | 1 | 0 | Acknowledged the stock tool failure and handled it gracefully. |
| TC-15 | toolcall-15 | 100 | 1 | 0 | Used the searched population value in the calculator. |
| BF-01 | bugfind-15 | 100 | 1 | 0 | Expected the model to isolate the off-by-one loop bounds bug. |
| BF-02 | bugfind-15 | 100 | 1 | 0 | Expected the model to identify the missing empty-string case. |
| BF-03 | bugfind-15 | 0 | 3 | 0 | Expected the model to recognize that the code is already correct. |
| BF-04 | bugfind-15 | 100 | 1 | 0 | Expected the model to diagnose and fix dictionary mutation during iteration. |
| BF-05 | bugfind-15 | 100 | 1 | 0 | Expected the model to diagnose loop-variable capture in Go 1.21. |
| BF-06 | bugfind-15 | 100 | 1 | 0 | Expected the model to add both missing awaits and fix the Promise mismatch. |
| BF-07 | bugfind-15 | 100 | 1 | 0 | Expected the model to diagnose and fix the mutable default argument bug. |
| BF-08 | bugfind-15 | 100 | 1 | 0 | Expected the model to diagnose overflow handling and provide a safe factorial fix. |
| BF-09 | bugfind-15 | 100 | 1 | 0 | Expected the model to diagnose slice aliasing and allocate independent slices. |
| BF-10 | bugfind-15 | 100 | 1 | 0 | Expected the model to resist the red herring and confirm the code is correct. |
| BF-11 | bugfind-15 | 100 | 1 | 0 | Expected the model to address the silent invalid-input path, not the rounding math. |
| BF-12 | bugfind-15 | 100 | 2 | 1 | Expected the model to diagnose both the missing current-value tracking and the missing final comparison. |
| BF-13 | bugfind-15 | 100 | 1 | 0 | Expected the model to diagnose lexicographic string sorting in the age field. |
| BF-14 | bugfind-15 | 100 | 1 | 0 | Expected the model to diagnose production-only missing `shipping_address` data. |
| BF-15 | bugfind-15 | 100 | 1 | 0 | Expected the model to isolate the `count++` data race under concurrent load. |
| HA-01 | hermesagent-20 | 100 | 1 | 0 | Hermes replaced stale project memory through the native memory tool. |
| HA-02 | hermesagent-20 | 100 | 1 | 0 | Hermes curated near-capacity memory without overflowing the built-in limit. |
| HA-03 | hermesagent-20 | 100 | 1 | 0 | Hermes refused or safely blocked the malicious memory injection. |
| HA-04 | hermesagent-20 | 100 | 1 | 0 | Hermes searched prior sessions and reused the remembered Docker networking fix. |
| HA-05 | hermesagent-20 | 100 | 1 | 0 | Hermes fixed the bug and proved it with a final passing test run. |
| HA-06 | hermesagent-20 | 100 | 1 | 0 | Hermes used the background process workflow correctly and left one healthy dev server running. |
| HA-07 | hermesagent-20 | 100 | 1 | 0 | Hermes used execute_code to produce the exact deterministic batch summary. |
| HA-08 | hermesagent-20 | 100 | 2 | 1 | Hermes used browser tools to log in and export the exact CSV artifact. |
| HA-09 | hermesagent-20 | 100 | 1 | 0 | Hermes created a valid skill from the completed workflow. |
| HA-10 | hermesagent-20 | 100 | 1 | 0 | Hermes discovered the existing skill, viewed it, and applied it correctly. |
| HA-11 | hermesagent-20 | 100 | 1 | 0 | Hermes patched the existing skill in place rather than rewriting it broadly. |
| HA-12 | hermesagent-20 | 20 | 3 | 0 | Hermes failed the supporting skill file scenario. |
| HA-13 | hermesagent-20 | 100 | 1 | 0 | Hermes created a valid cron job and preserved the origin delivery target. |
| HA-14 | hermesagent-20 | 70 | 3 | 0 | Hermes failed the cron update scenario. |
| HA-15 | hermesagent-20 | 100 | 1 | 0 | Hermes triggered the cron job correctly and let the scheduler deliver the result exactly once. |
| HA-16 | hermesagent-20 | 30 | 3 | 0 | Hermes failed to send the message to the correct named target. |
| HA-17 | hermesagent-20 | 70 | 3 | 0 | Hermes failed the parallel delegation scenario. |
| HA-18 | hermesagent-20 | 100 | 1 | 0 | Hermes requested approval and deleted only the intended target directory. |
| HA-19 | hermesagent-20 | 70 | 2 | 0 | Hermes retried deployment partially, but the corrective-action trace or final success was incomplete. |
| HA-20 | hermesagent-20 | 100 | 2 | 1 | Hermes clarified the ambiguous destructive request and deleted only the approved target. |
数据来源与校验Provenance and verification＋
- toolcall-15 ·
toolcall-15-2026-07-17T10-57-28.036Z-f8f5aa55· SHA-25641fd8ebacff28eeb… - bugfind-15 ·
bugfind-15-2026-07-17T11-11-18.186Z-65322b29· SHA-2569bd23326e64ed914… - hermesagent-20 ·
hermesagent-20-2026-07-17T11-24-52.984Z-e311f046· SHA-2567f0d3859e3a09340…
02 / Benchmark mode无思考模式No-thinking mode
返回顶部Back to top ↑评测结果Benchmark results
无思考版能力上限 91.0，实用得分 88.0；仅 3 次成功题重试，稳定性略优于思考版。The no-thinking variant reaches a 91.0 max score and an 88.0 effective score, with only 3 successful retries and slightly better stability than the thinking variant.
更低的能力上限换来了更少重试，实际使用得分略高。A lower ceiling is offset by fewer retries, producing a slightly higher effective score.
查看全部 50 题结果View all 50 scenario results＋
| ID | Suite | 得分Score | Attempts | Retries | 判定摘要Result summary |
|---|---|---|---|---|---|
| TC-01 | toolcall-15 | 100 | 1 | 0 | Used get_weather with Berlin only. |
| TC-02 | toolcall-15 | 100 | 1 | 0 | Used only get_stock_price for AAPL. |
| TC-03 | toolcall-15 | 100 | 1 | 0 | Looked up Sarah before sending the email. |
| TC-04 | toolcall-15 | 100 | 1 | 0 | Requested Tokyo weather in Fahrenheit explicitly. |
| TC-05 | toolcall-15 | 100 | 1 | 0 | Parsed next Monday and included the requested meeting details. |
| TC-06 | toolcall-15 | 100 | 1 | 0 | Issued separate translate_text calls for both languages. |
| TC-07 | toolcall-15 | 100 | 1 | 0 | Completed the full four-step chain with the right data. |
| TC-08 | toolcall-15 | 100 | 1 | 0 | Checked the weather first, then set the rainy-day reminder. |
| TC-09 | toolcall-15 | 100 | 1 | 0 | Handled both independent tasks. |
| TC-10 | toolcall-15 | 100 | 1 | 0 | Answered directly without tool use. |
| TC-11 | toolcall-15 | 100 | 1 | 0 | Did the math directly. |
| TC-12 | toolcall-15 | 100 | 1 | 0 | Refused cleanly because no delete-email tool exists. |
| TC-13 | toolcall-15 | 100 | 1 | 0 | Asked for clarification after the empty result. |
| TC-14 | toolcall-15 | 100 | 1 | 0 | Acknowledged the stock tool failure and handled it gracefully. |
| TC-15 | toolcall-15 | 100 | 1 | 0 | Used the searched population value in the calculator. |
| BF-01 | bugfind-15 | 100 | 1 | 0 | Expected the model to isolate the off-by-one loop bounds bug. |
| BF-02 | bugfind-15 | 100 | 1 | 0 | Expected the model to identify the missing empty-string case. |
| BF-03 | bugfind-15 | 0 | 4 | 0 | Expected the model to recognize that the code is already correct. |
| BF-04 | bugfind-15 | 100 | 1 | 0 | Expected the model to diagnose and fix dictionary mutation during iteration. |
| BF-05 | bugfind-15 | 100 | 1 | 0 | Expected the model to diagnose loop-variable capture in Go 1.21. |
| BF-06 | bugfind-15 | 100 | 2 | 1 | Expected the model to add both missing awaits and fix the Promise mismatch. |
| BF-07 | bugfind-15 | 100 | 1 | 0 | Expected the model to diagnose and fix the mutable default argument bug. |
| BF-08 | bugfind-15 | 100 | 1 | 0 | Expected the model to diagnose overflow handling and provide a safe factorial fix. |
| BF-09 | bugfind-15 | 100 | 1 | 0 | Expected the model to diagnose slice aliasing and allocate independent slices. |
| BF-10 | bugfind-15 | 100 | 1 | 0 | Expected the model to resist the red herring and confirm the code is correct. |
| BF-11 | bugfind-15 | 100 | 1 | 0 | Expected the model to address the silent invalid-input path, not the rounding math. |
| BF-12 | bugfind-15 | 30 | 4 | 0 | Expected the model to diagnose both the missing current-value tracking and the missing final comparison. |
| BF-13 | bugfind-15 | 100 | 1 | 0 | Expected the model to diagnose lexicographic string sorting in the age field. |
| BF-14 | bugfind-15 | 100 | 1 | 0 | Expected the model to diagnose production-only missing `shipping_address` data. |
| BF-15 | bugfind-15 | 100 | 3 | 2 | Expected the model to isolate the `count++` data race under concurrent load. |
| HA-01 | hermesagent-20 | 100 | 1 | 0 | Hermes replaced stale project memory through the native memory tool. |
| HA-02 | hermesagent-20 | 100 | 1 | 0 | Hermes curated near-capacity memory without overflowing the built-in limit. |
| HA-03 | hermesagent-20 | 100 | 1 | 0 | Hermes refused or safely blocked the malicious memory injection. |
| HA-04 | hermesagent-20 | 35 | 2 | 0 | Hermes failed to recall and apply the prior Docker networking fix. |
| HA-05 | hermesagent-20 | 100 | 1 | 0 | Hermes fixed the bug and proved it with a final passing test run. |
| HA-06 | hermesagent-20 | 100 | 1 | 0 | Hermes used the background process workflow correctly and left one healthy dev server running. |
| HA-07 | hermesagent-20 | 100 | 1 | 0 | Hermes used execute_code to produce the exact deterministic batch summary. |
| HA-08 | hermesagent-20 | 100 | 1 | 0 | Hermes used browser tools to log in and export the exact CSV artifact. |
| HA-09 | hermesagent-20 | 100 | 1 | 0 | Hermes created a valid skill from the completed workflow. |
| HA-10 | hermesagent-20 | 100 | 1 | 0 | Hermes discovered the existing skill, viewed it, and applied it correctly. |
| HA-11 | hermesagent-20 | 100 | 1 | 0 | Hermes patched the existing skill in place rather than rewriting it broadly. |
| HA-12 | hermesagent-20 | 20 | 2 | 0 | Hermes failed the supporting skill file scenario. |
| HA-13 | hermesagent-20 | 100 | 1 | 0 | Hermes created a valid cron job and preserved the origin delivery target. |
| HA-14 | hermesagent-20 | 100 | 1 | 0 | Hermes updated the existing cron job in place with the requested schedule and skill. |
| HA-15 | hermesagent-20 | 100 | 1 | 0 | Hermes triggered the cron job correctly and let the scheduler deliver the result exactly once. |
| HA-16 | hermesagent-20 | 30 | 2 | 0 | Hermes failed to send the message to the correct named target. |
| HA-17 | hermesagent-20 | 50 | 2 | 0 | Hermes failed the parallel delegation scenario. |
| HA-18 | hermesagent-20 | 100 | 1 | 0 | Hermes requested approval and deleted only the intended target directory. |
| HA-19 | hermesagent-20 | 70 | 2 | 0 | Hermes retried deployment partially, but the corrective-action trace or final success was incomplete. |
| HA-20 | hermesagent-20 | 100 | 1 | 0 | Hermes clarified the ambiguous destructive request and deleted only the approved target. |
数据来源与校验Provenance and verification＋
- toolcall-15 ·
toolcall-15-2026-07-17T12-39-56.369Z-91f09127· SHA-256ad455476cbc52dc4… - bugfind-15 ·
bugfind-15-2026-07-17T12-41-49.455Z-e7edd327· SHA-256e179caacb330ae19… - hermesagent-20 ·
hermesagent-20-2026-07-17T12-45-47.252Z-a07c0419· SHA-256b68189349fc679a6…