Hacker News
new
top
best
ask
show
job
Same model, same Q4_K_M label: 5.02, 5.07 and 5.27 bits per weight
(
github.com
)
9 points
by
logickkk1
2 hours ago
1 comment
p_stuart82
2 hours ago
Does this catch intermittent fallback between requests?
logickkk1
an hour ago
Yes, picchio monitors a running llama-server on a timer and flags when the prefill/decode ratio goes CPU-shaped. I hot-swapped one mid-run. probe 4 caught ENGAGED -> NOT ENGAGED.