Same model, same Q4_K_M label: 5.02, 5.07 and 5.27 bits per weight | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		Same model, same Q4_K_M label: 5.02, 5.07 and 5.27 bits per weight (github.com/logxio)
		19 points by logickkk1 61 days ago \| hide \| past \| favorite \| 2 comments

p_stuart82 61 days ago [–]

Does this catch intermittent fallback between requests?

logickkk1 61 days ago | [–]

Yes, picchio monitors a running llama-server on a timer and flags when the prefill/decode ratio goes CPU-shaped. I hot-swapped one mid-run. probe 4 caught ENGAGED -> NOT ENGAGED.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact