One of the authors here. We're diving deep into the workings of ai inference optimization. This deep-dive post reports on our actual runs with various agentic coding setups using quantization.

Some caveats here, which I'll get back to, but useful for anyone considering to use quantization.