Full: https://reddit.com/r/kimi/comments/1v9aqsi/comment/p0cbe0n/
Mirror: https://redlib.us.catsarch.com/r/kimi/comments/1v9aqsi/k3256...
First you spend like 2-3 hours working on a plan, once you have that you just tell the model to go and implement it, do adversarial sub-agent review loops before each commit and also make sure that all tooling and tests pass (including coverage requirements). You do need to poke it in a slightly different direction every few hours, though. Not even any novel work, just some refactoring and SSE notification hardening, bug fixes, alongside environment tuning and getting rid of some bottlenecks (also migrated from Oracle to PostgreSQL but that's mostly done).
That said, Kimi somehow manages to use less context in the main thread than Anthropic's models (even when you use sub-agents and also dynamic workflows in Claude Code), might have something to do with either how the model is tuned or their Kimi Code harness - because even in most of the longer form sessions it doesn't seem to fill up quite as quickly (note: because the kimi vis tool doesn't have a full summary view across all agents, these are the main long running agent stats across some sessions, not sub-agents):
total tokens cache hit rate wall time peak context
283M 98% 3963m 466k
258M 97% 2724m 467k
98M 94% 1353m 393k
67M 97% 614m 434k
75M 98% 1447m 498k
53M 99% 191m 375k
6M 96% 139m 124k
7M 98% 86m 118k
11M 99% 61m 147k
If I feel like the model and I explored a lot of options I won't want to keep context as it might be confusing.
I think the more you use it the better judge you are of whether you should purge, compress, or just keep the context before executing the plan.
Haven't gone back to it, have been using Claude Code on a private project I'm still architecting.
But yeah, I have no idea about anything about software because you made an assumption off very little to go by.
Having a lot of active context increases the per-token cost (flops issued and bytes read per token out) so it makes sense to pass that cost on to users. I'm actually surprised it's implemented as a hard cutoff instead of a smooth gradient.
Fewer nodes dedicated to prefill per instance, and fewer nodes in total since you don't need to support a higher KV cache.
Disaggregated inference also means they can tune the balance of compute dedicated to prefill seperately from decode
Doubt these are related, but it made me laugh a little.
They did exceed the expecations of pretty much everyone! I've also blogged about using the model, it's a bit on the slow side but pretty good!
> so were totally surprised and overwhelmed by the requests ... or do they just don't have the hardware being in china?
Yes, this is mostly the case: https://x.com/Kimi_Moonshot/status/2078855608565207130
As a user, I much prefer that to service disruptions or severely degraded or secretly quantized performance. However if I didn't have an account, I'd be pretty pissed off about not being able to give them money and become a user.
I am now rather pretty pissed towards antrophic for stopping my flow and forcing me to search for alternatives.
See Kimi Vendor Verifier.
Kimi had become that popular. I was a subscriber of Kimi back when latest version was Kimi K2. Later I unsubscribed because I jumped over to GLM subscription (they had amazing deal). Now when I wanted to try out Kimi K3 to find out what the fuzz was all about, I couldn’t subscribe to them.
I remember reading a post from Moonshot team about this, they are doing this because they are almost at peak capacity and want to reserve it to keep the quality for their current customers.
We are actually witnessing an open-weight model catching up at catching mainstream users attention. And instead of behaving like Anthropic, they actually care about their users experience.
When they first announced, I created an account but didn't subscribe, so I am stuck on a waitlist now.
It's not ideal that people can't join but as a user I am happy that they focused on serving existing customers.
Is this the exact same model just with less VRAM allocated for context window?
Cheaper?
I usually keep my context in chats below 256k anyways so this would be tremendous honestly.
For pressure at the country level leading to this kind of thing I think it's very unlikely. Here in Sweden it wouldn't just require a vote in the Swedish parliament and before this there'd have to be förarbeten and you can't just brazenly push things through with insane arguments, Swedish social convention goes against it-- and there's just no way to get it through.
It also might not even be legal. "We aren't at war with China and I'm a communist, and the US LLMs are so aligned with values inimical to my political ideology that this is interference with opinion formation" might be an actual legal argument that the ECHR or CJEU might actually have to accept.
Of course - Anthropic and OpenAI have an advantage in the amount they can subsidize the usage, but I think those days are waning.