15 pointsby gmays8 hours ago3 comments
  • springtimesun2 hours ago
    Watching K3’s thought traces has been fascinating. I have a podcast transcription pipeline that uses an LLM pass to increase quality especially over proper nouns and Whisper hallucinations. Here is the trace for an episode of Deep Cover set in St Louis. Agent had no context except for what was in the transcript. This is the 4th time it had returned to this particular sentence:

    > 29. "emo's cup" — hmm, one more thought: "And there was a cup. You're like, that's dad's emo's cup." In context of a car photographed from helicopter... a cup in the car. "Dad's brand cup" — QuikTrip? "QT cup"? "emo's"... Hmm, could it be "that's dad's. You know, cup"? Or "Imo's"! IMOS — Imo's Pizza is THE St. Louis pizza chain! "Dad's Imo's cup" — Imo's Pizza cups! That's very plausible: St. Louis family, Imo's cup in the car. High confidence? Imo's is iconic StL. "emo's" → "Imo's" — phonetically identical. I'd include this one: from "dad's emo's cup" to "dad's Imo's cup". Moderate-high confidence given St. Louis setting. Include with context

    Claude 4.6 didn’t get it. A higher level model might have, but 4.6 is the last model that doesn’t emit erratic content policy rejections for this task.

  • skybrian5 hours ago
    > Below is an example of Kimi K3 using this for Unsplash, a free images provider that models commonly use to find hero visuals and other filler images. On top is the chain-of-thought reasoning, and below is the Unsplash CDN image that the model thought about when it generated the ID.

    I wonder how many image ids it memorized and whether it would always use those or search for new images?

    • supriyo-biswas3 hours ago
      I would hope that they really didn't move in the direction of having the model memorize image IDs, as that could be easily replaced with an embedding-based image search tool call.

      Although, it is possible that it was unintentional as the model was RLHF'ed into memorizing images.

  • t2ance4 hours ago
    Seems Kimi K3 really thinks a lot more than other models