1 pointby tanishqxyzan hour ago1 comment
  • tanishqxyzan hour ago
    How I cut my agent’s inference bill by 85% with two cache breakpoints.

    Every part of the prompt has a stability horizon, how long it stays true. Sort the prompt by it, and give each part a cache that lives about as long: cache each part of the prompt for no longer than it stays true.