Patterns for Reducing App LLM Costs (Video Series)(usagetap.com)
1 pointby troymagennis8 hours ago1 comment
  • troymagennis8 hours ago
    I've been documenting ways to reduce my applications LLM spend for the last year. I've put together the first 6 parts on youtube. Its pretty easy to reduce spend by 60-80% from my eperience by just getting the basics right. I'd love to hear YOUR ideas as well, as this is an important part of launching an AI first app as a startup where saving cash is essential.

    Troy.

    • bhagyeshsp8 hours ago
      Hi Troy,

      I skimmed through the outline. Will take a look at the individual videos when I'm on PC.

      But I have been through this cost-saving phase. I didn't see "prompt distillation" as one of the techniques in your outline.

      The idea is to reduce your fixed prompt token size such as "system prompt" by removing semantic words completely and other methods. I saw a whopping 60% decrease in my fixed prompt token budget.

      Pls note, my scale is small but the technique works nonetheless.

      Edit: so, have you tried it? Or if you have tried it, how did it go?

      • troymagennis4 hours ago
        That's a great one. THANKS. I'll do some more research