1 pointby platypiia day ago2 comments
  • barbazooa day ago
    Lots of LLM observability tools out there. Lots of them integrate with model sdks or llm frameworks" and sends traces via OpenTelemetry to providers like Braintrust, that's the one I know. You shouldn't have to build all this yourself if you're willing to spend some money.
    • platypiia day ago
      We're willing to spend money, but I've had the "datadog billing problem" before where it starts reasonable and then grows to a non-trivial percent of saas budget, and then theres a scramble to refactor. Trying to get ahead of that as the LLM logs are MUCH larger that my APM logs.
      • barbazooa day ago
        Then I'd try to integrate via a standard connector, OTel for example. Then the cost of switching is much lower. But yeah not sure myself how this will scale and how expensive or even useful it will be.
  • dewolfe1037aa day ago