Show HN: We built open OpenRouter that turns usage into a better model(github.com)
144 pointsby SilenN8 hours ago11 comments
  • Areibman7 hours ago
    Could you say more about how caching works? One major advantage of sticking with a single model is saving money on cached input tokens. I'd imagine if you swap between a bunch of models, you may improve performance but cost would would balloon out of control
    • SilenN7 hours ago
      The trick is to rarely switch, or switch at task boundaries. Often the conclusion of routing is actually "this one model is actually at the pareto front for this task, just use it always".
      • cameronh905 hours ago
        But then it's better to just not have a gateway switch models at all.

        Just have the harness able to choose which model its sub-agents use, then tell it how to split up tasks and which models to use when doing so.

        • SilenN4 hours ago
          That is another way to do. Or we can automatically figure out which models the subagents should be using for you. And update them as new models come out and the work your subagents do changes. More than one way to skin a cat.
    • purplecats7 hours ago
      and caching is related to performance too ofc
  • akshay_akula4 hours ago
    Open source and no markup is the right default for a gateway. The caching question above is the one I would want answered before swapping models though.
    • SilenN4 hours ago
      Ans: we rarely switch, often times it's just a "switch to using this model for your agent"
  • ceroxylon4 hours ago
    >The gateway adds under 1 ms for BYOK requests

    Amazing! Really brilliant idea, thank you for sharing this project. There is so much ground to cover in the LLM gateway / routing / reporting world, and this is a great start. The Tinker implementation is my favorite part, fine tuning is much better than a sea of context files.

    • kfallah154 hours ago
      Thanks! We are going to add continual RL via Tinker soon too
  • sangwookan hour ago
    What online signal recalibrates simulated rankings against actual task success? Also do you have a plan to support semantic caching at the router level?
    • kfallah15an hour ago
      For the online signal, we use a LLM judge with a rubric calibrated offline by the user via TUI. UX of the calibration is a major focus area. Semantic caching is interesting, open to supporting it but not currently planned.
  • swthbht2 hours ago
    Very cool. Does your gateway decide effort levels as well? Or just models?
    • SilenN2 hours ago
      Yep! One interesting example is often Opus 5 on low reasoning ~= Opus 5 on high reasoning.
  • forgetme2020an hour ago
    what's the business model here. How does experiential labs make money
    • kakugawaan hour ago
      They make money on enterprise plans: https://www.experientiallabs.ai/pricing#enterprise

      Look at the Intelligence features in the Enterprise plan:

      * Per-prompt model optimization

      * Caching

      * A model you own, trained on your traffic

      • kfallah15an hour ago
        yep, it will be through enterprise licenses and our own hosted platform built on the repo
  • 0xbadcafebee4 hours ago
    You started it a week ago? I look forward to checking back in 3 weeks when you've exited for $1B
  • cheema335 hours ago
    I have not tried it yet. Is it similar to LiteLLM? If so, what sets it apart?
    • kfallah155 hours ago
      Router and model optimization from traffic is the main differentiator
      • SilenN4 hours ago
        Also a hosted marketplace, not just BYOK
  • 23david6 hours ago
    Super interesting and congrats on the release. Curious if you initially had this in Python and then rewrote in Rust?
    • SilenN6 hours ago
      Yep! If you look at the commit history that's exactly what happened.
  • ashermania7 hours ago
    Finally an open source tool doing this!
  • jing099284 hours ago
    [dead]