The hardest part was the cost simulator. Comparing price-per-million-tokens across models is misleading — different models burn different amounts of tokens for the same task. So we normalize token counts to estimate what a swap would actually look like. When we recommend an alternative, we filter out anything that drops more than 10% on any benchmark or can't handle your context window size. Still improving this.
The SDK never sees your prompts or responses — just model name, token counts, and a customer ID. Limitations: simulator recommends from six vendors only, no custom/fine-tuned models, USD only.
Stack is Rails and Postgres. Happy to answer anything.