Just added a website for the same benchmark, easier read: https://aideploybench.com

Hi HN. I built a benchmark for myself that measures not just coding an app, but rather the end-to-end loop: Build, deploy, troubleshoot. Then an independent verifier (currently opus 5.5) checks the live URL and performs basic tests.

The benchmark runs five web apps: (1) static website for say a company page or the like (2) the usual CRUD with auth for say SaaS product (3) an API to be consumed by other services (4) a dashboard with data to mimic internal company dashboards and the like (5) a realtime chat application with support for web sockets.

It currently compares four platforms but can be easily extended to more, feel free to open GitHub issues or send me a PR.

I was specially interested in comparing human intervention (Human Time). As in, how much time is spent for a human to deploy to particular platforms the first time?

The benchmark tracks build/deploy time, tokens, agent cost, security, compliance review of the deployment (secrets handling, least privilege, terms fit, attack surface, etc.). Methodology and reports (including screenshots) are in the repo.

Repo: https://github.com/cloudishai/ai-deploy-bench

Full disclosure: I'm working on Cloudish. Happy to answer questions about the methodology or results.

Edit: Removed table due to formatting issues.