3 pointsby TylerJewell3 hours ago1 comment
  • TylerJewell3 hours ago
    We created a self-improving, spec-driven delivery harness to test whether frontier models can port complete systems.
    • TylerJewell3 hours ago
      There were a few interesting findings:

      1. We were able to execute complete, unattended rewrites of existing systems that pass the original's unit and integration tests.

      2. Low effort models were more efficient than high effort models.

      3. The delivery harness and its structure had a bigger impact on consistency than model capability.

      4. We saw a LOC or performance improvement on 57 of 65 ports.