Jev Plays Manic Miner(www.atomic14.com)
2 pointsby iamflimflam16 hours ago1 comment
  • verdverm5 hours ago
    Did you watch Jev play pokemon? It showed interesting patterns with that game too. I'm going the llm route and having it play my first RPG. Turn based so latency is acceptable, prefer the smarts for this one.

    I've been pondering if I even bother with map views, or just provide navigable locations as aist like Jev-Pomemon. It'll end up having both options I'm sure, since I'm using the project to explore harness evals.

    • iamflimflam15 hours ago
      Interesting idea - so instead of "which move do you want to make?" it's "which location do you want to go to?".
      • verdverm5 hours ago
        yeah, that was how jev-pokemon worked

        since mine is more of a harness/eval framework, I'm going to experiment with different toolsets and context constructions

        It's definitely easier to skip that part and just use A* for navigation, but I also want to see it explore and learn the map, through more basic moves. Right now I'm not exposing direct button presses to the agent. Similar for menus, give a list of actions (actor->fight->target) and an algo converts that to button presses, so each turn in a battle, it produces an action for each party member.

        • iamflimflam14 hours ago
          The danger is, that you end up passing the decision model something where the choice is trivial. There's so much in the harness that you don't need any intelligence.
          • verdverm4 hours ago
            I have yet to see and credible accounts of these "system one" models being better than alternatives in the range from [task specific classifier ... llm]

            I've seen way more about how brittle and inconsistent Jev is with decisions that should be straight forward, like from pokemon...

            - I'd like to buy a potion!

            - Are you sure?

            - No...

            - I'd like to buy a potion!

            - Are you sure?

            - No...

            - <repeat for 5 minutes>

            it took it waaaay too many tries to buy item

            Then there are the examples like coin flipping or dice rolling, Jev doesn't understand basic probability, picked heads like 84% of the time or something...