2 pointsby Siddaartha day ago1 comment
  • Siddaartha day ago
    Well, part of its brain, that is. And right now, it's controlling a simulated Mercedes-Benz on my computer without any training.

    Yes. Zero backpropagation. Zero reinforcement learning.

    I downloaded the fly's connectome, filtered it down to the motor pathways, and built an artificial retina so it can see the world through a car’s dashcam (just 900 pixels total, black and white!).

    The result? It uses just 1,555 FLOPs per frame. Despite the circuit having zero concept of a “car” or a “road” to begin with, it seems to (very primitively) keep a lane, turn, and dodge stuff it’s never explicitly seen before.

    Code + writeup + demos at the link (addresses the limitations too, of course!): siddaarth.com/fly-by-wire

    Really curious to hear your thoughts and learn what you would have done differently.

    • andsoitisa day ago
      This is cool. Are you considering doing a real world experiment? Obviously not with a real car, but maybe something like a small object that the fly neurons can steer through a maze or something?
      • You could use a real car too! (As long as low speed / open lot), I just did an experiment like this last weekend, a similar stack (build off of Openpilot/Comma 4) could work with the fly model, will email you some details if you're curious
        • Siddaartha day ago
          Wild that you did this just last weekend. I think with the proper manual overrides this could be a great experiment. After the RC car, I guess :)
          • True RC car is definitely the next logical step :) Yeah as long as you set a low enough speed limit, do it in an empty parking lot, have a human always ready to brake, I think it should be relatively safe haha
      • Siddaartha day ago
        I was thinking exactly this! Either an RC car, or more interestingly, a drone. I’ve only tested it on simpler obstacles and environments so far, but who knows what more it can do?