Show HN: Tine – Drive Wayland Around with Agents(github.com)
4 pointsby tarboreus3 hours ago1 comment
  • aayushkumar1212 hours ago
    Nice approach combining AT-SPI2 + grid fallback.

    Have you run into issues where the a11y tree is incomplete (e.g. Electron apps)? Wondering how often the grid/OCR path becomes the primary path.

    • tarboreus2 hours ago
      Yes. It works really well with Firefox. Chrome is basically a big blank, could expose the Chrome a11y tree but would have to turn on developer / debug mode or whatever which has downsides. It's really nice for Firefox and GTK apps and it does OCR + grid stuff for everything else.
      • aayushkumar1212 hours ago
        That makes sense. Feels like the fallback path might end up being the default for a lot of real-world apps.

        Have you thought about combining weak a11y signals + OCR to build more stable refs over time, or is that too brittle in practice?

        • tarboreus2 hours ago
          Yeah. Some of the tree is a mess, could try to ocr those bits. It's on the roadmap. :)

          I will say I have some feelings about Wayland and how hard it makes some stuff I do. I'm visually impaired and have a whole stack of tools. But this project has helped me port over 70-80% of those tools and it helps me bridge some gaps on Wayland temporarily so I can get infra set up. It's also great for the many sites that Claude blocks for whatever reason (Reddit, I am a sub mod but the a11y on Reddit is terrible, AmEx, LinkedIn).