194 pointsby kevinlinxca day ago13 comments
  • eitallya day ago
    Is there an opportunity to partner with (or sell to) one of the big digital sheet music vendors (like Musescore or Music Notes, etc)? I've never come upon a compelling personal use case for smart glasses, but as a pianist this could be it. I would HAPPILY purchase both glasses and a subscription from one of the big music vendors if this worked seamlessly and I could do things like embed a metronome or link it to my DAW so I could control things like tempo, rewind, even key transposition.
    • fennecfoxy12 hours ago
      I feel like this would be sold as more of an app for a smart glasses platform than an individual product.

      >I've never come upon a compelling personal use case for smart glasses

      There are tonnes, it's just the technology isn't there yet; glasses are too bulky and heavy, the fov sucks, the resolution sucks, light transmittance sucks.

      But the use cases are incredibly plentiful; stuff like this (music sheets, documentation, web browsing), getting realtime directions with a blue line or directional hints when walking around an unfamiliar place, overlays/information at tourist sites, home automation/controlling devices.

      I remember an old anime or some show where it's a world where a digital world is overlaid the real world where AIs and devices from the digital layer can be interacted with in a similar way...what was it hmmm.

    • kevinlinxca day ago
      This would make the most sense, since MuseScore is notoriously litigious about usage and redistribution of their library/MusicXMLs, so a collaboration would be necessary to get a usable music catalog for smart glasses
    • adrianha day ago
      Just a quick plug: check out Soundslice. It's interactive sheet music with a ton of learning tools built in, including easy navigation, looping, tempo changing and transposition.

      We've also got a scanning feature that does OCR for sheet music, to get music into our system. Plus there's a full-featured notation editor. A good overview is at https://www.soundslice.com/features/

  • pedalpetea day ago
    This is such a great use case. I stand in front of my monitor with my guitar and have to scroll the sheet music. So that means stop playing. I often wander away from my computer and then come back if I forget how a section goes.

    I'm using tabs not notes, but I'm assuming/hoping your solution will adapt quite easily.

    I wonder if you could use a microphone to listen for the notes in order to get auto-scrolling. Because you know the general timing, you're not searching through the entire song (likely) but honing down on the exact point that person is at. An inobtrusive metronome might be nice to.

    Congats! One of the best projects I've seen in a long time, and particularly such a good use case for the early stage of this hardware.

    • kevinlinxc15 hours ago
      Thanks for the kind words! As I briefly mention in my video, my teammate actually had guitar tabs going, with lyrics chords and even web scraping/search. I think a bit of refinement and better hardware and we'll get what you're looking for
    • a day ago
      undefined
  • mdp2021a day ago
    Nice project, but I do not get the competitiveness of part of its implementation details:

    -- the project uses "Even Realities G1" AR glasses (640x200, 25°FoV, 1bit green), while the "Epson Moverio" AR glasses can have overwhelmingly superior specs (1920x1080, 34°FoV, full RGB) for possibly an even lower price;

    -- software wise, it «uses AugmentOS's SDK to communicate with Mentra servers which talk to the mobile app which talks to your ... glasses» - while an Epson Moverio system would just directly use the glasses as a display for an Android device...

    Both gaps between the available and the employed make very little sense.

    • alex1115alex20 hours ago
      You can absolutely find more feature-packed glasses. Epson, XReal, Viture, Vuzix Blade, TCL RayNeo X2/X3, INMO Air 1-3, etc. The issue is all these weigh north of 70g (too heavy for comfortable daily wear), and/or require a wire that connects to an external compute box (not socially acceptable for daily wear). Glasses like the Even Realities, otoh, are lightweight, look normal, and are wireless, so can be worn all-day as you would normal prescription glasses.
    • turtlebits21 hours ago
      Makes perfect sense- usability. Look at both sets of glasses.

      One looks normal enough to wear all the time.

  • Aidevah16 hours ago
    Great job! For converting music to readable images, the latex of music typesetting is lilypond, which has the ability to create legible music at any size by scaling the notational glyphs accordingly[1]. This sounds like what you were trying to achieve achieve with opencv.

    With that being said, although lilypond is very intelligent about all sorts of typesetting minutiae, but it's probably difficult to wrangle it to run on smart glasses.

    [1] https://lilypond.org/doc/v2.24/Documentation/essay/engraving...

    • kevinlinxc15 hours ago
      I tried using lilypond actually but ran into an error that no one seemingly knew how to solve. Can try digging through my history if you're interested
  • KyleBrandta day ago
    A full orchestra on stage playing with no music stands sure would be make for a nice sight (assuming the glasses looked like regular old glasses -- (or maybe blues brothers shades)).
    • kevinlinxca day ago
      Agreed! These glasses do look very normal - only tell is that at a certain angle you can see the green of the screen, and the part near the ear is a bit bigger (but easy to conceal with hair)
  • florena day ago
    Three seconds to send a bitmap? And I thought the Brilliant Monocle/Frame was slow! In the video it looks like you don't get more than a bar or two on-screen at a time... wouldn't any reasonably fast piece outpace the rate at which you can get the next bar on the device?
    • kevinlinxca day ago
      Yeah, it's a big deal for sure, I was bugging Mentra all hackathon to try and lower it, and also reached out to Even for suggestions (which Mentra is implementing). Regardless, I made it work and next gen hardware, firmware and software are all definitely going to be better for bitmaps
      • florena day ago
        If they're using the same Nordic BLE chips everybody else is, there's just gonna be a cap on how quickly you can move stuff, I think.

        I've found the display capabilities of the current gen smartglasses pretty disappointing. Yes they're less obtrusive, but the resolution is pitiful. I've found the Vufine a lot more useful, if more ridiculous looking.

        • alex1115alexa day ago
          Mentra here.

          The Nordic MCU they use isn't actually the limiting factor, rather it's the glasses' firmware. For bitmaps from third party apps (like AugmentOS), they enforce 194 byte chunk sizes and do not support RLE. Their first-party app does not have these limitations. We're stuck with this problem for the G1, but we're working with hardware partners to make sure future glasses don't have these issues.

        • kevinlinxca day ago
          If I were designing around this limit, I would put enough memory to be able to store a nice buffer of bitmaps in either direction and then do sends that don't change what's currently displayed. I feel like that memory probably exists, I just don't have access to the firmware sadly
  • analog31a day ago
    Very cool idea and demo.

    The ability to adapt paper music would be useful. In some genres -- I play big-band jazz -- virtually no material is available in printed form, or it's in the composer's preferred format, which is typically PDF.

  • zharknadoa day ago
    Congrats! Great video write up also!
  • paul7986a day ago
    Great and cool to see this, as well see some fellow smart glass enthusiasts on Hacker News.

    I've been an avid enthusiast and promoter of Meta Ray Bans since Oct 2023. They are very handy and I think for anyone person who wear sunglasses or glasses and uses their phone to take pics or vids then they make a ton of sense (both things you can do with them without needing your phone.. also ask them for the time). Though Im not sure even the HN population is much about them.

    Albeit I love them I do not think as you see the media and i guess Zuckerberg saying they are the next computing platform that to be true. You can not take selfies with smart glasses unless they offer a pop out tiny drone in the glasses to take pics of u lol. Thus, I think they will be complementary to our personal pocket smart and or upcoming pocket AI devices, which will able to take the best selfies of you ever (ur AI friend see on the lock screen directs you to the best light to get the best selfies).

  • MarcelOlsz21 hours ago
    This would be incredibly useful for sight reading.
  • a day ago
    undefined
  • Instead of 30 pedals, give the conductor a butt-on.
  • bad_haircut72a day ago
    [flagged]
    • a day ago
      undefined