Show HN: DeskSlice – controlling a VS Code agent from my phone(github.com)
3 pointsby frudas24a day ago1 comment
  • Sean-Dera day ago
    Would you mind explaining the complexity issues around WebRTC more? Why did you need to do renegotiation? What RTP stuff hit you?

    thanks

    • frudas24a day ago
      The WebRTC complexity came from our pipeline being ffmpeg → H.264 RTP over UDP → pion/webrtc TrackLocalStaticRTP (instead of a “normal” WebRTC source). Any time we changed monitor/crop or restarted the capture, the RTP stream effectively reset (SSRC/seq/timestamps and sometimes SPS/PPS cadence), and mobile browsers can stall the decoder and just stay black. We added “restart/renegotiation” because recreating the PeerConnection is the most reliable way to recover from those discontinuities.

      What we still need to debug to make WebRTC solid:

      Capture-side: full ffmpeg stderr logs + exact args when it goes black. RTP ingest: log SSRC/PT/seq gaps and verify SPS/PPS are regularly re-sent (e.g., with every keyframe). WebRTC states: log signaling/ICE/connection state transitions to catch races and “remote description not set” timing. Confirm whether the black screen is a capture issue vs a decode/packetization issue (capture works via MJPEG, so likely the latter).

      • Sean-Dera day ago
        Sorry you hit these issues :(

        Instead of restart/renegotiation can you re-timestamp the packets? The example swap-tracks[0] shows a good way to do that. The renegotiation (especially multiple times with no real changes) is gonna be a PITA :)

        Also you should share in https://pion.ly/discord other people would love to see this. Super cool project.

        [0] https://github.com/pion/webrtc/blob/master/examples/swap-tra...

        • frudas24a day ago
          Thanks, that’s a great suggestion :)

          You’re right that re-timestamping is the proper way to avoid renegotiation, and the swap-tracks example is exactly the direction to take. In our case, monitor/crop changes usually required restarting ffmpeg, which often reset more than just timestamps (SSRC, sequence continuity, SPS/PPS timing), so renegotiation became the brute-force fallback.

          That said, I’m definitely going to try your recommendation and experiment with re-timestamping / track swapping to see how far we can get without renegotiation, especially on mobile browsers. Thanks as well for the Discord link, I’ll share the project there. Appreciate the concrete pointers.