Self-Hosting a Firefox Sync Server(blog.diego.dev)
322 pointsby shantara9 days ago22 comments
  • 3np8 days ago
    Note that this setup still relies on Mozilla infrastructure for several surrounding services for auth, email, account metadata, etc.

    https://github.com/dan-r/syncstorage-rs-docker/blob/main/app...

    It's a good start though and as long as you're OK migrating your account you can plug those in later. Fully self-hosting all the components is just as doable but a little bit more involved. Compared to when I last set it up they seem to have been improving on docs and devex.

    https://github.com/mozilla/fxa

    https://mozilla.github.io/ecosystem-platform/tutorials/devel...

    https://mozilla-services.readthedocs.io/en/latest/howtos/run...

    • eloisius8 days ago
      It looks like you used to be able to configure Firefox iOS to use a different FxA/Sync server, but I tried to find the setting and it's gone.

      > For Firefox for iOS version 9.0 or later:

      > - Go to Settings.

      > - Tap on the Version number 5 times.

      > - Tap on “Advance Account Settings”

      > - Enter your content-server URL

      > - Toggle “Use Custom Account Service” to on.

      https://mozilla-services.readthedocs.io/en/latest/howtos/run...

      • layoric8 days ago
        Is the different names options under “Advanced Sync Settings” the same thing just with different name? “Use Custom FxA Content Server”?
    • user39393828 days ago
      Maybe eventually the whole stack can make its way into a docker container with a shell script for config.
      • have-a-break8 days ago
        Maybe it's just me but having dedicated employee's to manage security is a prerequisite for me; atleast I could point my finger at someone if things happen.
        • dvdkon8 days ago
          I personally don't see the value in that. I'd rather spend time fixing something than getting people to fix it for me, and having someone just to point fingers at them after a failure feels cowardly. But maybe I'm just not a managerial type.
          • bayindirh8 days ago
            That's a time vs. outsourcing trade-off. When everything is working, it's invisible and great, but if something breaks, and if you're in the middle of the something, you either have to postpone it, or stop everything and fix the problem to be able to continue your life unaffected.

            I'm into self-hosting, and host some of my services myself, but get some services too, because I have limited time and don't want to stop drop and roll when something breaks.

        • andra_nl8 days ago
          I completely understand the feeling, but when's the last time pointing fingers actually got you anywhere? :)
        • treve8 days ago
          Of course it's not just you. That seems somewhat obvious. Also plenty of people that do like tinkering on their homelab or self host. It's definitely the more interesting choice!
        • DecoySalamander8 days ago
          Sounds like a recipe for getting people with knowledge of security best practices and people with knowledge of the systems being developed, but not both at the same time.
      • 3np6 days ago
        It's enough different components to not be a good fit for an all-in-one image at all. That road lies madness.

        docker-compose is a much better fit. That already exists. What we need is more people actually deploying and engaging (;

    • zx80808 days ago
      > this setup still relies on Mozilla infrastructure for several surrounding services for auth, email,

      Why does email rely on mozilla infra?

      • wtallis8 days ago
        Mozilla's email-related service is https://relay.firefox.com/ which obviously needs to rely on quite a bit of Mozilla infrastructure in its default incarnation, and is probably a fair bit harder to self-host than Firefox Sync.
        • zx80808 days ago
          Is Mozilla Relay (email) really needed for Sync?

          Never used Sync, so maybe I don't understand something. But does not sound safe to put all eggs into this busket at once.

          • 3np6 days ago
            Relay is unrelated to that conversation.
        • 3np6 days ago
          That is also unrelated. Firefox Sync itself requires Mozilla Accounts to function at all, which requires email for verification and password reset. To properly self-host Firefox Sync you need those parts too, which includes an SMTP server (or substitute).
      • 3np6 days ago
        The other way around. Server-side email verification for accounts and password-reset is an integrated part of the flow and also needs to be hosted and wired up, unless you want to dig into the code and patch those parts out.
    • nonesuchuser8 days ago
      Just cut out the middleman and use Google directly.
  • gudzpoz8 days ago
    Garuda Linux has been hosting their own Firefox Sync server for a while now for their FireDragon browser [1]:

       defaultPref("identity.sync.tokenserver.uri", "https://ffsync.garudalinux.org/token/1.0/sync/1.5"); // --> FIREDRAGON CONFIG
    (They seem to be running the older non-Rust sync server though. [2])

    [1] https://firedragon.garudalinux.org/

    [2] https://gitlab.com/garuda-linux/infra-nix/-/blob/main/docker...

    • spinningD208 days ago
      A little off topic, but I had to refresh my memory as to what Garuda was. As a long time user of arch in various "forms" (initially manjaro until I grew frustrated with all the extra/different things, then antergos because it was mostly just plain arch with a nice installer/sane default packages, then endeavouros because it was the closest thing to what antergos provided me), I'm not sure how I missed garuda. Some of their utilities look convenient.

      Thanks for the mention! I'll have to try them again on my next machine.

  • kyz28 days ago
    This reminded me to post my own experience:

    https://kyzer.me.uk/syncserver/

    It can be done, but by golly is it a slog. The previous version (abandoned and hopelessly out of date, written in Python 2) was SOOO much better.

    It doesn't require Docker though, nor does it require Python. It does require MySQL for now, but there's like one volunteer guy working on supporting sqlite, like the old version supported out of the box. Meanwhile, the main contributors are LARPing being a corporation and writing justifications for collecting metrics and introducing new metrics frameworks, rather than make it any easier to self-host.

  • travelthrowaway8 days ago
    Take a look/follow https://github.com/canvas-ai/canvas-browser-extensions

    Its meant to sync your browser tabs (chrome/firefox) to a self-hosted virtual "directory" tree powered by lmdb and roaring bitmap indexes. In retrospect, we should have used rxdb or pouchdb from day one (keeping the bitmaps) but eventually we get there.

    Workflow

    - Create a new context path based on the task you are working on(universe://travel/2024/03/barcelona for example)

    - Depending on your configuration, your browser stores your current tabs and closes them, you then start your airbnb journey adding new tabs to the context path

    - Your significant other may even open the same context and work on it with you(as in, add additional ones, if auto-sync is enabled close yours etc)

    Server component is under a rewrite atm, my amateurism at play! You can do stupid things with a 2 user user-base (like a full api + backend + module rewrite started simultaneously, breaking existing functionality beyond repair) but with the new tools available these days (cursor, claude 3.7), the most important limiting factor for an idea becomes .. well, time!

  • charles_f8 days ago
    I did that for a while, including hosting the auth myself. Then I realized Mozilla is probably one of the few companies I trust with my data, even though it's probably misplaced trust. The sync server is relatively straightforward to setup, but auth is quite more involved. I don't think it's worth it, compared to using the public hosted service
    • chungy8 days ago
      Mozilla just burned out the last remaining bits of good will with the "terms of use" fiasco. At least there's LibreWolf to keep a privacy-respecting fork.
    • dwedge8 days ago
      Mozilla rebranding themselves as an advertising company and getting the majority of their funds (and assumedly direction) from Google should have you at least questioning that trust.
    • charles_f8 days ago
      Gonna admit, that aged poorly, and very quickly.
  • sandreas8 days ago
    Thank you, this is the only thing what slowed me down switching to Librewolf.

    Seems that MySQL and Spanner are the only supported DB-Engines by now, but there are efforts to support sqlite and Postgres[1]. I really hope these get merged soon, would make it much easier to backup your whole sync storage contained in one single sqlite file. However, a multiuser instance would probably be better stored via postgres due to better parallel update support.

    EDIT: After trying this out: How exactly do I force Firefox (Librewol) using the self-hosted URL? Is there an about:config setting?

    1: https://github.com/mozilla-services/syncstorage-rs/issues/49...

  • mrbigbob8 days ago
    Something similar to Firefox Sync is Floccus. If you only need bookmark and tabs sync it looks straightforward in setting up

    https://floccus.org/

  • zymhan8 days ago
    What's the privacy benefit, given that Firefox Sync is E2E encrypted?

    https://support.mozilla.org/en-US/kb/sync

    • drdaeman8 days ago
      To best of my awareness, the login page (including JS code) is still served over the Internet, and encryption keys are derived solely from the password. That’s not a good design for something as sensitive as one’s full browser data.
      • 1oooqooq8 days ago
        send a pr.

        but don't know how adding a key owned by moz would make this more trustworthy

        • AshamedCaptain8 days ago
          It used to be that you could have separate login and encryption passwords. It used to be that you could self-host it as easily as pointing to a separate WebDAV server (trivial to spin up if you have _any_ web server at all).

          Not anymore.

        • eadmund7 days ago
          As AshamedCaptain notes, Mozilla actively don’t want an actually-secure password storage protocol. They chose to weaken their security.
    • vedmed8 days ago
      [flagged]
  • therein8 days ago
    Would be nice if librewolf and WaterFox changed their flow to make it clear that Mozilla's Sync Server is being used, and make it opt in with something like:

    To sync please select one of the options below:

    [x] Custom Sync Server: ___________

    [ ] Mozilla Sync

    • yborg8 days ago
      LibreWolf is opt in and if you click the checkbox named "Firefox Sync" thinking you're getting some kind of sync that isn't Firefox Sync I don't think there's any UI change that can help you.
      • jraph8 days ago
        Your parent comment's idea is quite sensible, no need to be contemptuous like this. Firefox branding is a mess, being both a browser and a set of services. It's reasonable to mix things up or to be confused.
    • drdaeman8 days ago
      Sync is nothing but an awfully overengineered blob storage (I have implemented an alternative Accounts+Sync server when 1.5 came out, https://gitlab.com/drdaeman/firesync/ - long abandoned and the code is awful proof-of-concept inly, but the point here is to prove that I have a slightly educated opinion on the matter).

      IMHO, It would be best (for technical users) if Firefox forks would rather replace Sync with something more standard like S3 or WebDAV. If my memory serves me right, there’s absolutely nothing in Sync that those protocols cannot do equally well (and even better because they’re actual mostly-interoperable standards).

      Heck, it would be nice if Mozilla would do the same thing, but I’m afraid NIH is way too strong with them. I’m very happy if that gives us Rust, but Sync is simply garbage that shouldn’t have been ever made.

      • layer88 days ago
        With only object storage, handling sync conflicts [0] becomes more complicated. I can understand why they don’t do that.

        [0] https://mozilla-services.readthedocs.io/en/latest/storage/ap...

        • AshamedCaptain8 days ago
          Yet stuff like CalDAV which is used by millions transparently handles syncing on top of WebDAV (arguably not just dumb storage , but not much better than it). Not only that, but Firefox Sync _itself_ used to be WebDAV.
      • 63stack8 days ago
        Is there a specification somewhere for the sync server that you followed, or did you figure out the endpoints/logic by looking at mozilla's sync server?

        I was thinking about taking a crack at this as well, but I can't find a spec, and I want to leave looking at the python/rust sync server source code as a last resort.

    • AshamedCaptain8 days ago
      For the record, for a lot of time Firefox Sync used to be exactly like that.

      https://blog.mozilla.org/services/2011/03/22/get-syncing-wit...

      Not only one of the first questions would be whether you want to use a Custom Sync Server, but this "Custom Sync Server" could be something as trivial as WebDAV server (if you have any kind of public-facing HTTP server, it almost certainly already supports this, or does so with minimal extra dependencies) . Then it would also ask you for the E2EE key, separately from the login key.

  • gsliepen8 days ago
    Congrats on getting this self-hosted. However, the solution looks more complicated than necessary. You shouldn't have to set up a database server and Docker just to sync a little bit of data. Ideally, there would be a sync server implementation that uses SQLite and can just run stand-alone.
    • jeroenhd8 days ago
      In theory you could set up a simple sync server that implements the necessary endpoints and nothing more, nothing less. Guides like these run the official, Mozilla-maintained sync server, which is obviously based on serving millions of users rather than being a minimal implementation for someone in a home lab.

      One could fork https://github.com/mozilla-services/syncstorage-rs, take out the plumbing, and reimplement it all without a secondary database and the Google Cloud base architecture, I'm sure, but like with so many things, you'd first need to gather a group of people with enough interest and spare time to do the development.

    • layer88 days ago
      Certainly, but at least it’s available at all for self-hosting, which isn’t the case for many browsers.
  • Mostlygeek8 days ago
    There’s also this: https://github.com/mozilla-services/go-syncstorage

    While it was never fully deployed into production it was tested with a few hundred thousand users. It is stable at least.

  • nairboon8 days ago
    Given the recent Firefox fiasco. What are the viable options for a free software browser? Can anyone point to the most promising alternatives (apart from the other corporate browser engines)?
    • layer88 days ago
      In terms of open source there are really only Chromium- and Firefox-derived browsers. With Chromium browsers, you’re still subject to Google’s whims in the long term, such as removal of V3 extension support. (I.e. a conceivable fork with V3 compatibility will inevitably become too difficult to keep up to date with the mainline.) If Mozilla dies, Firefox and derivatives will in all likelihood wither away as well. IMO there is no alternative to supporting Mozilla, and also keeping them accountable and criticizing them where criticism is due. They are still roughly the good guys, even if sometimes misguided.

      Regarding the “fiasco”, there was an update that you may have missed which is at least a partial fix: https://blog.mozilla.org/en/products/firefox/update-on-terms...

    • seba_dos18 days ago
      Right now, there are none. The "fiasco" is still about the least bad option available.

      Some could become viable in a few years, but I wouldn't count on it to "just happen". You'll still need to actively try to get your specific use cases well-supported for a while, and if not enough people do that these projects will never get there.

    • henriquemaia8 days ago
      Ladybird browser: https://ladybird.org/
      • layer88 days ago
        “Promising” maybe, but it’s pre-alpha and as such won’t be an actual alternative for some time to come.
    • IYasha8 days ago
      Librewolf, Falkon, somthing-chrome-derived (I forgot)
  • IYasha8 days ago
    Closed article after "docker".

    Oh god... It's still seems faster to hand-merge bookmarks from all of my browsers and devices together.

    • AshamedCaptain8 days ago
      Why is this downvoted?

      Why does this bookmark syncing server need to be anything other than a dumb file storage? It used to be a simple WebDAV server -- Mozilla unilaterally changed it to this monstrosity of a service with little real gain for the user to be seen.

      • pieter_mj8 days ago
        It's not about bookmarks only. Also history. A syncserver is the only way to access firefox mobile history, as its api (browser.history) is severely restricted or unavailable. So short of rooting your device to directly access places.sqlite, there's no alternative.
        • AshamedCaptain8 days ago
          > A syncserver is the only way to access firefox mobile history, as its api (browser.history) is severely restricted or unavailable. So short of rooting your device to directly access places.sqlite, there's no alternative.

          What I'm pointing is that in previous versions, Firefox ITSELF used to allow setting an arbitrary WebDAV server as "sync server", and STILL sync history, tabs, whatever the heck you can think of. https://arstechnica.com/information-technology/2010/02/mozil... . It is much later that the option to use a plain WebDAV server disappeared from Firefox Sync, and also later that the option to set your own E2EE key also disappeared (and forced to use the login pw instead, making it easier for Mozilla to capture). The "restriction" you are pointing out is entirely Mozilla-made. All in the name of user convenience of course.

          Again, why is anything other than a dumb storage server required, considering than previous versions used to be able to do it with just a dumb storage server ?

  • replete8 days ago
    FYI You can use Floccus and a WebDAV server to sync bookmarks across browsers, set and forget
  • grimgrin8 days ago
    I'll jump to the wolfside but I'm going to absolutely miss "send tab to device", where I'm often driving this from mobile firefox, sending to one of a few other devices for Next time I'm sitting at a desk

    There are alternative techniques but maybe none with so few clicks, where the outcome is something literally in your face. A new tab opening on a computer elsewhere. It's that "I'm almost forced to see this" that I like

    • sshine7 days ago
      I'll miss this, too. But it's an acceptable loss.

      Quick thoughts on how to get it back: Orion Browser by Kagi has an iOS app, and they support Firefox plugins. So in spite of how much effort it takes to get a mobile app to do what you want, it might be possible for LibreWolf and Orion for iOS to sync.

  • pava08 days ago
    Asking a related question: is there anyway to sync history between browsers, fully self-hosted?
  • promiseofbeans8 days ago
    Great timing, with Mozilla's new privacy policy problems the other day
    • EasyMark8 days ago
      why would those matter since your sync data is encrypted and they can't see it anyway?
      • anilakar8 days ago
        The issue is not Mozilla getting access to your data, but losing access to Sync itself.
      • 1oooqooq8 days ago
        because they can update the client tomorrow to not use e2e...

        meta did it. google did it. so not without precedent.

      • cship28 days ago
        Why have your data sitting on a timebomb when quantum risks are just around the corner. Self host if you care about your data!
        • jraph8 days ago
          How do you protect your self host against quantum risks?
          • cship27 days ago
            Self host one have control over the post quantum custom encryption you want to use, not at wimp of some corporate over Lord who know they are really doing it for user benefit or just a user acquisition trick.
            • jraph6 days ago
              > the post quantum custom encryption

              Do you have a custom encryption that is post quantum?

      • jraph8 days ago
        If you self host, you don't need to wonder whether you need to respect their acceptable use policy.
  • nileshtrivedi8 days ago
    It should be possible to use syncthing to achieve sync across multiple devices, without having a third-party server - even if untrusted, right?
    • kevincox8 days ago
      No, any concurrent modifications would cause problems.

      The main difficulty with sync is conflict resolution and none of Syncthing's solutions work well for the things that Firefox syncs.

  • linwangg8 days ago
    [dead]
    • 8 days ago
      undefined
  • 8 days ago
    undefined
  • sciens38 days ago
    I’m no longer using Firefox. They have stated clearly that they sell your data.
    • jbaber8 days ago
      What's your alternative?
      • sciens3_8 days ago
        Brave, for now.
        • JetSpiegel8 days ago
          So the alternative to Firefox is some cryptocurrency software?
    • redder236 days ago
      [dead]
  • maxlin8 days ago
    I've use Firefox as my tertiary browser option, for some specific uses like for example I have it always have an VPN plugin on. But I'm starting to feel anything in its vicinity should be avoided like hostile malware, regardless if an attempt to remove bad parts of it is made.

    A corporation can do a lot of bad stuff with their software and terms, and most have probably normalized that in their mind. But after now Firefox for example implicitly lawfully claims all the google docs you edit as their IP, that should be a bridge too far for anyone using it even slightly professionally. God help if you also use it as a person that Mozilla considers you their political blood enemy (which isn't exactly hard nowadays). Not exactly the Firefox 1.0 release I still remember. Not even as benign as its worst enemy it was built to be against at the time. Truly Dark Knight -esque.

    • layer88 days ago
      Mozilla is changing the wording: https://blog.mozilla.org/en/products/firefox/update-on-terms...

      There is still cause for criticism, but if you think Mozilla is worse than Google (Chrome) or Microsoft (Edge), then that’s hard to take seriously.

      • maxlin8 days ago
        Hard to? Missed the part about Mozilla basically being an advertising company nowadays, and "a global band of activists"?

        Something Chrome-adjacent, like Chromium or Brave doesn't have the stain Firefox and everything close to it now has. That is not a position that is hard to maintain.

        Firefox has come so far from what it once was. I did once cheer for it, but it simply has become its own worst enemy. Almost nothing is left, crazy to think they're the only competing browser engine independent from the chromelikes.