156 pointsby levlaz9 hours ago10 comments
  • dsp_person7 hours ago
    The libfuse github has some good examples for C/C++ in [0] of increasing complexity:

    - passthrough.c mirrors existing filesystem, "Its performance is terrible."

    - passthrough_fh.c "performance is not quite as bad."

    - passthrough_ll.c implemented with low level api and "the least bad among the three"

    - passthrough_hp.cc high performance version written in C++

    Some interesting fuse projects in my notes: [1] splitting large files into segments; [2] show ZFS incremental snapshots as files; [3] transparent filesystem compression; [4] and [5] options for mounting archives as filesytems.

    - [0] https://github.com/libfuse/libfuse/tree/master/example

    - [1] https://github.com/seiferma/splitviewfuse

    - [2] https://github.com/UNFmontreal/zfs_fuse_snapshot

    - [3] https://github.com/FS-make-simple/fusecompress

    - [4] https://github.com/google/fuse-archive

    - [5] https://github.com/mxmlnkn/ratarmount

    • craftkiller3 hours ago
      While [2] might be a good code example, this functionality is already built into ZFS. At the mountpoint of every dataset is a hidden ".zfs" folder that doesn't show up, even on a `ls -A`. You just have to believe its there and cd into it. Under that is a "snapshot" folder, and inside that is a folder for each snapshot of that dataset. Those folders contain the files in the snapshot.

      So for example, /etc/hosts from my snapshot zrepl_20241011_010143_000 would be at /.zfs/snapshot/zrepl_20241011_010143_000/etc/hosts

      If you don't like the magic hidden nature of it, you can even configure it to behave like a normal folder with `zfs set snapdir=visible <dataset>`

  • aargh_aargh7 hours ago
    Just wanted to throw out there that although I'm a fan of FUSE, it's not the only option. I've had fun implementing a virtual filesystem via the 9p protocol not too long ago.

    IIRC, I used py9p and the python experience was much nicer than fuse-python. You can mount a 9p service via FUSE (9pfuse) if you want. I just used the kernel v9fs client. If you're just looking to pass a filesystem through the network, I think I used the diod 9p server.

    Overall, it's a nice little ecosystem to explore.

    • packetlost5 hours ago
      9p is such a great little protocol. diod[0] has a good amount of documentation on the protocol itself, but it's pretty simple.

      I have some notes here [1], but it's mostly just linking to primary sources. FUSE is great, but 9P is more general and has high quality implementations all over the place, even in Windows!

      One thing I'm not so sure about is the performance properties of 9p. I've seen some places indicate it's rather slow, but nothing definitive. Does anyone have any benchmarks or info on that?

      [0]: https://github.com/chaos/diod/blob/master/protocol.md [1]: https://athenaeum.wiki/Zettelkasten/9p

      • mananaysiempre4 hours ago
        > 9P [...] has high quality implementation[...] in Windows

        Do you know if it’s possible to mount one’s own 9P servers under Windows? I seem to remember a comment from a Microsoft employee on GitHub something-or-other that said that capability is private to WSL2, but I can’t find it right now.

        • packetlost2 hours ago
          I'm not sure if you can mount a 9P filesystem from windows normally, I'll try. I'm not seeing any resources online about it either.
    • hathawsh5 hours ago
      It looks like py9p was last released in 2013 and it's still marked as "beta". Cool project though!
  • kapnap11 minutes ago
    Off topic but whenever I see a blog with some 90s/2000s vibes, I always go to their first page of posts. Never disappoints to sneak a peak into that time capsule - including gwolf.org!
  • iamjackg8 hours ago
    I wish I had known about this a month ago, when I had to go through the exact same process!

    In a desperate attempt to find a less frustrating way to interact with Jira, I had the silly idea of starting a jira-as-filesystem project that uses our internal issue categorization to build a tree: directories represent issues, with files representing issue fields and subdirectories for linked issues. I ended up choosing fuse-python.

    I haven't worked on it in a minute, but I was already bumping into issues (pun not intended) with the abstraction: using just the issue ID as directory name makes automation easier, but it makes it hard for humans to browse the tree, since a `ls` would just show you a bunch of inscrutable IDs. I ended up adding a parallel `<issue-type>-with-summary` directory type where the slugified summary is appended to each issue ID.

    • maicro7 hours ago
      Hmm, I'm not saying it's a good idea, but what about a daemon that keeps a symlinked version of the entire jira environment up to date? So you have one jira-as-filesystem that's the raw files, but then for human consumption/interaction, you have a tree of symlinks, including multiple links to the same file wherever it's relevant. Might be adding more layers than needed, based on my lack of understanding, but might technically solve the (current/stated) abstraction issue.
      • iamjackg7 hours ago
        That's sort of what I'm doing behind the scenes, because I keep one global list of downloaded issues (they're lazily loaded when you access them) and then the folders are really only "views" into the downloaded issues. Representing identical ones across trees as symlinks is a fantastic idea though, I can't believe I didn't think of that! Thanks for the inspiration.
      • xg156 hours ago
        Would you even need a daemon for that? That sounds as if the FS could just generate the symlinks on-the-fly in the same way that it generates the folders.

        (Unless symlinks are somehow special - but at least both /dev and /proc also provide symlinks and to my knowledge they don't have any actual storage behind them, so it should be possible, I think)

      • inferiorhuman7 hours ago
        May as well just implement that in the FUSE driver.
      • paulddraper6 hours ago
        State syncing is always harder than state reading
    • jrms5 hours ago
      Why not just 1234-human-sense? You have both type of info there and it's easy to parse too I think.
      • iamjackg2 hours ago
        Yeah, sorry, I think I was a bit confusing: that's exactly what I'm doing. For example, an Epic folder is laid out like this:

            EPIC-123
            ├─── user-stories
            │    └─── STORY-234
            └─── user-stories-with-summary
                 └─── STORY-234-add-support-for-feature-a
    • renewiltord7 hours ago
      Referencing the same two ways is normal in Unix fs. On a modern Linux you will see disks referenced by block device and UUID. I think your approach is good and consistent with expectations.

      Though I, personally, would not use it as JIRA is complicated enough for me.

      • iamjackg2 hours ago
        Yeah, the /dev/disk/by-uuid paradigm was actually the inspiration for adding the second folder!
      • xuQH9W3HP85 hours ago
        [flagged]
  • mcoliver3 hours ago
    So many fuse mount options out there with varying tradeoffs, performance, and features (s3fs, goofys, seaweed, minio, Google drive, etc..). JuiceFS is pretty interesting for doing things like mounting an object store and accessing it via posix with all the metadata you would expect on a traditional filesystem. https://juicefs.com
  • memset7 hours ago
    Nice!

    Adjacent question: lately I’ve been seeing people implement NFS base filesystems since that is a more widely supported protocol. I think rclone does this for Mac. Is there a guide, or even a comparison, for this approach?

    • mbirth2 hours ago
      Fun fact: Now that recent macOS versions require you to disable security features to install macFUSE, there’s the awesome fuse-t. It works as a drop-in replacement, doesn’t need the kext and will open up an NFS server in the background and mount that using macOS features. Performance is pretty good, too.
    • formerly_proven7 hours ago
  • peterldowns2 hours ago
    If you're interested in seeing what a finished product looks like, check out azuline/rosé — a music manager with a virtual filesystem. Really good codebase with a lot of comments and explanations and types and tests, which should make it easy to learn from.

    https://github.com/azuline/rose

  • alkh8 hours ago
    I've recently discovered sshfs and learned about needing to have FUSE as a dependency for OS X, which spiked by interest. The code looks very clean and easy to understand, so thanks for that! Is there any guide/course you would recommend for the introduction to FUSE? It looks like all you have is to provide implementations to certains functions your filesystem will use but it's hard without knowing the details(ex. I wouldn't know I had to implement readdir without your code, and so on)
    • pkaye7 hours ago
      I've used sshfs in the past and I know the original authors stopped maintaining it though others took over. I did find the network error handling wasn't the greatest. Like it would unmount the fuse mount due to network error and I'd be writing files to the local mount directory silently until space filled up. Perhaps its a Linux specific issue or I've used the wrong options though.
      • alkh7 hours ago
        To be honest, I knew about the speed limitations of sshfs already, so I typically use rsync to work with large files. This way, I wouldn't write the data locally even if the connection fails. I've checked the github repo and it looks like there are a number of issues related to network timeout that hasn't been addressed for a long time[1]. However, I mostly used it on OS X, so my experience might be different from yours Thanks for the info as well, I was under the impression sshfs was under active development (: [1]https://github.com/libfuse/sshfs/issues/77
      • beeboobaa33 hours ago
        > I'd be writing files to the local mount directory silently until space filled up

        that's why you `chattr +i` the mountpoint

  • rnd04 hours ago
    An user-space filesystem running in an interpreted language? Is that as bad as I think it is?
  • gmartens108 hours ago
    [flagged]
    • adhamsalama8 hours ago
      You say they embark on a language whose future is uncertain, while being the most popular language in the world, and advise them to learn Scheme instead? Lmao.
      • pxc6 hours ago
        If students' first and best-known programming language leaving college is relatively small or niche in a way that makes it unlikely to find it practical to unconditionally use it everywhere, that might actually be good for the software engineering discipline as a whole.

        One healthy thing a move like that might encourage is turning which language to use into a more deliberate choice based on the virtues of the language itself with respect to a task. With Python it's kinda too easy to just always reach for the familiar whether it's actually a good fit or not.

      • jdgasd8 hours ago
        You are citing the Tiobe index for the "most popular language"? The Tiobe index just counts the searches that include "Python". Which means that beginners who search for "leftpad python" or people who search for "python sucks" are included.

        Google is reducing its Python investment. Python is just the most marketed language.

        Students learning Scheme learn good habits, and Scheme will not go away.

        • exe348 hours ago
          can confirm, I learnt scheme and it allows me to do things in python that real software engineers find impressive.