- passthrough.c mirrors existing filesystem, "Its performance is terrible."
- passthrough_fh.c "performance is not quite as bad."
- passthrough_ll.c implemented with low level api and "the least bad among the three"
- passthrough_hp.cc high performance version written in C++
Some interesting fuse projects in my notes: [1] splitting large files into segments; [2] show ZFS incremental snapshots as files; [3] transparent filesystem compression; [4] and [5] options for mounting archives as filesytems.
- [0] https://github.com/libfuse/libfuse/tree/master/example
- [1] https://github.com/seiferma/splitviewfuse
- [2] https://github.com/UNFmontreal/zfs_fuse_snapshot
- [3] https://github.com/FS-make-simple/fusecompress
So for example, /etc/hosts from my snapshot zrepl_20241011_010143_000 would be at /.zfs/snapshot/zrepl_20241011_010143_000/etc/hosts
If you don't like the magic hidden nature of it, you can even configure it to behave like a normal folder with `zfs set snapdir=visible <dataset>`
https://github.com/UNFmontreal/zfs_fuse_snapshot/blob/64b17b...
Appreciate these links. I'm a little rusty on it, if anyone has any Fuse tutorials or guides they found helpful, happy to receive.
You should give ciopfs* a shot, it's worked well enough for my use case (playing games indirectly based on nwjs)
IIRC, I used py9p and the python experience was much nicer than fuse-python. You can mount a 9p service via FUSE (9pfuse) if you want. I just used the kernel v9fs client. If you're just looking to pass a filesystem through the network, I think I used the diod 9p server.
Overall, it's a nice little ecosystem to explore.
I have some notes here [1], but it's mostly just linking to primary sources. FUSE is great, but 9P is more general and has high quality implementations all over the place, even in Windows!
One thing I'm not so sure about is the performance properties of 9p. I've seen some places indicate it's rather slow, but nothing definitive. Does anyone have any benchmarks or info on that?
[0]: https://github.com/chaos/diod/blob/master/protocol.md [1]: https://athenaeum.wiki/Zettelkasten/9p
I also found this[2] from 2016 which points out some performance barriers, and it doesn't sound like an easy fix without some new extensions.
However there's also this[3] post from 2022 about some patches suggesting a 10x improvement in Linux 9p performance.
Would be interesting to see how things are these days.
[1]: https://www.usenix.org/legacy/events/usenix05/tech/freenix/f...
If yes, do you still use it today or why did you stop?
Do you know if it’s possible to mount one’s own 9P servers under Windows? I seem to remember a comment from a Microsoft employee on GitHub something-or-other that said that capability is private to WSL2, but I can’t find it right now.
One of the things I ran in to that made it painful, was that the block sizes for FUSE were really small, it made for a lot of latency and churn of HTTP calls to the back end that ended up needing some fairly complicated caching/pre-reading logic to handle. Kernel read-ahead logic never seemed to kick in (and I didn't do any investigation in to that at the time, other than not finding any particular way to induce it)
2) You can rechunk it in your HTTP client to make fatter HTTP requests: https://go.dev/talks/2013/oscon-dl.slide#51 -- there is even a way in FUSE of pushing the fatter results into the kernel page cache to minimize overhead of coming reads: https://pkg.go.dev/bazil.org/fuse#Conn.NotifyStore
can read the faq in https://virtio-fs.gitlab.io
When I played with 9P, this version of py9p was what worked to share a directory so a NetBSD client could mount it. It worked natively with https://man.netbsd.org/mount_9p.8 without an external client. The important difference from diod was that py9p could speak 9P2000.u, which NetBSD understood, while diod only spoke 9P2000.L.
In a desperate attempt to find a less frustrating way to interact with Jira, I had the silly idea of starting a jira-as-filesystem project that uses our internal issue categorization to build a tree: directories represent issues, with files representing issue fields and subdirectories for linked issues. I ended up choosing fuse-python.
I haven't worked on it in a minute, but I was already bumping into issues (pun not intended) with the abstraction: using just the issue ID as directory name makes automation easier, but it makes it hard for humans to browse the tree, since a `ls` would just show you a bunch of inscrutable IDs. I ended up adding a parallel `<issue-type>-with-summary` directory type where the slugified summary is appended to each issue ID.
(Unless symlinks are somehow special - but at least both /dev and /proc also provide symlinks and to my knowledge they don't have any actual storage behind them, so it should be possible, I think)
Though I, personally, would not use it as JIRA is complicated enough for me.
EPIC-123
├─── user-stories
│ └─── STORY-234
└─── user-stories-with-summary
└─── STORY-234-add-support-for-feature-aAdjacent question: lately I’ve been seeing people implement NFS base filesystems since that is a more widely supported protocol. I think rclone does this for Mac. Is there a guide, or even a comparison, for this approach?
Are there other recommended NFS server codebases?
CUSE is userspace character device driver emulation. It allows you to emulate hardware without compiling a new kernel module. I just used it recently to write a hardware device supporting IOCTLs using Python. However I didn't find any good Python libraries that worked easily and documentation was lacking, but I found it easy enough that I ended up writing it using just the ctypes ffi library. The only part that wasn't immediately intuitive for me, as someone who has never written kernel drivers, is how IOCTL's require the buffers to be resized for each read and write which means the calls come in pairs, but luckily CUSE has a debug mode which shows all bytes in and out. CUSE was originally implemented to create userspace sound devices¹ but has also been use for things like custom TTYs. I used it for creating a virtual SPI device. Hopefully someone finds this useful and this project can get more attention.
It's built on hde llfuse[2]. But that required implementing a bunch of low level APIs that were not really related to dokuwiki. So I created easyfuse[3][4] as a wrapper, which implemented the things that were unrelated the dokuwiki implementation. If you're interested it in building a FUSE system it might be worth looking at.
[1]: https://github.com/JelteF/dokuwikifuse [2]: https://pypi.org/project/llfuse/ [3]: https://pypi.org/project/easyfuse/ [4]: https://github.com/JelteF/easyfuse
Or email, why not expose imap through a file system, so your RAG app (like gpt4all) can just access everything directly ?
that's why you `chattr +i` the mountpoint
It doesn't seem to be a complex interfac
It's quite nice, and really shows that git internally is already half a file system. It's also quite simple, because everything is read-only.
I will link to the video when it comes online, should be later today.
e.g. https://github.com/UNFmontreal/zfs_fuse_snapshot/blob/master...
One healthy thing a move like that might encourage is turning which language to use into a more deliberate choice based on the virtues of the language itself with respect to a task. With Python it's kinda too easy to just always reach for the familiar whether it's actually a good fit or not.
Google is reducing its Python investment. Python is just the most marketed language.
Students learning Scheme learn good habits, and Scheme will not go away.