Io_uring Without Readahead(frn.sh)
58 pointsby porridgeraisin3 hours ago2 comments
  • amluto17 minutes ago
    I’m curious why the choice is between syscalls and, specifically, io_uring with O_DIRECT. AFAIK Turso is like SQLite and supports multiple processes accessing the same database, and I would expect buffering to be a huge win in some workloads. What’s wrong with io_uring without direct? There’s also the middle ground of RWF_DONTCACHE.
  • marginalia_nuan hour ago
    If you're doing contiguous readahead in userspace, why not just use preadv? It'll limit you to doing readahead up until the next resident page, but at least in my experiments in Marginalia's index, preadv beats io_uring in all cases you can use a single preadv call to do the full read.
    • vlovich123an hour ago
      Not sure what you mean. Nothing about preadv lets you indicate you only want to read what's already in the page cache. And io_uring and preadv aren't orthogonal - you can give io_uring a preadv op to do the scattered read instead of issuing separate read OPs although I'm not 100% sure how much of a win that is in practice.

      Also, I think you misunderstood the blog as it's describing application read-ahead which is what you have to do when using O_DIRECT.

      • marginalia_nu41 minutes ago
        So I have a buffer pool with O_DIRECT reads.

        I implement read-ahead in the application by (optionally) preadv:ing a single read into multiple destination buffers in the pool, leaving them unpinned, since as long as you aren't up against the bandwidth limit of the drive, a larger read is generally as fast as multiple smaller one on modern hardware.

        I've tried doing this with io_uring as well, but found just eating the preadv syscall cost was faster.

    • Asmod4n13 minutes ago
      I found out that using mmap and just telling uring to read form there to beat anything else.
      • 0c3ca839 minutes ago
        Why would you use uring to read from an mmap? Couldn't you just memcpy?
        • jeffbee7 minutes ago
          Maybe they meant using IORING_OP_MADVISE with MADV_WILLNEED to bring ranges into memory?
          • marginalia_nu6 minutes ago
            Though at that point you can just use FADV_WILLNEED.
            • jeffbee4 minutes ago
              I'm just steelmanning here ¯\_(ツ)_/¯
      • marginalia_nu10 minutes ago
        Depends a lot on the memory pressure. If you can be fairly certain the data is (or will be) resident in memory, mmap is basically unbeatable. If you can't (because the data is larger than RAM or there's other stuff competing for RAM), mmap can have gnarly system-wide performance implications[1].

        [1] Mandatory mmap=poop-emoji link: https://db.cs.cmu.edu/mmap-cidr2022/