18 pointsby dvfjsdhgfva month ago2 comments
  • vlad1719a month ago
    Hi, any experience of using FTS5 with rqlite, i am interested in replacing Elasticsearch to have a single source of truth DB and full text search. Thanks
    • otoolepa month ago
      rqlite creator here.

      Yes I do have practical experience to share, I wrote a blog post on rqlite and FTS5: https://philipotoole.com/building-a-highly-available-search-...

      Will it allow you to reach the same scale in terms of data set size that Elasticsearch supports? Almost certainly no, but it might be enough depending on your use case.

      • vlad1719a month ago
        Hi, thanks, yes i have read it before this is why i am considering it. Will have to try.
  • otoolepa month ago
    rqlite[1] creator here, happy to answer any questions.

    https://rqlite.io

    • sgarlanda month ago
      Have you done any updated benchmarks on maximum supported writes/sec since the talk you gave comparing single-zone, single-region, etc.?

      I recently had a potential use case for this, but it required somewhere around 600 writes/sec at a minimum, and it wasn’t clear what the ceiling was for rqlite without sacrificing durability guarantees.

      Terrific bit of software, BTW!

      • otoolepa month ago
        rqlite creator here. I have performed a fair amount of performance testing, some of which I outlined in a talk to the CMU Database Group a few years ago. Details:

        - https://www.philipotoole.com/2021-rqlite-cmu-tech-talk - see slide 33.

        - There is also a recording that goes with the talk: https://www.youtube.com/watch?v=JLlIAWjvHxM

        You can also read about Performance in the docs at: https://rqlite.io/docs/guides/performance/

        An important thing to note: this testing was done 4+ years ago, on moderately-powerful hardware for the time. With higher-end, more modern hardware you may get even better results.

        • sgarlanda month ago
          Thanks for the reply. Yeah, that’s the talk I was referring to.

          I suppose I could try my own benchmark out tbf. I’m curious to see what it can do on today’s hardware. I would think it’s mostly network-bound for Raft consensus, though the 10x ping time increase you demonstrated without an appreciable drop in writes suggests it’s more complex than that.

          • otoolepa month ago
            Yes, fast networks matter.

            I did introduce Queued Writes[1] since that talk, allowing you to trade off performance versus immediate durability. It may interest you -- network is much less of a factor then, and you should get a 10-100x increase in throughput.

            [1] https://rqlite.io/docs/api/queued-writes/

            • sgarlanda month ago
              Unfortunately for the application I was looking at using rqlite for, the possibility of data loss - however remote - was not an acceptable trade-off.
              • otoolepa month ago
                OK, well, you could try client-side batching too, if you can. That will also improve performance substantially.

                Otherwise, if you try with more modern networks and disks, let me know what you see.