22 pointsby olucasandrade7 hours ago4 comments
  • KronisLV28 minutes ago
    I haven't seen people often refer to the stack in the site as a "microservices architecture": databases, key-value stores, message queues, object stores and other specialized solutions are typically viewed in a category of their own, orthogonal to what the application architecture is, unless you'd have for example 3 different PostgreSQL instances for the same project.

    From what I've seen, even if you'd have PostgreSQL, Redis, Garage (S3) and RabbitMQ, that might still be considered a monolith if you just have a singular deployment unit for your app (even if scaled to multiple instances). It'd still be a distributed system, sure, much the same way how an app that's connected to a singular DB is (just fewer points of failure), you'd just be separating the concerns across specialized solutions.

    Aside from that, you can get pretty far with just PostgreSQL alone, but sometimes it's nice to have the more specialized solutions because the libraries and APIs that they have are optimized for working within a particular domain - for example, object storage easily letting you handle file uploads, downloads, Range requests, presigned URLs and so on, so you don't have to build anything yourself or think that much about what would be the best way to store a bunch of binary data in a RDBMS.

    On the flip side, if you can somehow avoid the pain of trying to put everything in PostgreSQL, the idea of having a single source of truth for all of the concerns in your app is pretty darn nice! Might as well take a step further and get rid of the front-end/back-end altogether and make the DB serve web requests (or use an embedded DB within your app process), but that gets dangerously close to the mess that was Oracle Forms or that is Oracle ADF.

  • olucasandrade7 hours ago
    You've heard the advice: "Use the right tool for the right job." Sounds reasonable. Even wise. So you followed it. You picked Redis for caching, Elasticsearch for search, Kafka for messaging, MongoDB for documents, Pinecone for vectors, InfluxDB for time-series, and Postgres for... well, the relational stuff. Congratulations. You now have 7 databases to maintain, 7 backup strategies to manage, 7 monitoring dashboards to watch, 7 security audits to run, and 7 monsters that can break at 3 AM.

    The thing nobody talks about, cause it doesn't sell: that advice "the right tool for the right job" is the battle cry of every vendor's marketing department.

    The uncomfortable truth is that PostgreSQL is not "just a relational database." It hasn't been for over a decade. It's a data platform that does what most of these specialized tools do, using the same algorithms, with a single connection string, a single backup strategy, and a single place to debug when everything breaks at 3 AM.

    Not "close enough." Not "good enough at small scale." The same algorithms.

      - Redis
      - Elasticsearch
      - Pinecone
      - Kafka
      - MongoDB
      - InfluxDB
    
    All these buzzwords, unless we're talking about truly unbelievable scale, are achievable with just Postgres.

    Let me show you. Actually, no... simulate it and see for yourself.

  • banashark42 minutes ago
    I'm a big fan of Postgres I've used it for a while and prefer it (partly due to experience) over other options. There is a lot that it _can_ do, especially when you take FDW, extensions, PLs, etc into the equation.

    However, it's not an "always better" situation. These other specialized services excel in ways that Postgres cannot at the moment. _If_ you have a small system, _and_ the featureset you need overlaps with postgres' abilities, _and_ you don't expect to outgrow those two properties, then it could definitely make sense to use Postgres.

    Let's imagine that adding all the pressure to the same system _didn't_ impact other parts of the same system (you wouldn't want a surge in kafka write traffic causing latency on your basic crud routes, right?).

    Redis vs Postgres unlogged tables:

    * Redis has a bunch of algorithms which are battle-tested and ease implementation of common patterns. Need a bloom filter, expiration with ttl that you don't need to write extra code for?

    Kafka/SQS vs postgres queueing: * There are pros and cons here, but you definitely don't want your other Postgres work being impeded by high spikes in traffic. Distributed logs and dedicated message queues are built to handle elastic scalability in ways that are difficult to achieve with Postgres. What if you have certain, super busy tenants whose queueing traffic comes in giant batches at unexpected times? With SQS and their recent fair queues, you not only don't need to worry about the spikes (as long as your writers can write fast enough), but you also don't need to worry about the distribution of in-flight consumer work being imbalanced due to the spikiness of a single tenant

    ElasticSearch: * Postgres FTS can work for a number of simple scenarios, but there's a number of edge-cases where performance deteriorates as well. What happens when large documents are normal? https://old.reddit.com/r/PostgreSQL/comments/1q5ts8u/postgre... shows some metrics on what happens when you get into TOAST territory. FTS also puts a decent load on the db both in indexing as well as searching.

    MongoDB: * Mongo, for all of the deficiencies that it's had over the years (which I hear are mostly resolved these days), can scale writes in a _muuuuch_ more simple way than Postgres.

    The claim is that "unless you're at unbelievable scale" you won't hit it, however I don't think that's true. I've worked on multiple Postgres databases that tapped out due to the amount of work it would take to scale things up further (full tenant sharding by db, multiple shards for some larger tenants, keeping that all going and working at a larger scale).

    Every additional piece of logic you add into Postgres complicates the story of how you fix things once it becomes too much. Once you've got a single write that triggers 10 different table writes, with 80 index updates, and you want to scale those writes up, you might hit scenarios where you need to choose what gets migrated out. Or you get smart with materialized views, but those require full refreshes. So you create your own version of incrementally-maintained-views, which is more writes and work.

    That all being said, I do think Postgres can work for more concerns than it currently is used for, even if I think the recent glazing is a bit much.

    Postgres is great, and having everything in a single transactional workload is extremely convenient and can remove a lot of race conditions and buggy behavior.

  • itmitica3 hours ago
    I'm sold!

    But, what about...