The tradeoff is always: initial development time (custom = longer) vs. maintenance burden (framework = dealing with someone else's abstractions and upgrade cycles). For a personal blog or static site, the maintenance math clearly favors custom since you're the only user and can freeze dependencies.
Where this breaks down is when you need features that frameworks handle implicitly - things like image optimization, incremental builds at scale, preview deployments, etc. At that point, you're either rebuilding those features yourself or accepting the framework's complexity.
The real question is whether the complexity in tools like Next.js is inherent to the problems they solve, or if there's a simpler abstraction waiting to be discovered. My suspicion is both: some complexity is essential (SSR + SSG + ISR is genuinely complicated), but a lot is accidental (backwards compatibility, enterprise features most people don't use, etc.).