That has been my concern as well. The script I wrote tries to bucket entries into categories, including "Backward Incompatible Change" so those are easier to spot. Since it is automated I am trading some accuracy for time saved, which seemed like the only practical choice for me, since I had to backfill a lot of history, but it’s been surprisingly decent so far.

I am also planning to add some PR templates so contributors include the context up front, which should make any release note generation more accurate.

Are you using any tooling to help with changelog curration? I know towncrier is all about fragments, so contributors must had write a brief summary of their contribution, which would be more in-line with your preference.

Not related really, as you can (and should) publish BREAKING.md separately. Release notes should inform more about new stuff than the update process. Usually PRs have details, so release notes can be easily automated. Migrations, on the other hand…

I hear what you are saying, there is a risk that auto-generated release notes end up as PR-title soup. I put a lot of effort in my script to mitigate against that.

If you are willing and interested enough to take a quick look, here is what my script generated for our 2025 changelog (no hand-curation yet, this is the raw output):

https://raw.githubusercontent.com/confident-ai/deepeval/refs...

I am curious: does this still seem too noisy in your opinion, or is it getting closer? And what would you want to see for breaking changes/migrations to make it actually useful?

I now have 2024 & 2025 generated; to fully hand-curate two years of history just wasn’t practical, so I’m trying to get the "80% draft" automatically and then curate over time.

Yeah, that matches what I have seen: if the upstream metadata isn’t reliable, automation can amplify the mess.

I tried to avoid relying solely on contributors to accurately label or tag things correctly. The script is tag-driven only for release boundaries (version tags), while categorization is derived from PR title & body with optional GitHub metadata. The script is idempotent and preserves edits/omissions so you can correct the few bad ones post-generation.

If you are curious, I am happy to share my script and would be genuinely interested whether it reduces the manual cleanup for your workflow. Also, if you run it with `--ai --github` and a PR body is sparse, it fetches a truncated PR diff and uses that as extra context for the LLM summary.

I agree with the philosophy of curating release notes for the consumer of the release. When I first started looking for a release notes strategy, I was considering towncrier for that exact reason. You are also right that commit messages are not intended for the consumer of the release, but a dialogue between developers.

Your points are well received and largely why I went PR-based (title/body with optional GitHub metadata) instead of commit-based. A PR title and body tend to be focused on the deliverable, whereas commit messages are narrowly focused on the code change at that moment with developers as the intended audience.

Re: git-cliff, I honestly hadn’t evaluated this one, but it looks solid for commit-driven changelogs. I like the rationale behind conventional commits being parsable and templates enforcing consistency. What constraints pushed you toward git-cliff vs writing release notes by hand, and do you have a config/template you have found works well for surfacing breaking changes?