They don't. It's your eyes re-calibrating to something brighter showing up. The brightness of the SDR content didn't actually change, just like it doesn't when someone turns on the lights in the room even though it ends up appearing darker.

The perceptual issue of simultaneous contrast ( https://en.wikipedia.org/wiki/Contrast_effect ) is a much bigger problem with HDR content in general than people tend to consider. There's far too much HDR content out there that just slams the brightness because it can, essentially resulting in just being an SDR video that had the brightness cranked. Nearly all mobile HDR video falls into this category in particular.

This is not helped at all by the fact that prior to the introduction of gainmaps in images, which many slammed as "a hack" or "not true HDR", the mapping between HDR and SDR values was undefined. BT2048 has attempted to retroactively define that PQ & HLG at "203 nits" maps to "graphics white" (aka, SDR white), but almost nothing is authored to this expectation. The huge advancement of gainmaps, beyond per-pixel local tonemapping, was that the mapping between SDR & HDR was rigidly defined by the spec. So you could actually author content that could be displayed next to SDR content without destroying people's eyes.

It works great on my MacBook M3 display. And also on my HDR compatible external monitor.

The HDR content is literally is brighter than white on my monitors.

I wonder what is different about your setup?