Thing that burned me was forgetting about page cache and THP, two runs of the same binary ten minutes apart differed by 15% until I dropped caches and set khugepaged off. Also worth reporting variance across reboots, not just across runs.

Biggest thing for me was pinning CPU governor to performance and disabling turbo entirely. Slower numbers but they stop drifting as the box heats up. Also isolcpus for the load generator, otherwise you're benchmarking your own noise.