It's because you're using XFCE, which might be one of the last desktops to adapt to Wayland. Mint has similar problems, they have also dragged their feet. So the tech stack you have selected has pretty much maximum problems.
You need to embrace Wayland and Pipewire, not avoid them. Linux has never been about being a luddite, rather the exact opposite.