VMs don’t typically own their own physical memory, so you randomly page vault. CPUs can can be preempted and system calls and other instructions can fault for arbitrary lengths of time in the hypervisor. It’s very hard to get real time scheduling when so many sources of latency exist and are outside of your control.