I am sceptical of these things however since they are a well worn path that so far has taken off only to a limited extent. In reality, you get bored very quickly in these environments, and it's impossible to implement impressive visual fidelity in worlds that users build themselves. Horizon Worlds is probably the marquee example of that. Am curious what the differentiating angle is here to previous efforts?
You're probably optimizing for XR, but 99% of people will experience your app for the first time on a different platform, so it's worth fixing the desktop render path.