I've seen/fixed similar issue in someone else's Windows 10 notebook few days ago. It seems to have been caused by wifi having the same ssid for 2.4 GHz and 5 GHz and not-that-great signal so OS was for some reason "constantly" switching between those two. After forcing it to use just 5 GHz it went from 1 Mbps to 100 Mbps. Reading this sooner would have prevented my headache.
Cross-compiling ? Aren't M1 and Pi both ARM processors ? I've been building apps for my M1 to run on Pi without any issue. Some optimization options maybe ?
Cross compiling is really defined as different platforms, not limiting it to instruction sets. Also, the M1 is an ARM, but the instruction sets aren't exactly the same.
If the app leverages the page size in some way you can get into very subtle crashes - app is built on M1 (with 16k page size), page size is stored in a constant in the binary because it's a macro, then you trt to access the entire range of a page in a pi (4k pages) - and things break.
Cross-compilling from MacOS to Linux. By default OCaml does dynamic linking of all C dependencies. Even with static linking there is a dependency on libc (or the MacOS equivalent) as far as I remember. I had some success so far with Rust, Nim and Go using a musl toolchain[1], but no luck for OCaml. At the moment I'm just using a docker container which mirrors the Debian distribution running on the Pi.
Of course wired network is almost always faster, more reliable, and easier (no selecting ssid, no password, just plug in and it works), but it's not always an option; in an apartment you may not be able to do wired, or placement of the machine can render it impractical.
If you don't need the pinouts, it's likely cheaper to get a used thin client or a N100 based machine.
You'd get similar power draw, plus a case and a PSU.
I suspect that either you care about performance and the pi loses badly, or you care about price and the pi 4 loses to... probably other pis, actually. Or used x86 off eBay. Or you care about absolute power consumption and again other pis win.