They probably mean leaving ssh open to all ips. Take a look at your auth failure logs to see the thousands of daily attempts to compromise your server using default passwords. Most of those are low effort and low risk. Sometimes the bots will try password stuffing. Disabling password auth in sshd config is good practice. Fail2ban also helps block repeated attempts like that.

There’s also the risk of a zero day RCE vulnerability in ssh (though I’ve not seen one in the 20 years I’ve been paying attention )

I tend to not expose ssh to the world and log in with some other method to pass the perimeter (VPN, IP whitelist, tailscale) and the ssh from inside.

ditto to everything here. If you really want to you can also change the port to something random to avoid bot spam. but you shouldn't have SSH accessible directly from the internet anyway.

If you are using only keys, make sure they are managed, tracked, securely stored and backed up. The last thing you want is to have a machine die that has the only private key for your environment.

Best practices usually call for not exposing the SSH endpoints to the public internet. The principal risk is vulnerabilities in the underlying SSH server implementation. Historically, critical flaws that can compromise you are few and far between. However, these days AI is already starting to become adept at reverse engineering.

If you must, you'd typically use a bastion host that's configured just for the purpose of handing inbound SSH connections, and is locked down to a maximal degree. It then routes SSH traffic to your other machines internally.

I'd argue that model is outdated though, and the prevailing preference is putting SSH behind the firewall on internal networks. Think Wireguard, Tailscale, service meshes, and so on.

With AWS, restricting SSH ports via security groups to just your IP is simple and goes a long way.

If your private key has a good passphrase and is suitably encrypted, say with ed25519, then that's probably as good as you can do other than physically going into work and storing everything in your head :-) Politely ask the client to suggest what they consider would be a suitable alternative. I also setup git hooks to prevent accidentally checking in private keys or passwords into git or other version control systems. And if I'm travelling into or from work I also encrypt some stuff just in case I have a problem and the laptop is stolen.

Runs counter to my understanding, I'd ask for clarification and find support material to show your approach is safer.

Treat it as a teaching moment for them

Compared to what?

How else would you do it?