Our productivity went up when our PM was absent. The impact was significant enough to be recorded and discussed at the following retro. And it's happened more than once.

We work on software systems. The PM larps as a software professional. When the PM is absent, our standups are rich with discussion about the state of the software, and what the next step is. When the PM is present, he is an information sink, not an information source. He asks stupid questions and needs things dumbed down. The developer's goal is to make sure the software works for the customer, now and forever. The PMs goal is to report to his boss, who is even less in touch with what's going.

There is no glamour in 'make the system fulfil its primary purpose', so the PM will always pull engineering focus away from that.