Idea: tap Accept still answers normally. Drag it slowly and options fan out around it (Speaker up, and to the right Record or whatever else makes sense). Release on one and the call is answered with it already on.
Only Apple can ship this, since iOS locks the call screen. The video is a mock of how it would work.
And for the love of god, I hate when the call audio jumps to my headset or my Mac instead of the iPhone. Then I have to tap Speaker and pick the right output. One more step every time.