Use of threats of physical and property harm to obtain something against the will of the victim is typically referred to as extortion. Blackmail typically refers to threats of exposing damaging information.

> If you don't do X for me, I will harm you physically. There are plenty of X's that will not be illegal, but harmful to reputation.

You're describing extortion. Blackmail is typically the use of harmful information to get something one wants against the other's will. Regarding the thing being obtained being harmful to the victim's reputation, it sounds like you have it a bit backwards.

Moreover, blackmail statutes often specifically cover situations in which the person being blackmailed actually did commit the crime, the threatened disclosure/allegation of which is being used to obtain something against their will. Hence, blackmail is certainly not limited to innocent victims but instead includes blameworthy ones.