Plus you arrive faster, meaning that your payload on the far end needs even more fuel to slow down.

TFA says that the next generation of rockets might be able to do that, but that sounds unlikely.