It's not a matter of opinion, this is a system designed with a written blueprint and manual for how to operate it, and that manual says Speakers have an inherent subpoena enforcement power, and that it's necessary to use that if a tyrant tries to take over.
You can't just remove an entire section of a machine and expect it to function properly. Not even if you jazz up the remaining parts with color-coordinated rhinestones.
You can call it mean, call it whatever you want, but future historians will have to write that the powers that were available were never invoked. Because that's the truth.
We had a republic but we may not be able to keep it because certain people thought that redesigning the machine on the fly, stressing parts never meant to function that way, was a better idea than just using it as designed.
The next election was never designed to be a mechanism to punish cheating in the last election. That would be a pretty poorly designed system easily undermined by even the least intelligent would-be tyrant.
The system is designed so that if you find out someone took power illegitimately, you remove them from power and bar them from ever being eligible to hold it again. See how much more sense that makes than "Don't tell anyone about the crimes, hope they choose right anyway"?