So I looked at some videos using a desolder pump and apparently I was doing it wrong, but it works well for clearing holes and you don't have to be fast. I also have a cheap desolder pump and it still works, although I want to look into buying a better one.
If the hole is just solder (you removed the lead) and you can access the top and the bottom of the board, clamp it so you can get to both sides. Place the tip of the desolder pump over the hole and from the other side of the board touch the hole and the solder will quickly liquify. Hit the vacuum button on the desolder pump and your hole is clear.
Hope it helps.