And use a C-clamp to press the piston back into the caliper next time but apparently you know this now.
Also, I use a one-man brake bleeder kit I bought from Autozone for around $7. It looks just like the one someone else posted a link to on Amazon. It was cheap but has a lifetime warranty and makes changing dirty brake fluid a fairly simple job and no air gets into the system. If you're doing all 4 wheels start from the one furthest away from the master cylinder and work your way closer.