Rust is always a pain to remove, but you can create a paste that pulls it out efficently. All you need is a little bit of lemon juice and some salt.
Recently, I had to clear off a rust stain in a tub after an ill-advised bleach session. So, I made a paste by dumping some lemon juice into the tub and then adding enough salt to thicken it. Then, I left it for about four hours. When I returned, I wiped the rust away with a rag. This method should work with everything from tools to clothes to bicycles. If it sounds like too much, try skipping the salt on the first try.