That's what I do - regular kitchen soap and dish sponge. Don't scrub really hard, especially on the bucket - you don't want to cause any scratches in the plastic - nasties can hide in there and avoid getting sanitized out.
Hot water, soap and elbow grease is the way to go. Rinse really well, then upside down to dry.
On brew day, another good rinse, then spray down with star-san or other sanitizer of choice. I then put the lid loosely on until it's time to rack in. (I do the rinse and sanitize while my wort chiller is working.)