I always remove bucket spigots and clean both separately, then baggie the spigot. On next use I'll rinse, assemble and then sanitize them together. Could be overkill, who knows, but I relax better doing it that way
If you can do it, I would remove all the parts after every batch and sanitize. The reason being the residue that can get infected over time if not thoroughly cleaned. Especially around the spigot area where there is more room for residue accumulation. Like day trippr said, it'll make you sleep better at night