Are you referring to the tan rough stuff that builds up over time at the krausen area?
I really don't use anything other than Oxyclean, warm water, fingers and occasionally a blue scruby on the back side of the sponge.... I have used my buckets for 2 years with no issue and I have no worries / concerns going forward.