There shouldn't be since the wine yeast is more overpowering, but if you go to brew a beer in it now you might have the issue of the wine yeast overtaking the beer yeast. Just be sure to clean it out very well with a day or two soaking with whatever cleaner you use. Also from what I believe, for wine you are better off using carboys since those are more oxygen proof than buckets since wines typically have to sit longer in the fermenter than beers.