It gives a very strong hop aroma - not all beers want it.
The hop aroma is definitely best when young so dry hopped beers don't always (not a fixed rule) age as well as other beers.
Some hops can give a grassy taste from dry hopping.
You lose some beer (soaked up by the hops).
They tend to clog bottling wands if dumped straight in.
You did only ask for the negatives, but I want to balance this by saying that dry hopping is IMO probably THE best and easist way for new brewers to get a phenomenal tasting beer. Hop aroma can cover a lot of brewing sins.....
I just chuck them in. Too lazy to wash hop socks. I don't know what a muslin viagra is?