Did the exact same thing with my first batch.

I was prepared to clean my arm with bleach and go in after it, but we fished it out with the spoon after a few minutes.
I doubt it'll hurt your beer, but it'd be nice to have it in there to seal up your lid.