I've added vanilla beans to the fermenter, after the primary has settled down (about 7-10 days in). You can leave your beer on the yeast cake for up to a month to allow the vanilla to settle in.
I would not suggest using anymore than 3 whole beans, maximum! I made a spiced vanilla porter once with 3 beans, cinnamon, and nutmeg before. It took about a year until it was tasty. The more you throw in, the longer it takes for the flavors to blend together.
Bottled: Dubbel Agent, Black Ops American Stout
Keg #1: Nothing
Keg #2: Nothing
Keg #3: Nothing
Keg #4: Nothing
Fermenting: Imperial American Brown
Next On Deck: Imperial Stout?
Brewed in 2012: 65 gallons
Brewed in 2013: 50 gallons (so far)