For my Porters, I add vanilla beans (at the end of fermentation) to my primary all the time. I have also added Oak, Cocoa Nibs & Dry Hops to the primary as well. As @ThickHead said, keeping them in there for 1-2 weeks on the yeast cake is fine. Although I dont keep my beer on most of these flavorings very long for fear of overdoing it, most of them get about <1 week in the primary before I am transferring to a serving keg.
Although I accidentally oaked the crap outta my last porter with 2oz of chips at 7days. Perhaps this is the beauty of doing a Liqour infused mixture, (small dosing) although I personally never liked the taste of bourbon flavored beers.