If there's any residual bourbon on the cubes, it could give the yeast that come into contact with it a hard time for a bit, but shouldn't be a problem. The problem with high alcohol and yeast comes when the yeast generally eat themselves into a situation where they've produced so much alcohol that the liquid they are living in becomes toxic. That isn't going to happen to you since bourbon will get diluted real quick. The bigger issue is that the dynamic nature of a primary may drive off some of the flavor you wanted from the oak and bourbon.
Your fermentation should be ok but the oakey and bourbon flavor will probably be less than you would otherwise get by adding them to the secondary.