I think it depends on the type of oak object you are using too: chips will have the fastest effect, followed by oak cubes, staves, then barrels (but even the size of the barrel has a different effect). Also, the amount of toast and previous use will effect the flavor too.
So, the best advice...