I've used S-04 in several batches with no off flavors, my stout even got up to 74 during it's primary before dropping and it was great. My Irish Blond came out perfectly with S-04. And I also used a starter as I do with everything but my Hefe. The trick is a healthy starter, lots of people use starters with dry yeasts, esp. with bigger beers. I also aerate my wort after cooling to give the yeasts a further chance to grow as needed. They use oxygen in their reproduction phase, what we sometimes call lag phase, although a starter will cut down on lag greatly. My ferments have all been cleaner with a lot of healthy yeasts to do the job from the beginning. After fermentation starts don't introduce oxygen though.
I try not to be in a hurry to drink my beer or have it done in short time. I go at minimum, 2wks prime, 2wks 2nd, 2wks bottle, and condition for 2wks to a month before I expect anything out of my beers. 3 months from grain is even better. Time is your friend.