I have used both a secondary and a bottling bucket for years. I have also left the batch in the primary the entire time and then bottled directly from it using carb drops. I can't tell much difference between results of the two methods other than "primary only" causes a bit more bottle sludge in the last few bottles and it cuts down on the stuff you gotta wash. The air exposure may be noticeable if you did an" exbeeriment" to test the same exact beer with and w/o, but with the little bit of additional fermentation in the primed bottles and possibly the O2 absorbing caps, if used, I don't think it's a big deal. I do limit the exposure as much as possible and try and get transfers and bottling done with as little agitation as I can, but there's only so much that can be done without going all keg and closed transfers.
A secondary allows the beer to clear a bit and it gives you a less yeasty place to dry hop and add stuff. If you use a carboy as secondary, watching it clear is part of the fun IMO. I'm no master brewer, but I like the best beer I can create. If it made a noticeable difference in the final product, I would be firmly in the "no secondary" camp. I just haven't seen it.
I do find myself leaning toward brewing simple recipes that I can leave in the primary - but not because of off-flavors or poor shelf-life - mostly because it's just easier.
I think idea of a "secondary" might have come from lagers or other styles that have a much longer fermentation period. Move the beer off the trub after a few days to let it finish fermenting without all the gunk in the bottom. Allows for bottling or kegging without the chance of stirring up all that gunk by accident and making the beer cloudy again.
Another way to look at it is maybe a secondary is just really a brite tank or bottling bucket in some sense.
Before I started using fermenters with a spigot, I always had issues with some of the trub getting disturbed when racking/siphoning. The secondary/bottling bucket removes most of the trub issues prior to bottling.