Letting beer sit on dead yeast for a long time is a good way to make your beer taste like a stack of cold cuts.
If you are kegging I can see the lack of need for a secondary, and you can just blow off the sediment before consuming. But letting a beer sit in secondary for a week will allow things to settle so your bottles will not have large amounts of that sediment in it. Also, I have found that beer tends to hit its peak after some amount of time, so letting things settle in a secondary fits right into the aging process.
Having brewed over 400 batches of beer over the past 20 years or so, I have never had an infection that was caused by a transfer from one carboy to the next. Beer is not something that can be rushed without some sort of consequence. To each his own, but there is a difference between making beer, and making great beer.