When you've hit you're desired final gravity you're right in saying that there is no need to let it ferment any further, however it does depend how 'young' your beer is as there may still be undesired flavour compounds present such as acetaldehyde or diacetyl and so the beer would benefit from extra time in the fermenter which is why a minimum of 2 weeks is usually advocated.
An additional argument would be to let more of the yeast drop out before bottling. The trub will also becomes more compact over time which has its benefits.