I have a Victory Kingpin....Victory destroys harley in performance, engine build quality and longevity. My uncle has a 1999 Victory V92C with 106,000 miles on it...the engine has NEVER needed anything but oil and filters! That does not happen with Harley.
You need to determine if you would rather have a cruiser, a standard motorcycle or a crotch rocket.
Cruisers are great for all around riding and awesome for long distance riding but pretty poor choice if you like curvy mountain roads. Standard motorcycles come in a variaty of sizes and packages but generally are versatile and do everything well. Take the Yamaha FZ6/FZ1/FZ8 for example...streetfighter looks, sport bike performance, enough comfort for a 1,000 mile road trip without putting you in a doctors office. A crotch rocker (Super Sport bike) is about one thing; PERFORMANCE. This is a great weekend ride, track bike or canyon/mountain carver but a poor choice for commute or long distance touring.
Then there is the sport touring bike. Examples Yamaha FJR1300. These bikes are sporty yet comfortable for the long haul.
It also comes down to price range and insurance costs. I also have a Yamaha FZ6S because it was a great bang for the buck and insurance was about 70% less than it would have been on a crotch rocket. Definitely get insurance quotes before you buy!