One of the issues with using the term "Belgian" is that it implies a beer type, i.e. "did you have the Sierra Nevada Belgian?" or "I've never had such a good Belgian". "Belgian" is not a type of beer like IPA, Stout or Porter is.
Belgium has the most diverse and idiosynchratic beer culture in the world. The beers can't accurately beer summarized by the all encompassing "Belgian" moniker. There are the monastic beers, regional farmhouse styles, saisons, the famed sour beers (Our Bruin, Flanders Red), Lambic, regional pales and blonds, Abbey beers, etc.
When someone says, "I dont like Belgians", I can get behind that if I know the actual type they refer to. I personally dont like Belgian blonde type ales.
Another big contributor is that people think that "Belgian" beers ferment hot. Yes, some producers of Belgian styles ferment warm, but that is because they have adapted thier yeast over time to give them the desired characteristics under those conditions. I am of the opinion that, at the homebrew level, if you are fermenting Belgian style yeast over 70 deg in the intitial stages of fermentation that you are unlikely to ever like "Belgian" beers.