Assuming hop flavor (humulone, cohumulone, adhumulone, posthumulone, prehumulone, lupulone, colupulone and adlupulone) is a class of flavonoids (see
https://en.wikipedia.org/wiki/Flavonoid), it would require ultrafiltration to remove flavonoids from solution.
Ultrafiltration is typically between 0.005 and 0.1 micron. Microfiltration is typically between 0.1 and 10 micron. Although filtration is nominal and not absolute, it is unlikely that a commercially available 1 micron polypropylene filter will capture flavonoids, and it certainly won't capture sufficient flavonoids to account for a 25% drop in detectable IBUs.
Microfiltration will remove almost all yeast. So if your palate expects some residual yeast character, then I wouldn't be surprised if the absence is detectable. I don't brew beers which require suspended yeast as part of their style, so I wouldn't notice the difference.
Microfiltration is unlikely to remove residual sugars (e.g., maltose) due to their relatively low molecular weights. I don't know the types or sizes of the complex carbohydrates which give beer its body, so I cannot comment about whether any of those components are removed.
While I cannot be certain that filtration doesn't remove some flavor, I have never brewed and filtered a beer which has caused me to question whether flavor had been lost due to the process. It's just not detectable to my palate.
It is true that you or I or maybe even most people may not be able to taste the flavor difference (I've never done a side-by-side, so I can't say).
That said, I'm not sure if it is hops flavinoids being removed or what specific component, just that lab-measured bitterness, as indicated by IBU will be lower.
Also, as you said, maybe yeast character isn't something you're looking for in beer (nor in mine, most of the time), but yeast character will settle out over time anyway as they floc.
But there is more being filtered out than hops flavor (perhaps questionable, as you stated), yeast, and (possibly?) carbohydrates. I don't think anybody would say that filtering only filters out yeast. It takes out proteins (the kind that can cause chill haze, among others) and who-knows what else (I won't claim that I do).
All I can say definitively is that
a) filtering removes more than just yeast
b) the material removed by filtering is not flavorless
I have no science to back it up, but I think it is a reasonable leap of logic to say that if the stuff you remove is not flavorless*, then you have removed some degree of flavor. The degree of flavor that is removed, and the impact or perceptibility may be up for debate (like I said, I've never done a side-by-side, though I welcome anyone with a filtering system to give it a try in the name of science), but the fact that flavor is removed really isn't.
*it's not - taste the residue on the upstream side of your filter next time - it will probably taste primarily of yeast, but there is other stuff in there like the aforementioned proteins)
***edit: In all fairness, it is possible that some of the additional material filtered out could, in fact, be detrimental to beer, especially shelf-stability (various staling compounds like lipids), so it is possible that there are benefits to filtering. However, I think from a shelf stability standpoint, at the homebrew level (it is different on the commercial/industrial level due to available technology and processes) you will do more stability harm than good due to the inevitable oxygen exposure during filtering.
*****Final addendum: But to each his own. If any brewer finds benefit to filtering, more power to him or her. That's the beauty of homebrewing. You can give the same set of ingredients to ten brewers and you will end up with ten different beers brewed by ten entirely different processes. And at the end of the day, most of you, like me, brew to our own palates. I hope my friends like what I brew, but make no mistake, I brew for my own pleasure, and I hope you do too.