Some of that flavor will mellow a bit with age, but once it is in there, it is in there. That yeast will always throw some banana notes, it's just part of what it does. The best way to keep the notes somewhat subdued is to keep the ferment temps in the low to mid 60's. SOme people just flat out don't like the banana notes in any beer, and in that case you may want to look at some of the similiar yeasts that throw less of that flavor profile.
If you cold crash the beer with a fining agent such as gelatin you may pull some of that flavor out, and also you will pull a bunch of the yeast out of suspension. That style is common to have some yeast in suspension, but in your case if you find it unpalatable it may be the best solution for you.