Fresh and young, they should taste smooth and bready, with nice fruit (banana) and spice. They have a 'richness' to them, but are also very refreshing.
With age they become more tart and lose the richness.
What's doing on? This is my guess only, but the bulk of the flavour of a hefeweizen is from the yeast, coupled with the bready tartness of the wheat. With age, the yeast derived flavours get cleaned up, leaving the tartness. It's just a guess though.
They don't really get 'bad' - I still have some that are 9 months old that are OK - they are definitely drinkable, but they are much better young. I drink these younger than any other beer - they are great (fermented with Wy3068) after about four days in the bottle/keg (naturally carbonated - don't force carbonate them) and slowly decline after that.