Yes, you want it subtle, so as not to dominate the wheat.
The question is, how do you handle bitterness? Tea is bitter, so you'd have to figure that into the IBU. Maybe you could just pop open a wheat beer with a known or low IBU (like Blue Moon) and add some already-brewed tea to see how much...