FWIW, I use a secondary for my fruit beers. I usually need the extra space that is freed up by taking the beer off of the trub, and I don't think you have to be super concerned about oxidation because the yeast is still active, especially after you add the mango.
Secondary may not be necessary...