At this point I'd say no, it's fine.
Thanks Unionrdr. After reading this I still prepared to transfer and maybe just sour the brew but when I tasted and it seemed all good and I think maybe the spots were just the yeast continuing after I'd transferred to secondary as it had dropped a little bit more since 2 weeks ago. I've bottled it all and it's tasting pretty good.