I have the same stuff but way worse on my aluminum kettle. I figured it was oxidation. I'm definitely not on expert on this stuff by I'd say don't worry about it - if it doesn't come of with cleaners and scrubbing, it probably isn't going to come off when brewing.
Use Bar Keepers Friend powdered stainless steel cleaner. It's frickin magic! My brew pot used to look like that, but I used the Bar Keepers friend and a scrubby and in one minute it looks brand new again!
Unless its burnt, I wouldn't worry too much about it. It should be organic deposits such as sugars from the wort. If it gets much thicker than the fine layer it currently is, then removing it will be necessary.
I use an aluminum kettle and I leave that brown layer in the kettle. It insulates the wort from the bare aluminum. It does not get any thicker in my system.