You can get TC gaskets made of different materials, each with their own advantages/disadvantages. Silicone is good for its high temperature range, so it makes sense on a kettle. But it is not as good at keeping oxygen out. So longer term seals like fermentors and kegs should look to use alternatives like EPDM.
I did some research/investigation on this for a review I was doing a few months back and it's hard to find good comparison data out there. I was able to find one research paper that was from the commercial brewing world and they measure oxygen uptake of the beer when they used Silicone vs. EPDM gaskets on their large (5"?) TC fittings. They found Silicone allowed in 25x as much oxygen as EPDM. Silicone tubing in fact is used in some medical applications specifically because of its ability to pass oxygen through it.
Now all of this being said, you'll probably say that you always had the TC fitting on your fermentor and you didn't change your gasket when you added the sight glass, so is any of this really relevant? Maybe not directly, but if you are looking at ways oxygen can get into your beer while sitting in your fermentor, you should seek out TC gaskets from a material other than Silicone.
Your sight glass will have some kind of caulk/sealant between the glass and stainless parts. Both glass and stainless themselves are great oxygen barriers, but maybe the sealant they used between these parts is the weak spot for oxygen ingress (a lot of caulks are silicone).
To get around this, you could look for a TC end cap made 100% out of some clear material like acrylic or glass (but I have no idea on the oxygen transmissibility of acrylic, so that's definitely worth some research). I didn't find much of these on the web, but did find this:
Tri-Clamp Sight Glass End Cap | Sanitary Fittings [Buy Online]
Stout tanks and kettles sells them too:
Buy End Cap Sight Glasses | Stainless Steel Parts & Accessories Store