Ideally, lagers are fermented at 45-55 degrees or so, depending on yeast strain. By fermenting it 3-4 days at room temperature, most of the fermentation would be over before it was moved to 50 degrees and then it takes a long time for the beer to drop about 20 degrees. Then bringing it back it to room temperature and dropping it again would stress it even more.
Some lager yeast strains do tend to produce sulfur, but I think the issue here is the stressed yeast. Fermenting it too warm, changing the temperature, etc, all will create off-flavors and odors.
When I make a lager, I make a huge starter (you need more yeast than for ales), pitch the at 48 degrees into 50 degree wort, and keep it at 50 degrees for about 10 days (or until the beer is 75% finished). Then the temperature can be raised for a diacetyl rest if I'm doing one.
After the diacetyl rest, the beer is racked and the lagering process begun. Lagering is at 34 degrees for about 8 weeks.
Your lager may improve during lagering. If the SG readings are stable, and the beer is finished fermenting, I'd go ahead and try lagering it.