You are correct.
Electricity needs a complete path to operate. Current must be able to go into the circuit and then come back out. The heating elements are connected to two wires, so cutting one of them breaks the circuit and no current can flow, so no hear is produced.
But, as you say, even though no current is flowing, that one line that is going out to the element (the line NOT going through an SSR) is energized. If you touch it, it will shock you and your GFCI breaker will trip.
I don't touch anything unless I flip the kill switch that cuts both lines compeletely. Even if I was using two SSRs, I wouldn't trust them and would flip a kill switch before I touched anything.
I do, however, trust the large mechanical device that connects or disconnects the wires completely.