Thats why I am confused, because when I do turn them parallel to the gas line, the pressure gauge drops to Zero, but if I turn them to the right the pressure gauge raises to 12psi, where I have it set.
The pressure goes up with them closed because it is not emptying in to anything, the space between the regulator and the check valve is 12psi. Give it time to pressurize and check for leaks while you wait.