Yes. It should be srping loaded to remain closed. Air should not leak out of it unless you pull on the vale ring to deliberately release air. Check the air is not espacing at any of the joints around the relief cutoff valve. You can spray water or a mild soapy solution on all the air connections and if one is leaking it will cause the solution to bubble.

Seal all threads with teflon thread sealing tape.