A relay attack is accomplished by simply relaying messages between a prover (e.g., an RFID tag) and a verifier (e.g., an RFID reader) with the goal of convincing the verifier of its close physical proximity to the prover. In almost all relay attack scenarios, the verifier essentially communicates with a prover that is outside the verifier's read-range. Relay attacks are notorious since they occur without the knowledge of the reader and/or tag, and has the potential to cause damage to honest parties (here, RFID reader and/or tag). Almost all means to address relay attacks in RFID systems to date are based on the proximity check idea that involves the measurement of message round trip times between tag and reader. With the speed of light at play, such measurements need not necessarily be accurate and could result in the false assumption of relay attack absence. Our review of published literature on approaches that use non-distance-based means to address relay attacks revealed ambient conditions' potential. We critically evaluate ambient conditions and develop a lightweight mutual authentication protocol that is based on magnetometer readings to address relay attacks.