Conventional wisdom dictates we must face our fears to conquer them. This idea is embodied in exposure-based treatments for anxiety disorders, where the intent of exposure is to reverse a history of avoidant behavior that is thought to fuel a patient's irrational fears. We tested in humans the relationship between fear and avoidance by combining Pavlovian differential fear conditioning with a novel task for quantifying spontaneous passive avoidant behavior. During self-guided navigation in virtual reality following de novo fear conditioning, we observed participants keeping their distance from the feared object. At the individual level, passive avoidant behavior was highly associated with maladaptive fear expression (fear-potentiated startle) during late extinction training, indicating that extinction learning was impaired following a brief episode of avoidance. Avoidant behavior, however, was not related to initial acquired fear, raising doubt about a straightforward link between physiological fear and behavioral avoidance. We conclude that a deeper understanding of what motivates avoidance may offer a target for early intervention, before fears transition from the rational to the irrational.