Anxiety is a common affective state, characterized by the subjectively unpleasant feelings of dread over an anticipated event. Anxiety is suspected to have important negative consequences on cognition, decision-making, and learning. Yet, despite a recent surge in studies investigating the specific effects of anxiety on reinforcement-learning, no coherent picture has emerged. Here, we investigated the effects of incidental anxiety on instrumental reinforcement-learning, while addressing several issues and defaults identified in a focused literature review. We used a rich experimental design, featuring both a learning and a transfer phase, and a manipulation of outcomes valence (gains vs losses). In two variants (N = 2 × 50) of this experimental paradigm, incidental anxiety was induced with an established threat-of-shock paradigm. Model-free results show that incidental anxiety effects seem limited to a small, but specific increase in postlearning performance measured by a transfer task. A comprehensive modeling effort revealed that, irrespective of the effects of anxiety, individuals give more weight to positive than negative outcomes, and tend to experience the omission of a loss as a gain (and vice versa). However, in line with results from our targeted literature survey, isolating specific computational effects of anxiety on learning per se proved to be challenging. Overall, our results suggest that learning mechanisms are more complex than traditionally presumed, and raise important concerns about the robustness of the effects of anxiety previously identified in simple reinforcement-learning studies.