Objective: It has been observed that toddlers tend to miscategorize several negative emotions (e.g., sadness, anger). On the other hand, when we infer others' emotions, an approach-avoidance tendency toward facial expressions is perceived antecedently before linguistically categorizing them into emotion-word-based categories, such as sadness, anger, and fear. Therefore, children may have categorized facial expressions in an approach-avoidance framework before they were able to classify them in emotion-word-based categories. Method: The present study explored this possibility by asking 3-year-old Japanese children (N = 36) to categorize facial expressions using both an emotion label and action response (i.e., approach-avoidance) in the same procedure. Results: revealed that while toddlers were confused by negative facial expressions using emotion labels, they could distinguish them using action responses like adults. Furthermore, children's categorization of facial expression by emotion label was partially similar to categorization by approach-avoidance framework. Conclusions: These findings suggest that although toddlers have difficulties in distinguishing between several negative facial expressions by emotion-word-category, they can distinguish how to react to these facial expressions just like an adult. Furthermore, toddlers might construct emotion-word-based categories based on approach-avoidance in early childhood. Finally, the developmental trajectories of categorization of facial expression by emotion-word-based category and approach-avoidance response are discussed.