Research suggests that certain characteristics of survey items may impact participants' responses. In this study we investigated the impact of several of these characteristics: vague wording, question-versus-statement phrasing, and full-versus-partial labeling of response options. We manipulated survey items per these characteristics and randomly administered the manipulated and nonmanipulated items to two groups of respondents. We examined differences in responses via multiple groups confirmatory factor analysis (MGCFA) and found differences in factor means, slopes (indicating differing discrimination), and/or intercepts (indicating differing response extremity) for all conditions. Implications for survey construction are discussed.