With the rise of the impact assessment revolution, governments and public opinion have started to ask researchers to give evidence of their impact outside the traditional audiences, i.e. students and researchers. There is a mismatch between the request to demonstrate the impact and the current methodologies for impact assessment. This mismatch is particularly worrisome for the research in Social Sciences and Humanities. This paper gives a contribution by examining systematically a key element of impact, i.e. the social groups that are directly or indirectly affected by the results of research. We use a Text mining approach applied to the Research Excellence Framework (REF) collection of 6,637 impact case studies in order to identify social groups mentioned by researchers. Differently from previous studies, we employ a lexicon of user groups that includes 76,857 entries, which saturates the semantic field, permits the identification of all users and opens the way to normalization. We then develop three new metrics measuring Frequency, Diversity and Specificity of user expressions. We find that Social Sciences and Humanities exhibit a distinctive structure with respect to frequency and specificity of users.