This study aims to evaluate the effectiveness and potential utility of using machine learning and natural language processing techniques to develop models that can reliably predict the relative difficulty of incoming chat reference questions. Using a relatively large sample size of chat transcripts (N = 15,690), an empirical experimental design was used to test and evaluate 640 unique models. Results showed the predictive power of observed modeling processes to be highly statistically significant. These findings have implications for how library service managers may seek to develop and refine reference services using advanced analytical methods.