In previous studies, problem solving appraisal has been identified as playing a key role in promoting positive psychological well-being. The Problem Solving Inventory is the most widely used measure of problem solving appraisal and consists of 32 items. The length of the instrument, however, may limit its applicability to large-scale surveys consisting of several instruments. This study investigated the possibility of reducing the number of items in the inventory using item response theory. We used the automated item selection procedure in Mokken analysis which identified 12 items as unscalable or loading on a separate scale or violating invariant item ordering. Rasch analysis, Mokken analysis, and classical test theory were then used to investigate the psychometric properties of the shorter version of the instrument. The results supported the reliability, validity, and dimensionality of the three subscales in the shortened version. The shortened version of the Problem Solving Inventory (PSI-20) and its subscales had very strong relationships with the original scale and its subscales, and the correlation of the total scale and the subscales of the shortened version with related variables was very similar to the relationships that the original scale and the subscales had with those same variables. The PSI-20 is thus beneficial in identifying changes in metacognition related to problem solving ability and can provide a basis for further intervention.