With the advent of big data and advances in technology, psychological assessments have become increasingly sophisticated and complex. Nevertheless, traditional psychometric issues concerning the validity, reliability, and measurement bias of such assessments remain fundamental in determining whether score inferences of human attributes are appropriate. We focus on three technological advances--the use of organic data for psychological assessments, the application of machine learning algorithms, and adaptive and gamified assessments--and review how the concepts of validity, reliability, and measurement bias may apply in particular ways within those areas. This provides direction for researchers and practitioners to advance the rigor of technology-based assessments from a psychometric perspective.