Penalized structural equation models (PSEM) is a new powerful estimation technique that can be used to tackle a variety of difficult structural estimation problems that can not be handled with previously developed methods. In this paper we describe the PSEM framework and illustrate the quality of the method with simulation studies. Maximum-likelihood and weighted least squares PSEM estimation is discussed for SEM models with continuous and categorical variables. We show that traditional EFA, multiple group alignment (MGA), and Bayesian SEM (BSEM) are examples of PSEM. The PSEM framework also extends standard SEM models with the possibility to structurally align various model parameters. Exploratory latent growth models, also referred to as Tuckerized curve models, can also be estimated in the PSEM framework and are illustrated here with simulation studies and an empirical example.