Knowledge tracing (KT) aims to predict students' future performances based on their former exercises and additional information in educational settings. KT has received significant attention since it facilitates personalized experiences in educational situations. Simultaneously, the autoregressive (AR) modeling on the sequence of former exercises has been proven effective for this task. One of the primary challenges in AR modeling for KT is effectively representing the anterior (preresponse) and posterior (postresponse) states of learners across exercises. Existing methods often employ complex model architectures to update learner states using question and response records. In this study, we propose a novel perspective on KT task by treating it as a generative process, consistent with the principles of AR models. We demonstrate that knowledge states can be directly represented through AR encodings on a question--response alternate sequence, where model generate the most probable representation in hidden state space by analyzing history interactions. This approach underpins our framework, termed alternate autoregressive KT (AAKT). In addition, we incorporate supplementary educational information, such as question-related skills, into our framework through an auxiliary task, and include extra exercise details, such as response time, as additional inputs. Our proposed framework is implemented using advanced AR technologies from Natural Language Generation for both training and prediction. Empirical evaluations on four real-world KT datasets indicate that AAKT consistently outperforms all baseline models in terms of area under the receiver operating characteristic curve, accuracy, and root mean square error. Furthermore, extensive ablation studies and visualized analysis validate the effectiveness of key components in AAKT.