IEEE AESS – Part I: LLM Basics

Garrett Hall, a Research Engineer at the Southwest Research Institute, will deliver an introductory presentation on Large Language Models. This talk is the first in a two-part series and covers several fundamental concepts, including tokenization, vector embeddings, and positional encoding.
Tokenization is the process of converting words or phrases into numerical values that machine learning models can understand. By breaking down text into smaller units called tokens, the model can more effectively process and analyze the data.
Vector embeddings are a crucial next step, where these tokens are transformed into dense vector representations. These vectors capture semantic meaning, enabling the model to understand relationships between words based on their contextual usage. Embeddings essentially map tokens into high-dimensional space where similar words are located closer together.
Positional encoding provides additional information about the order of the words in a sentence, establishing a foundation for sentence structure. It embeds positional information within the tokenized data so that the model can recognize the sequence and context of words, which is essential for understanding the meaning of the text as a whole.
Finally, the presentation will illustrate Retrieval-Augmented Generation (RAG) processes. RAG combines retrieval-based and generative models to enhance the generation of relevant and accurate text by incorporating external information sources. This section will demonstrate how the preceding concepts of tokenization, embeddings, and positional encoding come together in RAG to create more coherent and contextually appropriate text.
Cookies and refreshments will be served.
Talk is restricted to US citizens.
Registration required by COB Monday 3/10 for admittance to SwRI grounds on day of event.
Speaker(s): Garrett
Bldg: Building 51, 6220 Culebra Rd, San Antonio, Texas, United States, 78238