How Models Process Information

Understanding what goes on inside an AI model can feel like staring into a black box - and to some extent, it is. But there are key stages in how models break down, represent, and work with information that are genuinely useful to understand. When you type a question into ChatGPT, that text goes through a series of transformations before the model can do anything with it: it gets split into pieces, converted into numbers, placed in a mathematical space where meaning is represented as direction and distance, and processed through layers that determine what to pay attention to. Each of these steps shapes what the model can and can't do. Context window limits determine how much information it can consider at once. Tokenisation affects how it handles unusual words or languages. Embeddings determine which concepts it treats as similar. None of this is arbitrary - these processing choices create the capabilities and constraints you encounter every time you use an AI tool. Knowing the basics helps you understand why models sometimes behave in surprising ways.