LLMs, plus Tokens, Context, and Cost#

A brain made of nested rings: how AI, machine learning, deep learning, and LLMs contain one another

To use AI technology effectively, you need to understand its key concepts. That is what lets you grasp how AI services and tools work under the hood, and why the latest news and trends matter.

AI is still advancing at a rapid pace: related services and tools are updated continuously, and new developments appear all the time. If you build a solid grasp of the fundamentals, you can follow the flow of change more easily and keep growing your ability to apply AI.

The first part of this chapter sorts out how AI, machine learning, deep learning, and LLMs relate to one another. The second part covers four practical concepts you will inevitably run into when wiring an LLM into a service or calculating its cost: tokens, the context window, cost structure, and embeddings.

AI (Artificial Intelligence)#

AI refers to computer systems that imitate human intelligence to perform tasks such as recognizing or generating text, images, and video. The term can also mean the technology itself, or the academic field concerned with implementing that technology. Discussions around AI began in the 1940s, and the name “Artificial Intelligence” was formally adopted in 1956.

AI is sometimes divided by capability into Weak AI and Strong AI. Weak AI refers to AI limited to specific tasks or problems; Strong AI refers to AI with human-level intelligence and consciousness. A similar classification is ANI (Artificial Narrow Intelligence) versus AGI (Artificial General Intelligence): ANI corresponds roughly to Weak AI and AGI roughly to Strong AI.

The two classification systems are used in slightly different contexts. Weak / Strong has been discussed for a relatively long time and is mostly used from a philosophical perspective, while ANI / AGI appears more often from a technical perspective. At the time of writing, most of the AI services we encounter in daily life fall under ANI, and there are still many differing opinions on when the arrival of AGI should be dated.

Machine Learning and Deep Learning#

Machine learning is a branch of AI in which you design a model, train it on data, and then use the trained model to perform the task you need.

For example, suppose you are given the equation (y = ax + b) and a known data point ((x_1, y_1)), and asked to find (a) and (b), then compute (y_2) for a newly given (x_2). Machine learning is having a computer solve this kind of problem by learning the rule from data. A function (or set of functions) such as (y = ax + b) that expresses the relationship between input (x) and output (y) becomes the model; once you train it on enough ((x, y)) data and find suitable parameters (e.g., (a) and (b)), it can predict (y) for a new (x).

Deep learning is a subfield of machine learning that designs and trains artificial neural networks (ANNs), modeled on the structure of the human brain. In other words, it is the branch of machine learning that learns complex patterns from data using neural networks.

LLMs are a branch of deep learning, deep learning is a branch of machine learning, and machine learning is a branch of AI.

Training Methods in Machine Learning#

Machine learning training methods fall into roughly four categories.

Supervised Learning#

This method trains a model on pairs of input and output data. For example, to build a model that can tell whether a photo contains a cat, you can train it on photos with cats and photos without. In this case the photo is the input, and whether or not a cat is present is the output. Another example is an email spam filter, which can be trained on spam and non-spam messages.

Unsupervised Learning#

This method trains a model using only input data, with no outputs. Supervised learning teaches the model what the correct answer is for each input, producing a model that solves problems like “This is A, and this is B. So what is that?” Unsupervised learning instead trains on data without answers, grouping items with similar characteristics together (this is called clustering) to produce a model that solves problems like “There are groups A, B, and C. Which group does this data belong to?” For example, you can use it to build a feature that groups similar news articles together, or one that learns from product data to offer related or recommended products.

Semi-Supervised Learning#

Semi-supervised learning combines supervised and unsupervised learning: the model is trained on data where some items have outputs and the rest do not. Real-world data is often not as clean as you would like. Suppose you are building the cat-detection model again, but among the photos you want to train on, only some are labeled with whether a cat is present, while for the rest you do not know. In that case, for a given input photo, you can find the data cluster it matches and then derive the result from the labeled items within that cluster.

Reinforcement Learning#

Reinforcement learning strengthens a model by repeatedly evaluating the outputs it produces. It can be compared to a trainer teaching a dog: when the trainer gives a command, the dog gets a reward if it responds in the right direction and no reward if it responds in the wrong direction, repeated over and over. Reinforcement learning is not mutually exclusive with the other methods. For example, you can train an initial model with supervised learning and then further train it with reinforcement learning to improve its performance. This is also how today’s LLMs are refined to give “the answers people prefer.”

LLMs (Large Language Models)#

An LLM is a deep learning model trained on vast amounts of text data so that it can understand and generate language. OpenAI’s GPT (Generative Pre-trained Transformer), Google’s Gemini, and Anthropic’s Claude are representative LLMs.

Deep learning is the technique of designing and training artificial neural networks, and those networks come in several types depending on the purpose. The main ones are CNNs (Convolutional Neural Networks), RNNs (Recurrent Neural Networks), and Transformers; today’s LLMs are built mainly on Transformers. CNNs are often used to process two-dimensional data such as images, while RNNs and Transformers are used for natural language processing, speech recognition, translation, and so on. In natural language processing in particular, Transformer-based LLMs have become mainstream and the role of RNNs has shrunk dramatically.

Put very simply, what an LLM does is “look at the text so far and predict the next piece.” Its ability to answer questions, write code, and translate is all the result of becoming extremely good at that prediction. Keep this simple principle in mind, and the concepts that follow, such as tokens, the context window, and hallucination, will make much more sense.

Tokens: The Unit an LLM Counts Text In#

An LLM reads and writes text not in characters or words but in tokens. A token is a frequently occurring chunk of characters; in English, one word is roughly one to two tokens, while Korean tends to produce more tokens than English for a sentence with the same meaning. The exact count varies by model, but a rule of thumb like “one Korean manuscript page (200 characters) is roughly 150 to 250 tokens” is good enough for practical work.

Tokens matter for two reasons. First, the limit on how much an LLM can read at once is set in tokens (the context window). Second, API usage fees are calculated by token count (cost). In other words, a token is both the “unit of length” and the “unit of currency” in the LLM world.

Context Window: How Much Can Be Seen at Once#

The context window is the maximum range of tokens a model can read and write in a single request. Not just the user’s question, but the system instructions, the previous conversation history, any documents you include, tool call results, and the answer the model generates all have to fit inside this range.

Everything the model sees in a single request has to be inside the context window.

Context windows have grown with each generation, but “large” and “well used” are different matters. A large window does not mean you should throw in every document you have; the model will easily miss the point, and the cost goes up. That is why the skill of choosing the right information in the right amount has become important, and this book covers it in Chapter 3 under the name context engineering.

Cost Is Calculated per Token#

When you use an LLM through an API, the fee is usually calculated by multiplying input tokens and output tokens by their own separate rates. Output tokens are typically several times more expensive than input tokens. And most providers offer several tiers of model within the same generation (top-tier / balanced / lightweight) at different performance and price points.

One more thing worth knowing here is prompt caching. The provider caches the leading part of a request (the parts that repeat identically every time, such as the system prompt or a long reference document), and if that part is the same in the next request, it is billed at a much cheaper rate. In an agent-style setup, where the same instructions are sent dozens of times over, this cache is what decides the cost. The specific design is covered in the Execution section, in “Adding AI Features to Your Product.”

Also keep in mind that using a chat service on a monthly subscription and using an API on a pay-as-you-go basis are completely different pricing schemes. A subscription favors “an individual who uses a lot for a flat fee,” while the API only becomes an option when “you call the model on behalf of users inside your own product.” This difference determines the cost structure of a one-person startup, so it gets its own treatment in the Execution section.

Embeddings: Turning Meaning into Numbers#

Finally, to understand RAG in the next chapter, you need to know about embeddings. An embedding is a piece of text (or an image) converted into a vector made up of hundreds to thousands of numbers. The important property is that text with similar meaning has vectors that are close together. The question “refund policy” and a document titled “reimbursement procedure for returns” use different words, but they sit close to each other in embedding space.

Thanks to this property, “the document chunk whose meaning is closest to the question” can be found mathematically. Split your documents into small pieces in advance, convert them into embeddings and store them (such a store is called a vector database); when a question comes in, pull out the chunks closest to the question’s embedding and show them to the model alongside the question. That is the backbone of RAG, explained in the next chapter. Embedding generation is also offered through dedicated models and APIs, and it is far cheaper than text generation.

Summary#

There are five things to remember from this chapter.

An LLM is a branch of deep learning: a model that “predicts the next piece.”
A token is the LLM’s unit of length and its unit of currency.
The context window is how much can be seen at once, and the instructions, history, documents, tool results, and answer all have to fit inside it.
Cost is determined by the input / output token rates and whether caching applies.
Embeddings turn meaning into vectors, making “find similar things” possible, and are the foundation of RAG.

Building on this foundation, the next chapter covers the ways of connecting a pre-trained LLM to your own service’s knowledge and tools: Fine-tuning, RAG, Function Calling, and MCP.