Key AI Concepts (3)#

In this chapter, we cover prompt engineering (how to structure questions and instructions) and context engineering (how to provide the background materials the model should use to produce an answer).

Prompt Engineering#

A prompt is the text you give to an AI model as a query (a question or instruction). For example, in ChatGPT, whatever you type into the input box is the prompt. Prompt engineering is the skill of writing prompts in a way that reliably produces the result you want. It may feel like “just typing in a chat box,” but the quality of the output can change dramatically depending on how you phrase and structure the input—so people call it “engineering” because there are principles, trade-offs, and know-how.

I Dug Into Prompt Caching and Found That Hit Rate Isn’t the Goal#

2026-08-05

It Started With “Where Does This Actually Happen?”#

I knew prompt caching reduced costs. My understanding went about as far as: when you send the same system prompt repeatedly, it gets cheaper from the second request onward.

But the more I thought about it, the stranger it seemed. If caching happens in some relay layer outside the model, then from the model’s point of view nothing is saved. And yet the savings are said to reach 90%. Would that mean most of the cost of using an LLM (Large Language Model) comes not from the model itself but from the layer around it?