Key AI Concepts (3): Prompt Engineering / Context Engineering#

This chapter covers prompt engineering, which is about “how to compose questions / instructions,” and context engineering, which is about “how to provide the material (background information) the model should consult when producing an answer.” Recall the context window from the previous chapter: prompt engineering is writing the “instructions” that go into it well, and context engineering is designing “everything” that goes into it.

Prompt Engineering#

A prompt is the text you use to query (question / instruct) an AI model. For example, when you use ChatGPT, what you type into the input box is the prompt. Prompt engineering is the skill of writing prompts to get the result you want from an AI model. It is just typing text into a chat box, yet we call it “engineering” because the quality of the output changes that much depending on how you write the input, and because it is a skill where experience and know-how have to accumulate on top of basic principles.

OpenAI and Anthropic each publish an official prompting guide (OpenAI, Anthropic). The detailed techniques differ slightly from model to model, but the core principles are shared. Here I condense those shared principles into six.

Principle 1. Write specific instructions#

Spell out the context of the result you want and the information you need. Rather than “How do I add up numbers in Excel?”, it is better to write “I want to sum rows of dollar amounts in Excel. I want to automatically add every value to the left of the ‘Total’ column and put the sum in the ‘Total’ column.” State the amount of output too: “in one sentence,” “three titles,” “about 500 characters,” and so on.

Principle 2. Give it a role (persona)#

If you set the model’s role in advance, such as “You are a Python programmer. When given code, add comments and explain it,” the perspective and vocabulary level of the answer adjust to that role.

Principle 3. Separate inputs with delimiters#

When instructions and the text to be processed are mixed together, the model gets confused. Make the boundaries clear with triple quotes or XML tags.

You are given a pair of articles on the same topic (delimited by XML tags).
First summarize the argument of each article, then explain which argument is stronger and why.

<article>First article content</article>
<article>Second article content</article>

Principle 4. Break work into steps and show examples#

For complex tasks, instruct in steps like “Step 1 summarize, Step 2 translate,” and for formats or styles that are hard to describe in words, show one example and say “do it like this.” Splitting one large prompt into several small ones, such as classifying a customer inquiry first and then applying different instructions per category, follows the same principle.

Principle 5. Provide reference material, and have it say so when it does not know#

For questions that need evidence, include reference material and attach the rule “if the answer cannot be found in the material, reply ‘Insufficient information.’” This one line is among the most effective devices for reducing hallucination (plausible-sounding made-up answers).

Answer the question using only the document delimited by triple quotes, and quote the passages you used.
If the document does not contain the needed information, write "Insufficient information."
"""
Document content
"""
Question: the question

Principle 6. Have it think first#

When the model has to judge “right / wrong,” reliability goes up if you have it work the problem out itself first and then compare, rather than asking for the verdict straight away. Instead of “Tell me whether the student’s solution is correct,” say “First solve the problem yourself. Then compare it with the student’s solution and evaluate it. Do not judge before you have solved it yourself.” Recent models also offer a reasoning mode with an internal “thinking step,” and even then an instruction saying “here is what to check” is still useful.

Prompts should be managed like code#

When using a chat service, the principles above are enough. But if you are a developer building and operating a service through an API, there is one more thing you need to know: when the model is updated, the same prompt can produce different results. A model that answered in a consistent format until yesterday may, after an update, subtly change its answer style or output format. From the service’s point of view, it feels as if the behavior suddenly changed.

That is why in production you usually pin a specific model snapshot (version) for predictability, version-control prompts like source code, and, whenever you modify a prompt or swap models, run evals, a prepared “exam set,” to confirm the key cases still work. The specifics of evaluation are covered in the “Operations” chapter of the Execution section.

Context Engineering#

If prompt engineering is designing “what to ask for (instructions / format),” context engineering is designing “in what way to supply the material (background information) the model consults when producing an answer.” Even with the same prompt, the accuracy and consistency of the answer vary greatly depending on what context you provide alongside it.

The first principle of context engineering is simple: from the model’s point of view, what is not in the context does not exist. The project rules in your head, the conversation you had with a colleague, the decisions written in some other document: none of these are visible to the model. When the model gives an off-target answer, before asking “why doesn’t it know this?”, first check “did I put this in the context?”

Principles of context that works#

• Put things in by priority: the important things first, arranged so that “what counts as the evidence for the answer” is clear.
• Manage length: too long, and the model misses the point, you burn through the context window quickly, and the cost rises. Summarize long conversations and pick only the needed parts of documents.
• Divide into sections: formats with clear boundaries, like Markdown headers or XML tags, help.
• Check freshness / consistency: mixing in outdated policy documents actually lowers quality.
• Make it answer from evidence: include rules such as “answer only from the documents below, and say you don’t know if they don’t cover it.”
• Repeated parts first, changing parts last: putting fixed instructions and reference material up front lets you benefit from prompt caching.

RAG and Function Calling are forms of context engineering too#

The RAG of the previous chapter is “a way of automatically finding the needed documents and attaching them to the context,” and performing DB lookups / calculations / API calls through Function Calling and then supplying the results as context is also an important part of context engineering. In other words, context engineering is a broad concept that spans everything from a person pasting in documents by hand to automatically assembling context through retrieval and tool execution.

How I actually do it: rule files, memory, skills#

Pasting long instructions into the chat box every time does not last. In my company projects and in personal projects including this blog, I manage context as files inside the repository. Coding agents (Cursor, Claude Code, and the like) automatically read certain files into the context when they open a project, so if you write instructions and material into those files, every session starts from the same context.

<project root>/
├── CLAUDE.md              # Entry point for agent instructions (acts as a table of contents, under 100 lines)
├── ARCHITECTURE.md        # Map of the project structure
├── .cursor/rules/         # Rules for Cursor (writing style, frontmatter, prohibitions …)
├── .claude/skills/        # Procedures for recurring tasks (e.g., "write a news brief," "publish an essay")
├── .memory/
│   ├── learnings.md       # Patterns, failure causes, and know-how discovered while working
│   └── current-focus.md   # Current interests and priorities
└── docs/decisions/        # Design decision records

Mapping this structure onto the five layers above gives the following.

Set up this way, “writing good instructions” becomes a project asset rather than an individual’s memory. Even in a fresh session, the agent immediately understands the project’s writing style, structure, and deployment method, and does not repeat the mistakes discovered in the last session. This structure is the starting point of the harness covered in the next chapter.

Summary#

• The core of prompt engineering comes down to six things: specific instructions, a role, delimiters, steps and examples, reference material, and having the model think first.
• If you operate a service, version-control prompts like code, pin the model snapshot, and verify with evals.
• Context engineering is designing “everything the model sees,” and what is not in the context does not exist for the model.
• Design context in five layers: fixed instructions, reference material, state, tool results, and the current request.
• Manage instructions and material as files inside the repository, and context becomes a project asset rather than an individual’s memory.