Chapter 2: Meet the Brain

Ravi is a junior dev at a small startup. His boss drops a task on his desk: "We need a thing that reads customer reviews and sorts them into 'positive', 'negative', and 'needs attention'. By Friday." Ravi's first instinct is the way he's always solved problems: write rules. If the review contains "love" or "great", mark it positive. If it contains "broken" or "angry", mark it negative. He spends a day writing regex patterns. By Wednesday he has 40 rules and they still miss half the reviews. "This movie was a killer deal" gets flagged negative. "Not bad at all" gets flagged negative. He's losing his mind. Then his coworker Priya leans over. "Just ask the model to do it." Ravi blinks. "Ask the... what?"

Why This Matters

By the end of this chapter, you'll have called an LLM from real Python code, shaped its output with a prompt, and met the three knobs that change how it behaves: tokens, temperature, and the prompt itself. You'll also have hit the one fact that shapes everything about building agents: the brain is stateless. It forgets everything between calls. Hold onto that — it's the reason Chapter 5 exists.

The Brain Is a Text Machine

Here's the least useful but most honest description of an LLM: it's a machine that takes in text and predicts what text comes next. That's it. You feed it a string; it gives you back a string. In between, something remarkable happens — but from the outside, from your code's perspective, it's a function: text_in → text_out.

From your code's perspective, the LLM is a function: text in, text out.

Ravi's review-sorting problem is a perfect fit. Instead of 40 regex rules, he sends the review to the LLM and asks it to classify. The LLM has read billions of sentences during training; it knows what a positive review sounds like. Ravi doesn't teach it English. He just asks.

Your First Model Call

Let's write Ravi's first call. We'll use the OpenAI Python library — it's the most common way to talk to an LLM, and the pattern is the same everywhere. (If you're using a different provider, the shape is identical; only the import changes.)

# pip install openai  — run this once in your terminal
from openai import OpenAI

client = OpenAI()  # picks up your API key from the environment

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You sort reviews. Reply with one word: positive, negative, or needs_attention."},
        {"role": "user", "content": "The headphones broke after two days. Worst purchase ever."}
    ]
)

print(response.choices[0].message.content)
# Output: negative

// Two messages: a "system" message that sets the rules, and a "user" message with the actual review.

That's it. Ravi just replaced 40 regex rules and a day of suffering with 12 lines of code. The LLM read the review and said "negative." Not because someone wrote a rule for "broke" — but because it understands the meaning of "broke after two days" and "worst purchase ever."

Note The messages list is how you talk to the LLM. Each message has a role and content. The system role sets the behaviour — the rules of the game. The user role is the actual input. You'll see a third role, assistant, later — that's the LLM's own previous replies.

There Are No Dumb Questions

Q: Where do I get an API key? And does this cost money?

A: You sign up at the OpenAI platform (or whichever provider you're using), create an API key, and set it as an environment variable called OPENAI_API_KEY. Yes, it costs money — but we're talking fractions of a cent per call for small models like gpt-4o-mini. The examples in this book will cost you less than a coffee to run end to end.

Q: Do I have to use OpenAI? What about open-source models?

A: Not at all. The patterns in this book work with any LLM provider — Anthropic, Google, local models via Ollama, open models on HuggingFace. The code changes by one line (the import and the client). The ideas — prompts, tokens, temperature, the loop — are universal. We use OpenAI in examples because it's the most common starting point, but nothing here is locked in.

Tokens: The Brain's Currency

The LLM doesn't see words the way you do. It sees tokens — chunks of text that might be a whole word, part of a word, or just a few characters. "ChatGPT" might be one token. "Unbelievable" might be three: "un", "believ", "able".

Three words, three tokens. Longer or unusual words might split into more.

Why should you care? Two reasons:

1. Tokens cost money. You pay per token — both for what you send in (the prompt) and what you get out (the response). A rough rule of thumb: 1 token ≈ 4 characters, or about ¾ of a word. A 1,000-word email is roughly 1,300 tokens.

2. There's a limit. Every model has a context window — a maximum number of tokens it can hold in its "working memory" for a single call. Send more than that and the model either truncates your input or refuses the call. We'll wrestle with this for real in Chapter 5 (Memory).

Geek Bits Why doesn't the model just use words? Because tokens let it handle prefixes, suffixes, and languages that don't use spaces between words. "Running" becomes "run" + "ning", so the model sees the connection to "run". It's a compression trick that makes the model smarter about word families. You don't need to think about this day-to-day — but it's why the model sometimes splits a word oddly.

Temperature: The Creativity Dial

When the LLM predicts the next token, it doesn't pick with certainty. It assigns probabilities to its options and picks from them. Temperature controls how adventurous that picking is.

Temperature widens or narrows the net. Low = predictable. High = surprising.

For Ravi's review sorter, the answer is obvious: temperature 0. He wants the same review to always get the same answer. There's one correct label; creativity is the enemy. But later in the book, when we're asking an agent to brainstorm solutions or draft an email, we'll turn the dial up.

response = client.chat.completions.create(
    model="gpt-4o-mini",
    temperature=0,  # ← the dial. 0 = deterministic, 1+ = wild
    messages=[...]
)

// temperature=0 means the same input always gives the same output. Great for classification.

Sharpen your pencil

You're building three things. For each one, what temperature would you set — low (0–0.3), medium (0.4–0.7), or high (0.8–1.0)? Think before you read on.

1. An agent that extracts dates from contracts and outputs them in ISO format.
2. An agent that writes taglines for a marketing campaign.
3. An agent that summarises a 50-page report into bullet points.

Answer: (1) low — you want exact, repeatable extraction. (2) high — you want variety and surprise. (3) low-to-medium — you want faithful summary, not creative rewriting.

The Prompt: Steering the Brain

The prompt is the instructions you give the brain. It's not a minor detail — it's the single biggest lever you have over how the LLM behaves. The same model, the same input, with a different prompt, gives you a completely different result.

Watch what happens when Ravi tweaks his system prompt:

# Prompt A — vague
"You are a helpful assistant. Sort this review."
# Output: "This review expresses strong dissatisfaction with the product..."
#          (a paragraph, when Ravi wanted one word)
# Prompt B — specific
"You sort reviews. Reply with ONE WORD only: positive, negative, or needs_attention."
# Output: "negative"  ← exactly what Ravi wanted

The model didn't change. The temperature didn't change. Only the instructions did. The prompt is how you program an LLM. Instead of writing code that tells the computer what to do, you write instructions that tell the brain what you want.

Watch it! A vague prompt is the #1 cause of "the LLM isn't working." Before you blame the model, ask: did I actually tell it what I want? Did I say the format? Did I give an example? Did I set the constraints? Nine times out of ten, a "broken" LLM call is a vague prompt.

The Big One: The Brain Is Stateless

Here's the fact that will haunt the rest of this book. When you call the LLM, it has no memory of the last call. None. Every single time you call client.chat.completions.create(), the model starts fresh. It's a goldfish.

Call 2 has no idea what happened in Call 1. The brain resets every time.

"But wait," you say, "ChatGPT remembers our conversation!" Yes — but ChatGPT is doing something extra. It's not that the model remembers. It's that the application sends the whole conversation history back with every new call. The memory lives in your code, not in the model. You send the past back in, every time.

# How ChatGPT seems to remember: it sends the whole history each time
messages = [
    {"role": "user", "content": "Hi, I'm Ravi."},
    {"role": "assistant", "content": "Hi Ravi! Nice to meet you."},
    {"role": "user", "content": "What's my name?"}  # ← the model sees ALL of this
]
# → "Your name is Ravi."  (because the history is right there in the messages)

// The model doesn't remember. YOU send the past back in, every single call.

Note This is why the messages list is a list and not a single string. It's the conversation, turn by turn, that you're choosing to replay. The model reads the whole thing fresh each time and responds to the latest message — but it can only "remember" what you put in that list.

There Are No Dumb Questions

Q: If I have to send the whole history every time, doesn't that get expensive and slow?

A: Yes! That's exactly the problem. As conversations grow, you send more tokens, pay more, and eventually hit the context window limit. This is the central problem of agent memory — and the entire subject of Chapter 5. For now, just hold the fact: the model is stateless, and memory is your job.

Q: So what's the difference between the "system" message and the "user" message?

A: The system message is the standing instruction — the rules of the game that don't change ("You are a review sorter. Reply with one word."). The user message is the actual input for this turn ("This review: ..."). The model treats the system message as higher-priority guidance. You set the system message once; the user message changes every call.

Where People Come Unstuck

Mistake #1: Expecting the model to remember

You call the LLM, tell it something, then call it again and expect it to know. It doesn't. You'll do this once, get a confusing answer, and then never forget. The fix is always the same: send the history in the messages list.

Mistake #2: Vague prompts, then blaming the model

"It's not doing what I want!" Check your prompt. Did you specify the format? Did you give constraints? Did you show an example? The model isn't psychic. A precise prompt is the difference between magic and disappointment.

Mistake #3: Using high temperature for tasks that need accuracy

If you're extracting data, classifying, or following a strict format, turn the temperature down. Creativity is wonderful for brainstorming and writing. It's your enemy when you need the same answer every time.

Brain Power

Ravi's boss comes back with a new request: "Now sort reviews into five categories — positive, negative, neutral, needs_attention, and spam — and give a one-sentence reason for each."

Sketch out how you'd change the system prompt and the output you'd expect. What format would you ask for? Would you change the temperature? Would one call be enough, or might you need to think about the structure of the response? You don't need code — just think it through in terms of prompt, temperature, and the text-in/text-out model.

Chapter Summary

The LLM is a text-in, text-out machine. From your code's perspective, it's a function: you send a string, you get a string back.
You call it via a messages list: a system message sets the rules, user messages carry the input, and assistant messages are the model's own past replies.
Tokens are the LLM's currency — chunks of text, roughly 4 characters each. You pay per token, and every model has a maximum context window it can hold.
Temperature is the creativity dial: 0 is deterministic (same input, same output), higher values are more varied. Use low for accuracy, high for creativity.
The prompt is how you program the LLM. A precise prompt — with format, constraints, and examples — is the single biggest lever over behaviour.
The brain is stateless: it forgets everything between calls. Any "memory" of a conversation exists because your code sends the history back in with every call. This is the problem Chapter 5 solves.

Chapter Challenge

The Prompt Lab. Below is one review. Write three different system prompts for it, each producing a different useful output:

"The camera takes great photos in daylight, but the battery dies by lunchtime. I'd return it if I could, but I lost the receipt. Customer service was polite but unhelpful."

1. A prompt that gets a single-word sentiment label.
2. A prompt that gets a JSON object with sentiment, product_issues (a list), and customer_action.
3. A prompt that gets a one-paragraph reply as if the agent were a customer service manager writing to the team.

For each, also state what temperature you'd use and why. Run all three if you have an API key — you'll feel the difference between a vague prompt and a precise one in your own hands.