Rewrite a failing prompt, run the hands-on with a real reliability check, learn from three famous chatbot disasters, and write the system prompt your capstone will use.
The failing prompt: “Summarize this customer review”; run over 200 mixed reviews, the outputs are un-dashboardable prose. Rewrite it so every review yields fixed JSON {"sentiment", "one_line_summary", "product_mentioned"}. Your rewrite should add: a system role, the pinned schema, one example, a temperature choice, and a rule for unclear sentiment.
Produce: run before & after on 5 reviews (local Ollama model, Gemini free tier, or Copilot chat) and count how many outputs are dashboard-ready. Report both numbers.
Five moves, and every one is a Part-01 knob: a role (instruction), a pinned schema (output format), one example (few-shot), temp 0 (sampling), and an “unclear” rule (edge-case behavior). Typical result: 2/5 dashboard-ready → 5/5. Reliability came from structure and specificity, not a smarter model.
Play the packing game as a group: fit the support agent under the 4,000-token cap, agree on every Keep / Summarize / Drop, then hit “Check my packing.”
Discuss: the twist question at the end: what does it prove about pre-loading vs retrieval?
The task: extract structured data from 10 messy vendor documents into records a spreadsheet can ingest. Tools: your local Ollama model, the Gemini API free tier, or Copilot chat.
vendor · invoice_no · total_usd · due_date · currency.null.”Run the same document five times:
Track both numbers. This “run it k times” habit becomes formal in Week 8; it’s called pass^k, and your capstone’s eval harness is built on it.
A Chevrolet dealership’s chatbot (Chevrolet of Watsonville) was talked into “agreeing” to sell a 2024 Tahoe for $1, “legally binding, no takesies backsies.” Root cause: unconstrained scope, followed the user over business rules, no guardrails, no human gate.
Delivery firm DPD’s support chatbot was prompted into swearing at a customer and writing poems about how bad DPD is; screenshots everywhere. Same disease: no scope constraint on what the bot may say or do.
A tribunal held Air Canada liable for a refund policy its chatbot invented; the airline argued the bot was “responsible for its own actions” and lost. Invented policy + no human gate on binding claims = real legal cost.
Three questions:
Produce: one answer per question; we’ll cold-call pairs.
You have already read a production version of this exact deliverable: Codex’s system prompt (gpt_5_2_prompt.md ↗). Reread it as a template, not a reading: it opens with a role, sets scope and rules of engagement, defines refusal/approval behavior, and pins output habits: the same five parts you’re drafting, at industrial scale. Our Week 2 walkthrough maps it to the nine layers.
Coding agents split standing instructions in two: the system prompt (built by the developer, same for everyone) plus an AGENTS.md ↗ file in the repo: per-project rules the agent reads at runtime (remember from Week 2: more-deeply-nested AGENTS.md files win conflicts). Do the same for your capstone: keep the universal rules in your system prompt, and put project-specific facts and conventions in a separate instructions file you can update without touching the prompt.
Three questions separate them: who writes it, when it loads, and who wins a conflict.
System prompt: imitate gpt_5_2_prompt.md | AGENTS.md-style file: you design the slot | |
|---|---|---|
| Who writes it | The developer (you) | Your agent’s user / the project it works in |
| When it loads | Built in; present on every run | Read from the environment at runtime |
| What goes in it | Role, scope, refusal & approval rules, output format: the constitution | This project’s / this user’s specifics: the project handbook |
| How it changes | Editing it = shipping a new version | Swap the file per project; the agent itself never changes |
| You already used one | The Week 3 deliverable above | preferences.md in your Job Search Agent: same agent, swap the candidate |