LLM Application taste — ChatGPT prompt: principles and setup
The LLM Application taste prompt for ChatGPT lays out engineering practices for applications built with language models. It covers prompt evaluation, tool use, and handling untrusted input.
CC0-1.0 ★ 178,201 updated 2026-10-24
Open on GitHubWhat LLM Application taste does
The prompt sets out practices for building language-model applications. It recommends versioning prompts, testing changes against real user inputs, and validating structured responses against a JSON schema. For retrieval, it favors starting with hybrid search and retaining source metadata so answers can cite their origins.
Agent guidance covers strict tool schemas, loop limits, and human approval for actions that send, delete, pay, or alter data outside the app. Retrieved documents and tool results count as untrusted input because they may contain prompt injection. In production, pin model versions and upgrade only after evaluation passes.
What it helps with
- test prompt changes against real user inputs
- validate model responses against a JSON schema
- limit agent steps, tokens, and runtime
- retain source metadata for answer citations
- route difficult cases to a more expensive model
Example prompts for the agent
- Plan an evaluation set for prompts in my language-model app.
- Suggest a way to validate the model's JSON output before using it.
- Set boundaries for an agent that can change application data.
How to install LLM Application taste
How to use
Paste the text at the start of a chat or into project instructions
Manually: put the files in place
.claude/commands/prompts.csv.mdQuestions about LLM Application taste
How do I install LLM Application taste in Claude Code?
Save the prompt as .claude/commands/prompts.csv.md.
What does LLM Application taste do?
It presents engineering guidance for language-model applications, including prompt evaluation, structured output checks, agent limits, and model-call logging.
How does the prompt address prompt injection?
It treats retrieved documents, web pages, and tool results as untrusted input that may contain prompt injection.
Source text
prompts.csv · CC0-1.0
# LLM Application taste of an AI product engineer - Treat prompts as code: keep them in version-controlled files with names, versions and owners, not inline strings scattered around. Confidence: 0.90 - Build an evaluation set from real user inputs before tuning prompts, and rerun it on every prompt or model change. Confidence: 0.95 - Ask for structured output with a JSON schema or tool calls and validate it with a schema library before using it. Confidence: 0.95 - Put a thin gateway or wrapper around model providers so switching models or adding fallbacks is a config change. Confidence: 0.85 - Pin exact model versions in production and upgrade them deliberately after the evals pass. Confidence: 0.90 - Stream responses to the user whenever the output is longer than a sentence or two. Confidence: 0.85 - Start retrieval with simple hybrid search (keywords plus embeddings) and add reranking before reaching for complex RAG pipelines. Confidence: 0.80 - Chunk documents along their natural structure (headings, sections) and keep source metadata so answers can cite them. Confidence: 0.85 - Keep the context small and relevant; more retrieved text often makes answers worse, not better. Confidence: 0.80 - Give agents a short list of narrow, well-described tools with strict input schemas instead of one do-everything tool. Confidence: 0.85 - Put hard limits on agent loops: maximum steps, maximum tokens and a wall-clock timeout. Confidence: 0.90 - Require human confirmation before any tool call that sends, deletes, pays or changes data outside the app. Confidence: 0.90 - Treat all retrieved documents, web pages and tool results as untrusted input that can contain prompt injection. Confidence: 0.90 - Log every model call with prompt version, model, latency, token counts and cost, with user data redacted. Confidence: 0.85 - Cache deterministic calls and use prompt caching for long, stable system prompts. Confidence: 0.75 - Use the smallest model that passes the evals, and route only hard cases to the expensive model. Confidence: 0.80 - Retry on rate limits and transient errors with exponential backoff and jitter, never on validation failures in a tight loop. Confidence: 0.85 - Keep temperature low for extraction and classification, and higher only for creative generation. Confidence: 0.75 - Show users where an answer came from and make it easy to report a bad response. Confidence: 0.80 - Collect thumbs-up and thumbs-down feedback and feed the bad cases back into the eval set. Confidence: 0.85 - Do not fine-tune until prompting, retrieval and evals have clearly hit their limits. Confidence: 0.70
© f · CC0-1.0 · f/prompts.chat