Skip to main content
prompt GP ChatGPT

Machine Learning Experiment taste — ChatGPT prompt

Machine Learning Experiment taste — a prompt for ChatGPT sets practical rules for planning and evaluating machine learning experiments. It covers data splits, baselines, and tracking results for reproducibility.

CC0-1.0 ★ 178,201 updated 2026-10-24

Open on GitHub

What Machine Learning Experiment taste does

The prompt lays out an experiment workflow: define a metric and a simple baseline before training, choose a data split that avoids leakage, and reserve the test set for the final evaluation. It calls for a separate validation set during tuning and a fixed compute budget.

It recommends recording runs with MLflow or Weights & Biases, alongside configuration, dataset versions, random seeds, and dependencies. For tabular tasks, it favors gradient-boosted trees; for training, it names PyTorch with a small loop or Lightning. Metrics should be broken down by slices, and label errors checked by hand.

The guidance also covers tests for feature transforms and data loaders, schema validation, and offline comparison with the production model before a shadow or A/B rollout. The Claude Code file path is .claude/commands/prompts.csv.md.

What it helps with

  • set a metric and baseline before model training
  • split data to reduce leakage risk
  • compare model quality across data slices
  • record configurations and dataset versions for repeatable runs
  • evaluate a new model before production rollout

Example prompts for the agent

  • Plan a tabular ML experiment and include a simple baseline.
  • Suggest a time-based split for this dataset to reduce leakage risk.
  • Draft checks for evaluating a new model against the production model.

How to install Machine Learning Experiment taste

How to use

Paste the text at the start of a chat or into project instructions

Questions about Machine Learning Experiment taste

How do I install Machine Learning Experiment taste in Claude Code?

Save the prompt to .claude/commands/prompts.csv.md.

What does Machine Learning Experiment taste do?

It lays out practices for ML experiments, including baseline selection, data splitting, run tracking, and model evaluation.

Which tools does the prompt name for experiment tracking?

It lists MLflow and Weights & Biases. For configuration, it names Hydra or Pydantic.

Source text

prompts.csv · CC0-1.0
# Machine Learning Experiment taste of an applied ML engineer
- Write the evaluation metric and a dumb baseline before training any real model. Confidence: 0.95
- Split data by time or by entity when leakage is possible, never with a random split by default. Confidence: 0.90
- Freeze the test set early and look at it only for final decisions, using a separate validation set for tuning. Confidence: 0.95
- Track every run with MLflow or Weights & Biases, logging config, metrics, code version and dataset version. Confidence: 0.90
- Keep experiment config in versioned YAML files loaded with Hydra or Pydantic, not in hardcoded constants. Confidence: 0.80
- Set and log random seeds for Python, NumPy and the framework, and report variance across several seeds. Confidence: 0.85
- Version datasets with DVC or immutable object storage paths so any result can be reproduced. Confidence: 0.80
- Prototype in notebooks but move reusable code into a tested Python package as soon as it is used twice. Confidence: 0.85
- Pin dependencies with a lock file and record the CUDA and driver versions with each run. Confidence: 0.75
- Inspect raw samples and label errors by hand before trusting any aggregate metric. Confidence: 0.90
- Prefer gradient boosted trees for tabular data and reach for deep learning only when they clearly fall short. Confidence: 0.80
- Use PyTorch with a thin training loop or Lightning instead of large custom frameworks. Confidence: 0.70
- Report metrics per slice such as region, device or class, not only a single global number. Confidence: 0.90
- Calibrate predicted probabilities when downstream code makes threshold decisions. Confidence: 0.75
- Tune hyperparameters with Optuna or a simple random search under a fixed compute budget. Confidence: 0.75
- Write unit tests for feature transforms and data loaders, and assert expected shapes and dtypes. Confidence: 0.85
- Validate input data with schema checks such as Pandera or Great Expectations before training. Confidence: 0.80
- Save the model together with its preprocessing pipeline as one artifact so training and serving cannot drift. Confidence: 0.90
- Compare a new model to the production model offline and then with a shadow or A/B rollout. Confidence: 0.85
- Monitor feature drift and prediction distributions after launch and alert on large shifts. Confidence: 0.80
- Write a short experiment note for every significant run with the hypothesis, the result and the next step. Confidence: 0.85
- Delete or archive failed branches of work instead of leaving dozens of half-finished notebooks. Confidence: 0.65

© f · CC0-1.0 · f/prompts.chat

Similar prompts

All prompts for AI and ML →

From the same set218

The whole set →
SvelteKit App taste The SvelteKit App taste ChatGPT prompt sets development conventions for SvelteKit apps. It covers project setup, data loading, forms, and testing. GP ChatGPT ★ 178,201repo stars Tauri Desktop App taste Tauri Desktop App taste — a prompt for ChatGPT that sets engineering practices for cross-platform Tauri apps. It covers Rust commands, UI boundaries, security, testing, and release builds. GP ChatGPT ★ 178,201repo stars Vue App taste The Vue App taste ChatGPT prompt sets conventions for building Vue applications. It covers framework choice, component patterns, data handling, and tests. GP ChatGPT ★ 178,201repo stars Background Jobs taste Background Jobs taste — a prompt for ChatGPT and Claude.ai guides background worker design and queue reliability. Sets clear constraints for idempotency, retry budgets, and payload versioning. GP ChatGPT ★ 178,201repo stars Technical Documentation taste Technical Documentation taste — a prompt defining standards for technical documentation for developers. It helps structure guides and API references using the Diataxis model to keep content accurate and accessible. GP ChatGPT ★ 178,201repo stars React Native App taste The React Native App taste ChatGPT prompt sets engineering preferences for cross-platform React Native apps. Ask for a project plan or an architecture decision to get guidance on Expo, navigation, state, and testing. GP ChatGPT ★ 178,201repo stars Feature Flag Rollout taste Feature Flag Rollout taste — a prompt for ChatGPT and Claude.ai establishes release engineering standards for feature flags. Guides safe rollouts without coupling code deployments to user enablement. GP ChatGPT ★ 178,201repo stars LLM Application taste The LLM Application taste prompt for ChatGPT lays out engineering practices for applications built with language models. It covers prompt evaluation, tool use, and handling untrusted input. GP ChatGPT ★ 178,201repo stars

Project sponsors

Companies that keep this analytics open