simulation-calibrator · davepoon — команда для Claude Code: разбор и запуск
Команда simulation-calibrator калибрует точность симуляций через поиск когнитивных искажений и бэктестинг. Пригодится при моделировании бизнес-процессов и архитектуры систем.
MIT ★ 3 785 обновлён 2026-10-25
/plugin marketplace add davepoon/buildwithclaude
/plugin install all-commands@buildwithclaudeЧто делает simulation-calibrator
Ассистент проверяет назначение симуляции и запрашивает порог допустимой точности. Для критических решений шкала требует от 95%, а для поискового анализа хватает диапазона 50–70%. Без понимания контекста агент не начинает калибровку. Он сначала выясняет тип сценария: бизнес-модель, масштабирование архитектуры или поведение людей.
Оценка опирается на бэктестинг и взвешенный скоринг. Четыре блока получают равный вес по 25%: подготовка исходных сведений, структура модели, строгость методики и ценность выводов. Инструмент оценивает итоговую погрешность и направление тренда.
Отдельный шаг ищет систематические искажения. Шаблон выявляет ошибки выжившего, переоценку недавних сведений, линейные упрощения и когнитивный оптимизм. Для исправления команда предлагает ред-тиминг и Байесовское обновление параметров.
Какие задачи решает
- настроить проверку прогностической точности через бэктестинг
- выявить скрытые смещения выборки и когнитивные искажения модели
- рассчитать общий индекс качества симуляции по четырём компонентам с весом 25%
- построить четырёхуровневый цикл валидации от логики до предсказаний
Примеры запросов агенту
- /simulation-calibrator откалибруй финансовую модель запуска нового продукта по квартальным отчётам
- /simulation-calibrator проверь симуляцию нагрузки бэкенда на линейные искажения и перекос выборки
- /simulation-calibrator настрой валидацию прогноза оттока клиентов с порогом точности 85%
Как установить simulation-calibrator
1Через маркетплейс плагинов
/plugin marketplace add davepoon/buildwithclaude/plugin install all-commands@buildwithclaude2Вручную: положить файлы
.claude/commands/simulation-calibrator.mdВызов — /simulation-calibrator. Для всех проектов — ~/.claude/commands/
Вручную: положить файлы
~/.codex/prompts/simulation-calibrator.mdВызов — /prompts:simulation-calibrator. В Codex этот формат помечен устаревшим, замена — скиллы
Вручную: положить файлы
.github/prompts/simulation-calibrator.prompt.mdВручную: положить файлы
.gemini/commands/simulation-calibrator.tomlGemini хранит команды в TOML: текст запроса — в поле prompt
Вопросы о simulation-calibrator
Как установить simulation-calibrator в Claude Code?
Сначала добавьте каталог маркетплейса /plugin marketplace add davepoon/buildwithclaude, затем установите плагин /plugin install all-commands@buildwithclaude. После этого команда станет доступна через файл .claude/commands/simulation-calibrator.md.
Что делает simulation-calibrator?
Команда структурирует тестирование симуляционных моделей. Она задаёт критерии проверки, находит систематические ошибки в логике или выборке и рассчитывает итоговый балл качества.
Какие требования к точности задаёт команда?
Шкала зависит от цены ошибки: для критически важных решений требуется точность от 95%, для стратегического планирования — 80–95%, для оптимизации процессов — 70–80%, а для исследовательского анализа достаточно 50–70%.
Исходный текст
simulation-calibrator.md · MIT
--- description: Test and refine simulation accuracy with validation loops, bias detection, and continuous improvement frameworks. category: simulation-modeling argument-hint: "Specify calibration parameters" --- # Simulation Calibrator Test and refine simulation accuracy with validation loops, bias detection, and continuous improvement frameworks. ## Instructions You are tasked with systematically calibrating simulations to ensure accuracy, reliability, and actionable insights. Follow this approach: **$ARGUMENTS** ### 1. Prerequisites Assessment **Critical Calibration Context Validation:** - **Simulation Type**: What kind of simulation are you calibrating? - **Accuracy Requirements**: How precise does the simulation need to be? - **Validation Data**: What real-world data can test simulation accuracy? - **Decision Stakes**: How important are the decisions based on this simulation? - **Update Frequency**: How often should calibration be performed? **If context is unclear, guide systematically:** ``` Missing Simulation Context: "What type of simulation needs calibration? - Business Simulations: Market response, financial projections, strategic scenarios - Technical Simulations: System performance, architecture behavior, scaling predictions - Process Simulations: Operational workflows, resource allocation, timeline predictions - Behavioral Simulations: Customer behavior, team dynamics, adoption patterns Each simulation type requires different calibration approaches and validation methods." Missing Accuracy Requirements: "How accurate does your simulation need to be for effective decision-making? - Mission Critical (95%+ accuracy): Safety, financial, or legal decisions - Strategic Planning (80-95% accuracy): Investment, expansion, or major initiative decisions - Operational Optimization (70-80% accuracy): Process improvement and resource allocation - Exploratory Analysis (50-70% accuracy): Option generation and conceptual understanding" ``` ### 2. Baseline Accuracy Assessment **Establish current simulation performance levels:** #### Historical Validation Framework ``` Simulation Accuracy Baseline: Back-Testing Analysis: - Compare simulation predictions to known historical outcomes - Measure prediction accuracy across different time horizons - Identify systematic biases and error patterns - Assess prediction confidence calibration Accuracy Metrics: - Overall Prediction Accuracy: [percentage of correct predictions] - Directional Accuracy: [percentage of correct trend predictions] - Magnitude Accuracy: [percentage of predictions within acceptable error range] - Timing Accuracy: [percentage of events predicted within correct timeframe] - Confidence Calibration: [alignment between prediction confidence and actual accuracy] Error Pattern Analysis: - Systematic Biases: [consistent over/under-estimation patterns] - Context Dependencies: [accuracy variations by scenario type or conditions] - Time Horizon Effects: [accuracy changes over different prediction periods] - Complexity Correlation: [accuracy relationship to scenario complexity] ``` #### Simulation Quality Scoring ``` Quality Assessment Framework: Input Quality (25% weight): - Data completeness and accuracy - Assumption validation and documentation - Expert input quality and consensus - Historical precedent availability Model Quality (25% weight): - Algorithm sophistication and appropriateness - Relationship modeling accuracy and completeness - Constraint modeling and boundary definition - Uncertainty quantification and propagation Process Quality (25% weight): - Systematic methodology application - Bias detection and mitigation - Stakeholder validation and feedback integration - Documentation and reproducibility Output Quality (25% weight): - Prediction accuracy and reliability - Insight actionability and clarity - Decision support effectiveness - Communication and presentation quality Overall Simulation Quality Score = Sum of weighted component scores ``` ### 3. Systematic Bias Detection **Identify and correct simulation biases:** #### Bias Identification Framework ``` Common Simulation Biases: Cognitive Biases: - Confirmation Bias: Seeking information that supports expected outcomes - Anchoring Bias: Over-relying on first estimates or reference points - Availability Bias: Overweighting easily recalled or recent examples - Optimism Bias: Systematic overestimation of positive outcomes - Planning Fallacy: Underestimating time and resource requirements Data Biases: - Selection Bias: Non-representative data samples - Survivorship Bias: Only analyzing successful cases - Recency Bias: Overweighting recent data points - Historical Bias: Assuming past patterns will continue unchanged - Measurement Bias: Systematic errors in data collection Model Biases: - Complexity Bias: Over-simplifying or over-complicating models - Linear Bias: Assuming linear relationships where non-linear exist - Static Bias: Not accounting for dynamic system changes - Independence Bias: Ignoring correlation and interaction effects - Boundary Bias: Incorrect system boundary definition ``` #### Bias Mitigation Strategies ``` Systematic Bias Correction: Process-Based Mitigation: - Multiple perspective integration and diverse expert consultation - Red team analysis and devil's advocate approaches - Assumption challenging and alternative hypothesis testing - Structured decision-making and bias-aware processes Data-Based Mitigation: - Multiple data source integration and cross-validation - Out-of-sample testing and validation dataset use - Temporal validation across different time periods - Segment validation across different contexts and conditions Model-Based Mitigation: - Ensemble modeling and multiple algorithm approaches - Sensitivity analysis and robust parameter testing - Cross-validation and bootstrap sampling - Bayesian updating and continuous learning integration ``` ### 4. Validation Loop Design **Create systematic accuracy improvement processes:** #### Multi-Level Validation Framework ``` Comprehensive Validation Approach: Level 1: Internal Consistency Validation - Logical consistency checking and constraint satisfaction - Mathematical relationship verification and balance testing - Scenario coherence and narrative consistency - Assumption compatibility and interaction validation Level 2: Expert Validation - Domain expert review and credibility assessment - Stakeholder feedback and perspective integration - Peer review and professional validation - External advisor consultation and critique Level 3: Empirical Validation - Historical data comparison and pattern matching - Market research validation and customer feedback - Pilot testing and proof-of-concept validation - Real-world experiment and A/B testing Level 4: Predictive Validation - Forward-looking accuracy testing and prediction tracking - Real-time outcome monitoring and comparison - Continuous feedback integration and model updating - Long-term performance assessment and trend analysis ``` #### Feedback Integration Mechanisms - Automated accuracy tracking and alert systems - Stakeholder feedback collection and analysis - Expert consultation and validation scheduling - Real-world outcome monitoring and comparison ### 5. Real-Time Calibration Systems **Establish ongoing accuracy monitoring and adjustment:** #### Continuous Monitoring Framework ``` Real-Time Calibration Dashboard: Accuracy Tracking Metrics: - Current Prediction Accuracy: [real-time accuracy percentage] - Accuracy Trend: [improving, stable, or declining accuracy] - Bias Detection: [systematic error patterns identified] - Confidence Calibration: [prediction confidence vs. actual accuracy alignment] Early Warning Indicators: - Prediction Deviation Alerts: [when predictions diverge significantly from reality] - Model Drift Detection: [when model performance degrades over time] - Assumption Violation Warnings: [when key assumptions prove incorrect] - Data Quality Alerts: [when input data quality degrades] Automated Adjustments: - Parameter Recalibration: [automatic model parameter updates] - Weight Rebalancing: [factor importance adjustments based on performance] - Threshold Updates: [decision threshold modifications based on accuracy] - Alert Sensitivity: [notification threshold adjustments] ``` #### Adaptive Learning Integration - Machine learning model updates based on new data - Bayesian updating for probability and parameter estimation - Expert feedback integration and model refinement - Context-aware calibration for different scenario types ### 6. Calibration Quality Assurance **Ensure systematic improvement and reliability:** #### Calibration Validation Framework ``` Meta-Calibration Assessment: Calibration Process Quality: - Validation methodology appropriateness and rigor - Feedback integration effectiveness and speed - Bias detection and mitigation success - Continuous improvement demonstration Calibration Outcome Quality: - Accuracy improvement measurement and tracking - Prediction reliability enhancement - Decision support effectiveness improvement - Stakeholder confidence and satisfaction growth Calibration Sustainability: - Process scalability and resource efficiency - Knowledge capture and institutional learning - Methodology transferability to other simulations - Long-term performance maintenance and enhancement ``` #### Quality Control Mechanisms - Independent calibration validation and audit - Cross-functional calibration team and review processes - External benchmark comparison and best practice integration - Documentation and knowledge management systems ### 7. Simulation Improvement Roadmap **Generate systematic enhancement strategies:** #### Calibration-Based Improvement Plan ``` Simulation Enhancement Framework: ## Simulation Calibration Analysis: [Simulation Name] ### Current Performance Assessment - Baseline Accuracy: [current accuracy percentages] - Key Biases Identified: [systematic errors found] - Validation Coverage: [validation methods applied] - Stakeholder Confidence: [user trust and satisfaction levels] ### Calibration Findings #### Accuracy Analysis: - Strong Performance Areas: [where simulation excels] - Accuracy Gaps: [where improvements are needed] - Bias Patterns: [systematic errors identified] - Validation Results: [validation testing outcomes] #### Improvement Opportunities: - Quick Wins: [immediate accuracy improvements available] - Strategic Enhancements: [longer-term improvement possibilities] - Data Quality Improvements: [input enhancement opportunities] - Model Sophistication: [algorithm and methodology upgrades] ### Improvement Roadmap #### Phase 1: Immediate Fixes (30 days) - Critical bias corrections and parameter adjustments - Data quality improvements and source validation - Process enhancement and workflow optimization - Stakeholder feedback integration and communication #### Phase 2: Systematic Enhancement (90 days) - Model sophistication and algorithm upgrades - Validation framework expansion and automation - Feedback loop optimization and real-time calibration - Training and capability building for users #### Phase 3: Advanced Optimization (180+ days) - Machine learning integration and automated improvement - Cross-simulation learning and best practice sharing - Innovation and methodology advancement - Strategic capability building and competitive advantage ### Success Metrics and Monitoring - Accuracy Improvement Targets: [specific goals and timelines] - Bias Reduction Objectives: [systematic error elimination goals] - Validation Coverage Goals: [comprehensive validation targets] - User Satisfaction Improvements: [stakeholder confidence goals] ``` ### 8. Knowledge Capture and Transfer **Establish institutional learning from calibration:** #### Learning Documentation - Calibration methodology documentation and best practices - Bias detection and mitigation technique libraries - Validation approach templates and reusable frameworks - Success pattern identification and replication guides #### Cross-Simulation Learning - Calibration insight sharing across different simulations - Best practice identification and standardization - Common pitfall documentation and avoidance strategies - Expertise development and capability building programs ## Usage Examples ```bash # Business simulation calibration /simulation:simulation-calibrator Calibrate customer acquisition cost simulation using 12 months of actual campaign data # Technical simulation validation /simulation:simulation-calibrator Validate system performance simulation against production monitoring data and user experience metrics # Market response calibration /simulation:simulation-calibrator Calibrate market response model using A/B testing results and customer behavior analytics # Strategic scenario validation /simulation:simulation-calibrator Test business scenario accuracy using post-decision outcome analysis and market development tracking ``` ## Quality Indicators - **Green**: Systematic validation, documented biases, automated monitoring, continuous improvement - **Yellow**: Regular validation, some bias detection, manual monitoring, periodic improvement - **Red**: Ad-hoc validation, undetected biases, no monitoring, no improvement tracking ## Common Pitfalls to Avoid - Validation theater: Going through validation motions without learning - Bias blindness: Not recognizing systematic errors and prejudices - Static calibration: Not updating models based on new information - Perfection paralysis: Waiting for perfect accuracy before using insights - Context ignorance: Not adapting calibration to different scenarios - Learning isolation: Not sharing insights across teams and simulations Transform simulation accuracy from guesswork into systematic, reliable decision support through comprehensive calibration and continuous improvement.
© BuildWithClaude Community · MIT · davepoon/buildwithclaude
Похожие слэш-команды
Все слэш-команды по теме «ИИ и ML» →Сначала — simulation-calibrator от других авторов.