Skip to main content
skill CX Codex

adf-validation-rules — Claude Code skill: setup

The adf-validation-rules skill checks Azure Data Factory activity nesting, pipeline limits, and common validation traps. Use it while designing or debugging ADF pipelines.

MIT 168 installs ★ 66 updated 2026-06-18

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add JosiahSiegel/claude-plugin-marketplace --skill adf-validation-rules -a claude-code
Open on GitHub

What adf-validation-rules does

The skill explains how ForEach, If Condition, Until, and Switch can be arranged in Azure Data Factory. It flags unsupported parent-child combinations and points to a child pipeline called through Execute Pipeline when direct nesting will not work.

Its reference covers activity and parameter caps, ForEach iterations and item counts, Lookup limits, and activity timeouts. It also covers linked services for Azure Blob Storage and Azure SQL, plus selected Data Flow constraints. A parallel ForEach cannot contain Set Variable, since that activity writes to pipeline-scoped variables.

What it helps with

  • check whether an ADF activity nesting pattern is allowed
  • trace a pipeline validation or ARM-template build error
  • refactor unsupported nesting through a child pipeline
  • verify ForEach, Lookup, and parameter limits
  • review linked service and Data Flow constraints

Example prompts for the agent

  • Check whether ForEach can be placed inside If Condition in this pipeline.
  • Trace the ARM-template validation error and redesign the flow with Execute Pipeline.
  • Verify whether Set Variable is valid inside a parallel ForEach using batchCount: 50.

How to install adf-validation-rules

1Skills CLI (Vercel)

npx skills add JosiahSiegel/claude-plugin-marketplace --skill adf-validation-rules -a claude-code

2Manually: put the files in place

~/.claude/skills/adf-validation-rules/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/adf-validation-rules/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about adf-validation-rules

How do I install adf-validation-rules in Claude Code?

Run npx skills add JosiahSiegel/claude-plugin-marketplace --skill adf-validation-rules -a claude-code, then open ~/.claude/skills/adf-validation-rules/SKILL.md.

What does adf-validation-rules do?

It checks Azure Data Factory nesting rules, resource limits, and common pipeline validation failures. For unsupported nesting, it recommends separating the logic into a child pipeline invoked with Execute Pipeline.

Can Set Variable run inside a parallel ForEach?

No. With isSequential: false, Set Variable is disallowed because the variable scope belongs to the pipeline rather than an individual iteration.

Source text

SKILL.md · MIT
---
name: adf-validation-rules
description: |
  ADF activity nesting rules, limits, and validation gotchas.
  PROACTIVELY activate for: (1) ADF activity nesting rules (ForEach inside If, Switch, Until), (2) ForEach limitations (no nested ForEach, item limits), (3) pipeline validation errors and ARM-template build failures, (4) linked service authentication issues, (5) Set Variable restrictions in ForEach, (6) Data Flow constraints (transformation limits, source/sink rules), (7) parameter and variable scope, (8) max activities per pipeline, (9) pipeline concurrency limits.
  Provides: nesting rule matrix, activity limit reference, validation error catalog, and refactoring patterns for unsupported nesting.
---

# Azure Data Factory Validation Rules and Limitations

## 🚨 CRITICAL: Activity Nesting Limitations

Azure Data Factory has **STRICT** nesting rules for control flow activities. Violating these rules will cause pipeline failures or prevent pipeline creation.

### Supported Control Flow Activities for Nesting

Four control flow activities support nested activities:
- **ForEach**: Iterates over collections and executes activities in a loop
- **If Condition**: Branches based on true/false evaluation
- **Until**: Implements do-until loops with timeout options
- **Switch**: Evaluates activities matching case conditions

### ✅ PERMITTED Nesting Combinations

| Parent Activity | Can Contain | Notes |
|----------------|-------------|-------|
| **ForEach** | If Condition | ✅ Allowed |
| **ForEach** | Switch | ✅ Allowed |
| **Until** | If Condition | ✅ Allowed |
| **Until** | Switch | ✅ Allowed |

### ❌ PROHIBITED Nesting Combinations

| Parent Activity | CANNOT Contain | Reason |
|----------------|----------------|---------|
| **If Condition** | ForEach | ❌ Not supported - use Execute Pipeline workaround |
| **If Condition** | Switch | ❌ Not supported - use Execute Pipeline workaround |
| **If Condition** | Until | ❌ Not supported - use Execute Pipeline workaround |
| **If Condition** | Another If | ❌ Cannot nest If within If |
| **Switch** | ForEach | ❌ Not supported - use Execute Pipeline workaround |
| **Switch** | If Condition | ❌ Not supported - use Execute Pipeline workaround |
| **Switch** | Until | ❌ Not supported - use Execute Pipeline workaround |
| **Switch** | Another Switch | ❌ Cannot nest Switch within Switch |
| **ForEach** | Another ForEach | ❌ Single level only - use Execute Pipeline workaround |
| **Until** | Another Until | ❌ Single level only - use Execute Pipeline workaround |
| **ForEach** | Until | ❌ Single level only - use Execute Pipeline workaround |
| **Until** | ForEach | ❌ Single level only - use Execute Pipeline workaround |

### 🚫 Special Activity Restrictions

**Validation Activity**:
- ❌ **CANNOT** be placed inside ANY nested activity
- ❌ **CANNOT** be used within ForEach, If, Switch, or Until activities
- ✅ Must be at pipeline root level only

### 🔧 Workaround: Execute Pipeline Pattern

**The ONLY supported workaround for prohibited nesting combinations:**

Instead of direct nesting, use the **Execute Pipeline Activity** to call a child pipeline:

```json
{
  "name": "ParentPipeline_WithIfCondition",
  "activities": [
    {
      "name": "IfCondition_Parent",
      "type": "IfCondition",
      "typeProperties": {
        "expression": "@equals(pipeline().parameters.ProcessData, 'true')",
        "ifTrueActivities": [
          {
            "name": "ExecuteChildPipeline_WithForEach",
            "type": "ExecutePipeline",
            "typeProperties": {
              "pipeline": {
                "referenceName": "ChildPipeline_ForEachLoop",
                "type": "PipelineReference"
              },
              "parameters": {
                "ItemList": "@pipeline().parameters.Items"
              }
            }
          }
        ]
      }
    }
  ]
}
```

**Child Pipeline Structure:**
```json
{
  "name": "ChildPipeline_ForEachLoop",
  "parameters": {
    "ItemList": {"type": "array"}
  },
  "activities": [
    {
      "name": "ForEach_InChildPipeline",
      "type": "ForEach",
      "typeProperties": {
        "items": "@pipeline().parameters.ItemList",
        "activities": [
          // Your ForEach logic here
        ]
      }
    }
  ]
}
```

**Why This Works:**
- Each pipeline can have ONE level of nesting
- Execute Pipeline creates a new pipeline context
- Child pipeline gets its own nesting level allowance
- Enables unlimited depth through pipeline chaining

## 🔢 Activity and Resource Limits

### Pipeline Limits
| Resource | Limit | Notes |
|----------|-------|-------|
| **Activities per pipeline** | 80 | Includes inner activities for containers |
| **Parameters per pipeline** | 50 | - |
| **ForEach concurrent iterations** | 50 (maximum) | Set via `batchCount` property |
| **ForEach items** | 100,000 | - |
| **Lookup activity rows** | 5,000 | Maximum rows returned |
| **Lookup activity size** | 4 MB | Maximum size of returned data |
| **Web activity timeout** | 1 hour | Default timeout for Web activities |
| **Copy activity timeout** | 7 days | Maximum execution time |

### ForEach Activity Configuration
```json
{
  "name": "ForEachActivity",
  "type": "ForEach",
  "typeProperties": {
    "items": "@pipeline().parameters.ItemList",
    "isSequential": false,  // false = parallel execution
    "batchCount": 50,       // Max 50 concurrent iterations
    "activities": [
      // Nested activities
    ]
  }
}
```

**Critical Considerations:**
- `isSequential: true` → Executes one item at a time (slow but predictable)
- `isSequential: false` → Executes up to `batchCount` items in parallel
- Maximum `batchCount` is **50** regardless of setting
- **Cannot use Set Variable activity** inside parallel ForEach (variable scope is pipeline-level)

### Set Variable Activity Limitations
❌ **CANNOT** use `Set Variable` inside ForEach with `isSequential: false`
- Reason: Variables are pipeline-scoped, not ForEach-scoped
- Multiple parallel iterations would cause race conditions
- ✅ **Alternative**: Use `Append Variable` with array type, or use sequential execution

## 📊 Linked Services Validation (Azure Blob, Azure SQL)

Detailed validation rules and templates for ADF Linked Services (Azure Blob Storage and Azure SQL Database) — auth types (key, SAS, managed identity, AAD), network configuration, and connection-string patterns — live in `references/linked-services.md`. Load that reference when authoring or validating a Linked Service JSON.

## 🔍 Data Flow Limitations

### General Limits
- **Column name length**: 128 characters maximum
- **Row size**: 1 MB maximum (some sinks like SQL have lower limits)
- **String column size**: Varies by sink (SQL: 8000 for varchar, 4000 for nvarchar)

### Transformation-Specific Limits
| Transformation | Limitation |
|----------------|------------|
| **Lookup** | Cache size limited by cluster memory |
| **Join** | Large joins may cause memory errors |
| **Pivot** | Maximum 10,000 unique values |
| **Window** | Requires partitioning for large datasets |

### Performance Considerations
- **Partitioning**: Always partition large datasets before transformations
- **Broadcast**: Use broadcast hint for small dimension tables
- **Sink optimization**: Enable table option "Recreate" instead of "Truncate" for better performance

## 🛡️ Validation Checklist for Pipeline Creation

### Before Creating Pipeline
- [ ] Verify activity nesting follows permitted combinations
- [ ] Check ForEach activities don't contain other ForEach/Until
- [ ] Verify If/Switch activities don't contain ForEach/Until/If/Switch
- [ ] Ensure Validation activities are at pipeline root level only
- [ ] Confirm total activities < 80 per pipeline
- [ ] Verify no Set Variable activities in parallel ForEach

### Linked Service Validation
- [ ] **Blob Storage**: If using managed identity/service principal, `accountKind` is set
- [ ] **SQL Database**: Authentication method matches security requirements
- [ ] **All services**: Secrets stored in Key Vault, not hardcoded
- [ ] **All services**: Firewall rules configured for integration runtime IPs
- [ ] **Network**: Private endpoints configured if using VNet integration

### Activity Configuration Validation
- [ ] **ForEach**: `batchCount` ≤ 50 if parallel execution
- [ ] **Lookup**: Query returns < 5000 rows and < 4 MB data
- [ ] **Copy**: DIU configured appropriately (2-256 for Azure IR)
- [ ] **Copy**: Staging enabled for large data movements
- [ ] **All activities**: Timeout values appropriate for expected execution time
- [ ] **All activities**: Retry logic configured for transient failures

### Data Flow Validation
- [ ] Column names ≤ 128 characters
- [ ] Source query doesn't return > 1 MB per row
- [ ] Partitioning configured for large datasets
- [ ] Sink has appropriate schema and data type mappings
- [ ] Staging linked service configured for optimal performance

## 🔍 Automated Validation Script

**CRITICAL: Always run automated validation before committing or deploying ADF pipelines!**

The adf-master plugin includes a comprehensive PowerShell validation script that checks for ALL the rules and limitations documented above.

### Using the Validation Script

**Location:** `${CLAUDE_PLUGIN_ROOT}/scripts/validate-adf-pipelines.ps1`

**Basic usage:**
```powershell
# From the root of your ADF repository
pwsh -File validate-adf-pipelines.ps1
```

**With custom paths:**
```powershell
pwsh -File validate-adf-pipelines.ps1 `
    -PipelinePath "path/to/pipeline" `
    -DatasetPath "path/to/dataset"
```

**With strict mode (additional warnings):**
```powershell
pwsh -File validate-adf-pipelines.ps1 -Strict
```

### What the Script Validates

The automated validation script checks for issues that Microsoft's official `@microsoft/azure-data-factory-utilities` package does **NOT** validate:

1. **Activity Nesting Violations:**
   - ForEach → ForEach, Until, Validation
   - Until → Until, ForEach, Validation
   - IfCondition → ForEach, If, IfCondition, Switch, Until, Validation
   - Switch → ForEach, If, IfCondition, Switch, Until, Validation

2. **Resource Limits:**
   - Pipeline activity count (max 120, warn at 100)
   - Pipeline parameter count (max 50)
   - Pipeline variable count (max 50)
   - ForEach batchCount limit (max 50, warn at 30 in strict mode)

3. **Variable Scope Violations:**
   - SetVariable in parallel ForEach (causes race conditions)
   - Proper AppendVariable vs SetVariable usage

4. **Dataset Configuration Issues:**
   - Missing fileName or wildcardFileName for file-based datasets
   - AzureBlobFSLocation missing required fileSystem property
   - Missing required properties for DelimitedText, Json, Parquet types

5. **Copy Activity Validations:**
   - Source/sink type compatibility with dataset types
   - Lookup activity firstRowOnly=false warnings (5000 row/4MB limits)
   - Blob file dependencies (additionalColumns logging pattern)

### Integration with CI/CD

**GitHub Actions example:**
```yaml
- name: Validate ADF Pipelines
  run: |
    pwsh -File validate-adf-pipelines.ps1 -PipelinePath pipeline -DatasetPath dataset
  shell: pwsh
```

**Azure DevOps example:**
```yaml
- task: PowerShell@2
  displayName: 'Validate ADF Pipelines'
  inputs:
    filePath: 'validate-adf-pipelines.ps1'
    arguments: '-PipelinePath pipeline -DatasetPath dataset'
    pwsh: true
```

### Command Reference

Use the `/adf-validate` command to run the validation script with proper guidance:

```bash
/adf-validate
```

This command will:
1. Detect your ADF repository structure
2. Run the validation script with appropriate paths
3. Parse and explain any errors or warnings found
4. Provide specific solutions for each violation
5. Recommend next actions based on results
6. Suggest CI/CD integration patterns

### Exit Codes

- **0**: Validation passed (no errors)
- **1**: Validation failed (errors found - DO NOT DEPLOY)

### Best Practices

1. **Run validation before every commit** to catch issues early
2. **Add validation to CI/CD pipeline** to prevent invalid deployments
3. **Use strict mode during development** for additional warnings
4. **Re-validate after bulk changes** or generated pipelines
5. **Document validation exceptions** if you must bypass a warning
6. **Share validation results with team** to prevent repeated mistakes

## 🚨 CRITICAL: Enforcement Protocol

**When creating or modifying ADF pipelines:**

1. **ALWAYS validate activity nesting** against the permitted/prohibited table
2. **REJECT** any attempt to create prohibited nesting combinations
3. **SUGGEST** Execute Pipeline workaround for complex nesting needs
4. **VALIDATE** linked service authentication matches the connector type
5. **CHECK** all limits (activities, parameters, ForEach iterations, etc.)
6. **VERIFY** required properties are set (e.g., `accountKind` for managed identity)
7. **WARN** about common pitfalls specific to the connector being used

**Example Validation Response:**
```yaml
❌ INVALID PIPELINE STRUCTURE DETECTED:

Issue: ForEach activity contains another ForEach activity
Location: Pipeline "PL_DataProcessing" → ForEach "OuterLoop" → ForEach "InnerLoop"

This violates Azure Data Factory nesting rules:
- ForEach activities support only a SINGLE level of nesting
- You CANNOT nest ForEach within ForEach

✅ RECOMMENDED SOLUTION:
Use the Execute Pipeline pattern:
1. Create a child pipeline with the inner ForEach logic
2. Replace the inner ForEach with an Execute Pipeline activity
3. Pass required parameters to the child pipeline

Would you like me to generate the refactored pipeline structure?
```

## 📚 Reference Documentation

**Official Microsoft Learn Resources:**
- Activity nesting: https://learn.microsoft.com/en-us/azure/data-factory/concepts-nested-activities
- Blob Storage connector: https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-blob-storage
- SQL Database connector: https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-sql-database
- Pipeline limits: https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits#data-factory-limits

**Last Updated:** 2025-01-24 (Based on official Microsoft documentation)

This validation rules skill MUST be consulted before creating or modifying ANY Azure Data Factory pipeline to ensure compliance with platform limitations and best practices.

## Progressive Disclosure References

For detailed validation matrices and resource limits, see:

- **Nesting Rules**: `references/nesting-rules.md` - Complete matrix of permitted and prohibited activity nesting combinations with workaround patterns
- **Resource Limits**: `references/resource-limits.md` - Complete reference for all ADF limits (pipeline, activity, trigger, data flow, integration runtime, expression, API)

© JosiahSiegel · MIT · JosiahSiegel/claude-plugin-marketplace

Similar skills

All skills for Cloud: AWS, Azure, GCP →
azure-pipelines-generator akin-ozer The azure-pipelines-generator skill writes azure-pipelines.yml for you: stages, jobs, steps and reusable templates. Useful when moving CI to Azure Pipelines or splitting a pipeline into templates. CX Codex +1 722installs azure-devops devops-skills Lets your agent write and fix Azure DevOps YAML pipelines: build, test, deploy to Azure. Useful when CI/CD lives in Azure Pipelines. CX Codex +1 360installs platform-validation-rule-generate forcedotcom platform-validation-rule-generate — a skill for building and editing Salesforce validation rules: it assembles the metadata, the formula and the error message. Useful when bad records must not be saved. CX Codex +1 6,400installs bim-validation-report datadrivenconstruction The bim-validation-report skill runs automated quality checks on BIM model elements, flagging missing attributes, bad naming, and geometric defects. It produces a structured compliance report directly in your agent workspace. CX Codex +1 133installs multi-agent-e2e-validation terrylica The multi-agent-e2e-validation skill for Claude Code and other AI editors spawns multiple validation agents in parallel to test database refactors comprehensively. CX Codex +1 128installs n8n-validation-expert czlonkowski n8n-validation-expert interprets n8n validation errors and warnings, identifies which block workflow execution and which are false positives. Explains why a node config fails validation and guides the fix in 2–3 iterations. CX Codex +1 7,300installs

From the same set96

The whole set →
azure-well-architected-framework azure-well-architected-framework reviews Azure cloud architectures against the Well-Architected Framework pillars using proven Azure CLI templates. CX Codex +1 174installs windows-git-bash-paths The windows-git-bash-paths skill helps handle Windows paths in Git Bash when working with SSDT, SqlPackage, and DACPAC files. Use it when Git Bash rewrites command arguments and SqlPackage receives the wrong path. CX Codex +1 173installs index-strategies index-strategies is a Claude Code skill that designs, reviews, and tunes SQL Server indexes, from clustered and nonclustered to filtered and columnstore. CX Codex +1 172installs databricks-2025 ADF and Databricks 2025 integration patterns, linked service templates, and updated connector recipes. CX Codex +1 168installs azure-openai-2025 azure-openai-2025 — a reference skill for Azure OpenAI Service: the 2025 model lineup, deployment through az, SKU and quota notes, Python SDK recipes. CX Codex +1 167installs github-actions-2025 github-actions-2025 guides your agent through modern GitHub Actions features, including 1 vCPU runners, immutable releases, and Node24 migrations. CX Codex +1 166installs defender-for-devops defender-for-devops is a Claude Code skill for adding security checks to Azure Pipelines. It activates for MSDO setup, code and dependency scans, container checks, and IaC analysis. CX Codex +1 166installs docker-watch-mode-2025 docker-watch-mode-2025 sets up Compose watch rules so source edits land in the running container, while dependency manifests trigger a rebuild. CX Codex +1 166installs

Project sponsors

Companies that keep this analytics open