troubleshooting-flows — Claude Code skill: what it does & setup

---
name: troubleshooting-flows
description: Diagnose and resolve Celigo flow failures -- total failures, partial errors, stuck jobs, empty runs, and performance issues. Use when a flow is failing, producing errors, returning no data, or running slowly.
---

<!-- TIER:1 -->

# Troubleshooting Flows

A flow is broken when it fails to move data correctly. This skill covers systematic diagnosis: identifying the problem type, isolating the failing step, inspecting errors, and resolving them.

Troubleshooting concerns:

- **Job status** -- understanding what `completed`, `failed`, `canceled`, and `retrying` mean for the flow
- **Error analysis** -- grouping errors by pattern to find root causes instead of reading them one-by-one
- **Request/response inspection** -- seeing exactly what was sent and returned at each step
- **Execution logs** -- record-level tracing through every stage of the pipeline
- **Retry and resolution** -- fixing error data and retrying vs bulk resolving
- **Delta/state issues** -- `lastExportDateTime` drift, stuck deltas, re-processing windows

## How to Run the Steps

This skill names each diagnostic step by what it reads or changes; the exact calls depend on how you reach the account. Read the one reference that matches your tools -- both follow the step numbers of the Diagnostic Workflow below:

- **Celigo MCP tools** (`list_flow_runs`, `list_flow_errors`, `triage_flow_errors`, ...) -- [references/mcp.md](references/mcp.md)
- **The `celigo` CLI** -- [references/cli.md](references/cli.md)

Diagnosis is read-only. Retrying errors, resolving them, running a flow, and changing configuration all write to the account or to the connected systems -- do them only when the user asked for the fix, not as part of finding the cause.

## Read Before You Answer

A diagnosis rests on what this account shows, not on what usually goes wrong. Work through these in order and stop only when the evidence names a cause:

1. **Pin down what the user means.** Which flow or integration, and which symptom: a failed run, some records failing, nothing exported, a stuck or late run, wrong data with no errors, or "it used to work". For "the most recent error", compare the errors' times, not which step has the most errors. If the user only says things are slow and no flow's recent runs are slower than its own history, ask one question -- which flow, run, screen or call is slow -- rather than tuning anything or reporting that nothing is slow.
2. **Read the run.** The latest run's status and each step's counts (success, error, ignore, pages). They name the [problem category](#problem-categories) and the step to look at.
3. **Read the errors themselves.** Open the failing step's errors: message, code, `source`, `classification`, and the values one or two failing records carry (their retry data). Group them by message. Quote what the errors say -- "the lookup has no entry for 'Local Pickup'", not "a lookup probably failed".
4. **Read the configuration the errors point at.** The mapping row, lookup, request body or URI, filter, query, hook or connection the error names. A cause has to explain why these records failed while others on the same step succeeded. Some settings act only in one mode -- an export's `once` callback only when its `type` is `once`, its `{{lastExportDateTime}}` tokens only when it is `delta` -- so read the `type` before naming such a setting as the safeguard or the cause. When the user calls it a platform bug, read the configuration before agreeing or filing anything.
5. **If it used to work, read what changed.** The audit log of the step, the flow and the step's connection, just before the first error or slowdown: a connection change (concurrency, auth, auto-recover) explains a failure or a slowdown as often as a step change.
6. **Before advising a retry or a re-run, say what it will do.** A retry runs each failed record's saved data through the step as the step is now, so it fails the same way until the configuration or that data is fixed. A re-run after a canceled or failed run reads the same window again, so a create-only import writes again the records it already wrote. An empty delta run leaves the checkpoint where it was, so the first run after the fix picks those records up -- no backfill run is needed. When the user asks for a retry that would write records a second time -- errors a postSubmit hook raised after the destination accepted the records, or creates that timed out -- say so and ask before retrying.
7. **Answer with the evidence.** Name the cause as a specific field, value or setting, list the reads that show it, give the fix, and say what not to do yet. An ignored record leaves no error and no retry data, so once its cause is fixed it comes through only if the source sends it again. When some records fail (or are ignored) and others go through, say what those records carry that the others don't -- the value, character or missing field in their data. Change nothing the user didn't ask you to change: when they ask you to fix it, make the change; when they ask why or how to fix it, say what to change and offer to make it. Either way, a retry, a run or a resolve waits for their go-ahead.

## Problem Categories

### Total Failure

Job status is `failed` with 0 successful records. The entire run collapsed before processing any data. Typically `numPagesGenerated: 0` (export-level failure) or pages generated but `numPagesProcessed: 0` (import-level failure on the first page).

Common causes: connection failure (credentials expired, endpoint down), export query error (invalid SQL, bad saved search ID), missing/deleted resource, permission denied.

### Partial Failure

Job status is `completed` but `numError > 0` alongside successful records. Some records failed while others processed normally. The failed share varies widely, from a handful of records to a large part of the run.

Common causes: validation errors on the destination (required fields missing, type mismatches), duplicate key violations, record-level lookup failures, rate limiting on specific batches, data-dependent issues (specific records have bad data).

### Empty Run

Job completes successfully with 0 errors AND 0 records processed.

Common causes: wrong `resourcePath` on the export (extracts from wrong JSON path), delta export with no changes since last run (legitimate), output filter too restrictive (all records filtered out), source query returns no results, webhook export with no inbound events, or the export now reads another source -- its `_connectionId` was switched to a different connection (a dev or sandbox store, another account). The export's audit log shows the switch; its old value names the connection the export read before, the one to point it back to if the switch was a mistake.

On a file export, also check its name filters (`fileNameStartsWith`, `fileNameEndsWith`): a file whose name doesn't match is skipped silently, and the match is case-sensitive on most servers, so `ORDER_` skips `order_123.csv`. A filter changed in the export's audit log around when the runs went empty is the likely cause; its old value is the one to set back.

A delta run that exports nothing leaves the checkpoint where it was, so once the cause is fixed, the next run picks up the records the empty runs missed -- no backfill run is needed. An empty execution log proves nothing here: the log holds one entry per record, so a run that exported nothing has none at any log mode.

### Stuck or Long-Running

Job stays in `running` or `retrying` status longer than expected.

Common causes: large dataset with no pagination limits, destination system slow to respond, script hook with long-running logic, on-premise agent connectivity issues, rate limiting causing backoff.

**A run that got slower.** Compare the slow runs with earlier runs of the same flow ([Monitoring Lenses](#monitoring-lenses-and-execution-metrics)). In a slow run's family, each step's child job has its own `startedAt` and `endedAt`; the step whose child job ends long after the others is the one holding the run. Before blaming a change, compare that step's records per run with its time per record: more records at the same pace is volume, not a fault. Then check what sets that step's pace: its connection's concurrency (`targetConcurrencyLevel` while auto-recover is on) against what the target system allows, rate-limit errors on that connection, and whether the step calls the target once per record -- a lookup step does, as does an import's dynamic lookup -- where a lookup cache answers from stored values instead ([configuring-lookup-caches](../configuring-lookup-caches/SKILL.md)). Answer with those numbers for the slow run and a normal one: each step's time, its records per run and its time per record or per call. Recommend the change that shortens the slow step, and leave out tuning for the steps that aren't holding the run, such as a larger page size on an export that takes one minute of a twenty-minute run.

### Intermittent Failures

Flow sometimes succeeds and sometimes fails with the same configuration.

Common causes: token/session expiry mid-run (long-running flows), rate limiting (varies with concurrent flows), transient network errors, source system maintenance windows.

## Error Diagnosis Framework

### Classification

The platform stamps each error with a read-only `classification` -- read it from the error itself (every error in a step's error list carries it) rather than guessing from the message or status code. The classifier is heuristic, so identical messages can classify differently, and nothing you configure changes a classification. It decides whether the platform recovers on its own:

| Classification | Typical cause | Recovers on its own? | Action |
|---|---|---|---|
| `intermittent` | Timeouts, 5xx responses, dropped connections | Yes -- up to four automatic retries, the first about 30 minutes after the error, while the flow's `autoResolveMatchingTraceKeys` is on (the default). A `retryAt` on the error means it's still in auto-retry | Wait. Once attempts are exhausted (no `retryAt`) or the setting is off, retry manually when the target is healthy. A timed-out create may have been written anyway, and every retry, automatic or manual, sends it again: for a create-only import, check the destination or make the import skip records that already exist (configuring-imports) |
| `rate_limit` | HTTP 429, quota exceeded | Through the connection, not the error list -- with auto-recover on, rate-limited records are held while concurrency drops and climbs back, so the flow can look stuck with no new errors | Open `rate_limit` errors mean auto-recover is off or gave up: re-enable it or lower `targetConcurrencyLevel` (see configuring-connections), then retry |
| `connection` | Credentials, permissions, expired token, offline connection | No -- and these errors carry no retry data | Fix the connection (ping it, or read its `offline` flag), then re-run the flow |
| `value` | The destination rejected a value (format, status transition, locked field) | No | Fix the mapping (retrying an import's error runs the fixed import) or, when the record itself is bad, edit its retry data; then retry |
| `missing` | A required field or referenced record is missing | No | Supply the data or fix the lookup or mapping, then retry |
| `duplicate` | The destination already holds the record | No | Fix the match or upsert criteria; resolve the error if the existing record is right. A resolve drops the refused record, changes included: when they should reach the destination (an insert refused on a unique key), make the import update or upsert, then retry |
| `parse` | Malformed JSON or XML, Handlebars or script syntax -- or a template naming a field the record doesn't have (`"<field>" not defined in the model`), which is missing data, not syntax | No | Fix the template or script, then retry. For a field "not defined in the model", read the record's retry data first: fix the data, or the import's handling of records that lack it |
| `too_large` | Request, response, or page too big | No -- never auto-recovered | Reduce page or batch size, or trim the payload, then retry |
| (none) | Nothing classified with confidence | No | Read the error's `code` and `message` and its raw HTTP request/response, then investigate |

Status codes are hints, not the classification: a 429 usually lands as `rate_limit`, a 401 or 403 as `connection`, a timeout or 5xx often (not always) as `intermittent`. Don't retry `value`, `missing`, or `parse` errors before fixing the cause -- they fail the same way again. `governance` is a legacy value, since split into `rate_limit` and `too_large`.

### Root Cause: Configuration vs Data

Every flow error has one of two root causes:

- **Static configuration** -- a hardcoded value in the step config is wrong (mapping expression, filter rule, hardcoded field, URI, query, SQL statement). Fix: change the resource configuration.
- **Dynamic data** -- the upstream source sent unexpected data (missing required field, wrong type, null where a value is expected, unexpected array/object shape). Fix: add input filtering or validation upstream, or fix the source system

**To distinguish:** check if the error reproduces with different input records. If the same error occurs for every record, it's configuration. If only some records fail, it's data. Compare the step's error count with the run's total records, then sample several errors and compare their messages and the records they carry. A candidate cause has to explain why these records failed while others on the same step succeeded -- anything every record shares is ruled out. Retry data is the record as it entered the step, so its ids are the source's: identify the destination record from the request the step sent, not from those ids.

### Which Step Failed?

Error location determines which resource and skill to investigate. Each error's `source` says where in the step it was raised -- `application` (the step's call to the external system: usually the destination's answer, but also a request that never went out, such as a URL template that failed to render, so read the `code` with it), `connection`, `mapping`, a filter, or a hook:

| Error location | Resource to check | Skill |
|---|---|---|
| Export / page generator | Export config (connection, query, resourcePath) | configuring-exports |
| Import / page processor | Import config (mapping, destination fields, operation) | configuring-imports |
| Script hook (`pre_save_page_hook`, `pre_map_hook`, `post_map_hook`, `post_submit_hook`, `post_response_map_hook`, ...) | Script code | writing-scripts |
| SuiteScript hook (`pre_send_hook_ss`, `pre_map_hook_ss`, `post_map_hook_ss`, `post_submit_hook_ss`) | The NetSuite step's SuiteScript hook -- a script in NetSuite's file cabinet that runs inside NetSuite, not a Celigo script. A `post_submit_hook_ss` error with a NetSuite code and message is NetSuite's refusal that the hook reworded, as Integration Apps do: diagnose the NetSuite error | configuring-exports, configuring-imports, managing-integrations |
| Tool step (`tool_input`, `tool_output`) | The Tool's input or output schema -- the record entering the Tool, or its mapped output, didn't validate | building-tools |
| Mapping | Mapping expression (field paths, lookups, hardcoded values) | writing-mappings |
| Filter | Filter expression (s-expression syntax, field references) | configuring-filters |
| Connection | Connection config (auth, URL, credentials) | configuring-connections |

**Integration App steps.** If the failing step belongs to an Integration App (it carries `_connectorId`), find the app setting that governs the behavior before touching the step. The app locks parts of its steps -- a write to a locked field answers 200 but keeps the app's value -- and the app's own code runs only when its settings are saved through the app ([using-marketplace-templates > Changing an Installed Integration App](../using-marketplace-templates/SKILL.md#changing-an-installed-integration-app)). So don't write the integration's `settings` or the app's steps to fix it: save the setting with `celigo integrations persist-settings`, or, where no tool saves it (MCP has none), tell the user which setting to change, from what value to what, on the app's settings page. Only the account's own changes to the step are customer-side causes: in the step's audit log, `source: connector` entries are the app's own and `source: system` entries the platform's. If the account made none, the packaged configuration is the vendor's -- take it to the app's documentation or Celigo support. An app's errors often put its own sentence before the original message, and on NetSuite steps they can carry a SuiteScript hook's `source` ([Reading an App's Errors](../managing-integrations/references/integration-apps.md#reading-an-apps-errors)).

## Quick Reference

### Symptom --> First Look

| Symptom | Look at first | Then |
|---|---|---|
| Flow totally failed | The latest run's counts (step 1) | `numPagesGenerated: 0` means the export failed -- check its connection and query, not the import |
| Every step on a NetSuite connection fails with `Invalid login attempt`, and the connection is offline | The connection's account, role and token (`netsuite.account`, `roleName`, `tokenName`) -- name them in the answer | NetSuite refused the token and records why. A NetSuite administrator checks, in order: the token revoked or made inactive; the role taken off the user, made inactive or missing *Log in using Access Tokens*; the integration record set to Blocked; the token created in another account (a refreshed sandbox needs new tokens) -- [configuring-connections > Minimum Required Fields](../configuring-connections/SKILL.md#minimum-required-fields), "A NetSuite token that stops working" |
| Partial errors | Open-error counts per step (step 2) | Group that step's errors by message (step 3) to find the root-cause pattern |
| Empty run (0 records) | The latest run's counts (step 1) | Export config: `resourcePath`, delta state, output filter, a file export's name filters -- and the export's audit log, for a `_connectionId` switched to another store or account or a name filter changed ([Empty Run](#empty-run)) |
| Some records never arrived, and no errors | Each step's `numIgnore` (step 1) | An ignored record raises no error: the run's execution log lists it at the step that skipped it, with status `ignore` (step 6). A record that no router branch took isn't counted at all -- check the branches' filters or the routing script ([building-flows > Branching Flows](../building-flows/SKILL.md#branching-flows)). If the run's log holds nothing for the record (the run captured no per-record log), say so, give the step counts, trace the record in an event report over the run's window, and offer a debug window for the next run, saying how long it can stay open (step 6) |
| Stuck / long-running | The flow's in-progress runs | Triangulate before diagnosing -- see Stuck-Flow Triangulation |
| Nobody was told about a failed or slow run | The notification subscriptions and long-runtime alerts covering the flow | Neither lives on the flow -- see [Alerts and Notifications](#alerts-and-notifications) |
| A canceled run, before running the flow again | The run's status (`canceling` until it lands) and its import steps' `numSuccess` | A re-run reads the same window again, so records an import already wrote are written again -- see Stuck-Flow Triangulation |
| Scheduled runs not appearing, or "it didn't run at 7" | The flow's `timezone` (UTC when unset) and its latest runs' start times | Check in order -- see [Missed Scheduled Runs](#missed-scheduled-runs). Never rewrite a working schedule as the fix |
| Intermittent failures | The flow's run history | Compare failing vs passing runs; check token expiry and rate limits |
| A flow that used to work started failing | The audit log, for changes just before the first error | A flow's own audit log includes changes to its exports, imports, their connections and the scripts they use, but an audit query by one resource's id returns only that resource's changes -- then read each step's and connection's too. A change made before the step's first successful run is configuration still under test, not a regression |
| Errors that "went away" | Resolved-error history | Who closed each one (`resolvedBy`) and the retry jobs' counts -- zero open errors is not proof of recovery (see Error Lifecycle and Retries) |
| Errors came back after a retry | The flow's runs of type `retry`, not the original run | The retry job's own counts, and its start time against the fix's time in the step's audit log (see [Error Lifecycle and Retries](#error-lifecycle-and-retries)) |
| Every record of a GraphQL step fails the same way (HTTP 406 on Shopify) | The step's connection: which of the app's APIs it is bound to (`http._httpConnectorApiId`) and its base URI | A GraphQL step on a connection bound to the app's REST API posts to the REST root. Point the step at a connection that reaches the GraphQL endpoint, and only that step ([configuring-connections > Connection Types](../configuring-connections/SKILL.md#connection-types)) |
| "It created duplicates" | The runs around the time the user names -- a duplicate the destination accepted raised no error, so it shows in the successes, not the error list | Follow [building-flows > Duplicate Records](../building-flows/SKILL.md#duplicate-records) |
| A Data Ingestion sync failed | The sync's own jobs and their errors | A sync isn't a flow: its runs and errors never appear among flow runs or flow errors, and its errors can't be retried or resolved -- fix the cause and run the sync again |
| **Silent logic bug** (no errors, wrong output) | A test run (step 5) | Check what each stage does to the record (step 5); if the test run can't reach it, arm execution logging (step 6) and run for real |
| **Production incident** (real traffic matters) | Execution logging (step 6), then a run | Read per-record I/O; the failing stage names the problem |

## Related Skills

- **building-flows** ([Quick Reference](../building-flows/SKILL.md#quick-reference)) -- flow structure, topologies, and configuration
- **configuring-exports** ([Quick Reference](../configuring-exports/SKILL.md#quick-reference)) -- export configuration and adaptor types
- **configuring-imports** ([Quick Reference](../configuring-imports/SKILL.md#quick-reference)) -- import configuration and adaptor types
- **writing-scripts** ([Quick Reference](../writing-scripts/SKILL.md#quick-reference)) -- script hook debugging and data shapes

<!-- TIER:2 -->

## Diagnostic Workflow

### 1. Check the run status

Make sure it's the right flow first: a cloned flow is named `Clone - <name>` by default, so an account can hold several look-alikes. Diagnose the copy that is enabled, triggered (scheduled, chained, or listening for events) and has recent runs, and say which copy each finding is about.

Start with the most recent run of the flow (its parent job), then open the run to see its per-step child jobs -- one per export and import.

Key fields: `status`, `numError`, `numSuccess`, `numIgnore`, `numPagesGenerated`, `numPagesProcessed`, `startedAt`, `endedAt`. The run's own counts add up every step, lookups included -- one record that passes an export and an import counts twice -- so to know how many records a step wrote, failed or ignored, read that step's child job. A `failed` status with `numPagesGenerated: 0` means the export itself failed -- don't look at import errors. `completed` with `numError > 0` means partial failure at the record level.

### 2. Get the per-step error counts

See which steps have open errors and how many. Focus on the step with the most errors first -- unless the question is about the latest failure: each step's entry also carries `lastErrorAt`, the time of its newest error, so compare those rather than the counts.

### 3. Analyze error patterns

Group the step's errors by message to find the root cause instead of reading them one-by-one. If most errors share the same message, that's your root cause. Multiple distinct patterns may indicate multiple issues. Group a large enough sample: a page of 100 can hide a second pattern in a step with thousands of errors.

### 4. Inspect individual errors

Once you know the pattern, look at specific records: each error's message and code, the record data the step failed on (its retry data), and the HTTP request/response that produced the error. The raw request/response is the most powerful diagnostic -- it shows exactly what was sent and what the destination answered. Only errors from HTTP-based steps have one (they carry a `reqAndResKey`); a NetSuite distributed import's record errors, for example, don't. For an error from a NetSuite step, [Diagnosing NetSuite Steps](../configuring-exports/references/netsuite.md) maps NetSuite's codes to their causes and lists the checks to run against the account.

### 5. Use test runs for safe iteration (try this first)

Test runs process one page from one generator and never send imports, so they're the safest way to see what every stage does, and they don't advance the delta timestamp. They aren't side-effect free, though (below). The flow may be enabled or disabled -- an enabled flow's test run answers 200 and runs (live-verified 2026-10-14) -- so don't disable a live flow to test it: that stops its schedule and listeners for nothing. A test run reports the stages each step went through; with per-stage results you can read every stage's `{name, input, output, errors}`. An Integration App's flow (`_connectorId` set) can't be test-run: the platform answers 422 `not_allowed` ("Debug flow is not allowed for integration apps."), so check a change there by reading the resource back and following its next real run (step 6).

Test runs don't advance the delta timestamp -- you can repeat them safely against the same data.

What each way of running something touches:

| Operation | Source systems | Destinations |
|---|---|---|
| A test run of a flow (or of a tool or API) | Read live, unless the export has a saved `mockOutput` -- then it's replayed | Not written. Most imports aren't called at all -- each returns its saved `mockResponse`, or the placeholder `{"id": "1234567890", "mockResponse": "replace me"}`; a NetSuite DA import still sends the page to NetSuite, which builds each record without saving it ([configuring-imports > Testing an Import Without Writing](../configuring-imports/SKILL.md#testing-an-import-without-writing)) |
| Invoking an export on its own | Read live -- except a preview of an export document that still carries `mockOutput` (as one read from the API does), which returns that snapshot instead; invoking the saved export by id reads live | -- |
| Invoking an import on its own | -- | **Written** -- no job is created, but the request is sent |
| A real run of the flow | Read live | **Written** |
| Retrying errors | Replays each error's saved retry data | **Written** |

Hooks execute in all of these, and nothing sandboxes the `integrator-api` module in test mode: treat a hook's writes -- to a lookup cache or state, or an import or flow it runs -- as real. In a test run an import's `response` and `parse` stages show the mock, not the destination's answer -- when that answer matters, escalate to step 6. To make downstream steps testable, save a realistic `mockResponse` on the import ([configuring-imports > Testing an Import Without Writing](../configuring-imports/SKILL.md#testing-an-import-without-writing)). A replayed `mockOutput` looks live too: the export still shows `request` and `response` stages, the `response` holding the mock under the source's URL, and the test run's HTTP log records a 200. Those stages don't prove the source was read -- check the export for a saved `mockOutput` first.

**No errors, wrong output: check what each stage does to the record.** A Transformation 1.0 transform, or Transformation 2.0 in `create` mode, rebuilds the record from only the fields it maps, so a later filter, router or template that reads a dropped field silently stops matching (`modify` keeps the rest). A filter only passes or skips a record, and a response mapping merges into it. An import's mapping and its preMap/postMap hooks shape only the request it sends ([writing-mappings](../writing-mappings/SKILL.md#gotchas), gotcha 11). A grouped record is an array (`$[0].field`, `$[*].field`): paths written for one object find nothing in it, and a Transformation 2.0 `modify` passes it through unchanged.

### 6. End-to-end debugging with execution logs (silent/logic bugs, production incidents)

When a test run can't answer it -- imports must actually submit, destination behavior matters, or it's a production incident -- arm execution logging, run the flow for real, then read the per-record I/O the run captured. Disarm logging when you're done.

1. Arm execution logging on the flow (a time-boxed debug window).
2. Trigger the run (or wait for the next scheduled run).
3. After the run, list the captured per-record logs for that job.
4. Drill into one record's steps, then read one stage's data by name (stage names: step 7).
5. Disarm logging.

A stage read returns entries that name their stage, and a mapping or hook entry carries `input`, `output` and `errors` (much like a test run's per-stage results), so the failing stage tells you where the record broke. The stage data for the API call holds the raw HTTP request and response. Errors carry retry data -- see step 8 to fix and retry.

**Tracing one record from its error.** The error's `_flowJobId` is the run, and its `traceKey` finds the record's entries in that run's index -- one per step it passed, each with its own `recordId` and `groupId`; neither is the trace key or the error's `oIndex`. With them, read the step's stages in order: `mapping` (wh...