Quick Diagnostics

Here are some quick diagnostics that you can do to review your pipeline performance.

Scenario	What does it mean?	How to resolve:
Components are showing "Queued" for long periods of time.	If a component within a pipeline is hitting max capacity (or is not configured), then it stops processing new messages until it reports ready. In the case of an extended capacity or misconfigurations, the previous component in the pipeline will check the status of the next step, and this may lead to a chain of pending until the Jobs will wait to avoid data loss.	See if large numbers of components are showing "Queued" counts. If so, look at the latest in flow presenting this issue to assess if configured properly.
Jobs are taking a long time to run.	This may be due to downstream pipelines queues (see Components are showing "Queued"), or due to creating large numbers of Jobs at the same time. The Job system will work to ensure data source rate limits are efficiently used.	Review the Pipeline within Portal to assess if "Queued" counts appear on the downstream components.

Scenario

What does it mean?

How to resolve:

Components are showing "Queued" for long periods of time.

If a component within a pipeline is hitting max capacity (or is not configured), then it stops processing new messages until it reports ready.

In the case of an extended capacity or misconfigurations, the previous component in the pipeline will check the status of the next step, and this may lead to a chain of pending until the Jobs will wait to avoid data loss.

See if large numbers of components are showing "Queued" counts. If so, look at the latest in flow presenting this issue to assess if configured properly.

Jobs are taking a long time to run.

This may be due to downstream pipelines queues (see Components are showing "Queued"), or due to creating large numbers of Jobs at the same time. The Job system will work to ensure data source rate limits are efficiently used.

Review the Pipeline within Portal to assess if "Queued" counts appear on the downstream components.

Updated about 4 hours ago