Skip to content

Conversation

@epipav
Copy link
Collaborator

@epipav epipav commented Oct 16, 2025

Creates a new tinybird pipe for copy pipe executions

Monitors Tinybird copy job executions and exposes their status as Prometheus gauge metrics. Consumed by datadog scraper (dd-tinybird-copy-pipe-executions-scraper)

Tracks the latest execution status for each copy pipe (starting today) and generates one-hot encoded metrics where each pipe gets a value of 1 for its current status ('ok', 'error', 'cancelled', 'queued', 'working') and 0 for all other statuses.

Metric: copy_pipes_latest_execution_status (gauge)
Labels: pipe_name, status

Note: The 'queued' status is virtual—it's derived by detecting the error message "You have reached the maximum number of copy jobs" rather than being a native Tinybird status. This is because tinybird retries these again when possible but returns an error status. Similarly, 'ok' is mapped from the native 'done' status for datadog color-scheme convention.

@github-actions
Copy link
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

@epipav epipav requested a review from themarolt October 16, 2025 10:51
@github-actions
Copy link
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

@cursor
Copy link

cursor bot commented Oct 23, 2025

This PR is being reviewed by Cursor Bugbot

Details

Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

@cursor
Copy link

cursor bot commented Oct 23, 2025

Bug: MultiIf Logic Fails on Consecutive Errors

The multiIf logic for max_jobs_err doesn't correctly categorize the second consecutive occurrence. When the current and immediately previous executions both have a max_jobs_err, the status falls through to the original value instead of being marked as queued, which is inconsistent with the pipe's described retry behavior.

Fix in Cursor Fix in Web

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants