Pipeline Trigger Overview

Learn how to configure and manage pipeline triggers in Mage

​

Overview

Pipeline triggers are automated scheduling mechanisms that determine when and how data pipelines should run in Mage. A single pipeline can have multiple triggers, each with different configurations for various use cases like production runs, testing, or data backfills.

​

Key Benefits of Pipeline Triggers:

• Automated execution - Run pipelines without manual intervention
• Flexible scheduling - Support for cron expressions and API calls
• Environment management - Different trigger configurations for dev, staging, and production
• Resource optimization - Control when pipelines consume compute resources
• Monitoring and SLA - Built-in performance tracking and alerting

Pipeline Schedule and Trigger are the same thing in Mage. You may see either term used throughout the product.

​

Trigger Types

New trigger creation supports Schedule and API triggers. Event triggers are deprecated and may only appear for existing legacy configurations.

​

Schedule triggers

Schedule triggers are the most common type, designed for recurring data pipeline execution at predetermined intervals. They’re perfect for ETL processes, data updates, and regular analytics jobs.

Available scheduling frequencies include:

• Run exactly once - Run one time when the schedule starts, then disable the trigger
• Hourly - Run at minute 0 of each hour in UTC
• Daily - Run once per day at 00:00 UTC
• Weekly - Run every Monday at 00:00 UTC
• Monthly - Run on the first day of each month at 00:00 UTC
• Always on - Keep streaming pipelines running continuously by starting a new run when the previous run completes
• Custom (cron syntax) - Use cron expressions for exact clock times and advanced scheduling patterns

Custom frequency allows you to use cron syntax for advanced scheduling patterns. This gives you complete control over when your pipeline runs.

​

Common cron examples

Here are some common cron patterns you can use with custom frequency:

# Every 15 minutes
*/15 * * * *

# Every weekday at 9 AM
0 9 * * 1-5

# Every day at 2:30 PM
30 14 * * *

# Every Sunday at midnight
0 0 * * 0

# Every 1st of the month at 6 AM
0 6 1 * *

# Every 5 minutes during business hours (9 AM - 5 PM, weekdays)
*/5 9-17 * * 1-5

Cron expression format: [minute] [hour] [day(month)] [month] [day(week)]

Timezone behavior: If you have the display_local_timezone setting enabled, the local cron expression will match your local timezone only if the minute and hour values are single values without any special characters (comma, hyphen, or slash). You can still use cron expressions with special characters for the minute/hour values, but they will be based in UTC time.

Use online cron expression generators or cron documentation to create more complex scheduling patterns that fit your specific needs.

​

API triggers

API triggers enable on-demand pipeline execution through HTTP requests. They’re ideal for webhook integrations, external system triggers, and manual pipeline execution from external applications. For more details, refer to the API Trigger documentation.

Use cases for API triggers:

• Webhook integrations with external services
• Manual pipeline execution from external applications
• Event-driven data processing
• Integration with CI/CD pipelines

You can make a POST request to an endpoint provided in the UI when creating or editing a trigger. The endpoint depends on the Mage domain and the trigger’s unique ID.

API endpoint:

POST {MAGE_HOST}/api/pipeline_schedules/3/pipeline_runs/abc123

Request payload: Runtime variables can optionally be included in the request payload and are accessible within each pipeline block.

{
  "pipeline_run": {
    "variables": {
      "env": "staging",
      "schema": "public"
    }
  }
}

​

Deprecated Event triggers

Event triggers are deprecated for new trigger creation. Use API triggers for new event-driven and webhook-style workflows. Existing legacy event triggers may still appear in older projects.

​

How to Create and Manage Pipeline Triggers

​

Creating a New Trigger

To create a new trigger for your pipeline:

1. Navigate to your pipeline in the Mage UI
2. Click the Triggers tab in the pipeline view
   • Direct URL: /pipelines/[pipeline_uuid]/triggers
3. Click + New trigger button
4. Select the trigger type (Schedule or API)
5. Configure the trigger settings based on the type you selected
6. Click Create trigger to create the trigger

The new trigger form opens as a draft. Mage does not create the trigger until you click Create trigger, and it warns you before leaving the page if the draft has unsaved changes.

Trigger storage: Triggers created via the UI are stored in the database. For environment synchronization and version control, you can save triggers in code to sync them across different environments.

​

Trigger configuration fields

When creating or editing a trigger, you’ll see the following configuration fields:

Basic settings:

• Name - A descriptive name for your trigger. Names are trimmed before validation, must be unique within the pipeline, must include at least one letter or number, and cannot exceed 255 characters.
• Description - Optional description to explain the trigger’s purpose
• Tags - Add or search tags to organize and categorize your triggers. Each tag name can contain up to 70 characters.

Schedule-specific fields:

• Frequency - Choose from preset options (Hourly, Daily, Weekly, Monthly, Once, Always on) or select Custom for CRON syntax when you need more precise timing
• Schedule start - Choose Start at next Frequency time to let the trigger run at the next scheduled Frequency time, or Do not start before date/time to set the earliest allowed start. That date/time is not the run time; Frequency still controls when runs happen. Schedule start only controls when that trigger plan is allowed to begin.
• Schedule load guidance - For @once triggers with Do not start before date/time, Mage may show load or quiet-hour guidance for the selected start time. This is planning guidance only; use it to decide whether the selected start time is acceptable. If overlap is high, use suggested lower-load times, or keep your selected time if overlap is acceptable.
• Enable landing time - Instead of scheduling starts at a specific time, schedule runs so the pipeline can complete by a specified time

Additional options:

• Runtime variables - Override pipeline variables for this specific trigger
• Run settings - Configure timeouts, SLA, and failure handling
• Advanced settings - Additional trigger-specific configurations

​

Viewing and Monitoring Triggers

​

Pipeline-Specific Trigger Management

To view all triggers for a specific pipeline:

1. Navigate to your pipeline in the Mage UI
2. Click the Triggers tab
   • Direct URL: /pipelines/[pipeline_uuid]/triggers
3. You’ll see a comprehensive table listing all configured triggers with the following information:

Frequency, status, tag, and type filters are preserved when you navigate away and return to this pipeline’s Triggers page during the same browser session. Saved filters are scoped to the pipeline. They are removed when you select Clear and are not restored in a new browser session unless they are part of the URL you open. Applied filters appear in a band below the toolbar. Select an individual chip to remove one value, or select Clear to remove all applied filters while preserving unrelated list state.

​

Project-Wide Trigger Dashboard

To view all triggers across your entire project:

1. Go to the Mage homepage
2. Click Triggers in the left navigation menu
   • Direct URL: /triggers
3. You’ll see all triggers from all pipelines in your project

UUID, type, and status filters are preserved when you navigate away and return during the same browser session. Selecting Clear removes the saved filters. Free-text search is cleared when you leave the page. Saved filters and search text are not restored in a new browser session unless they are part of the URL you open. On the List view, applied filters appear in a band below the toolbar. Select an individual chip to remove one value, or select Clear to remove all applied filters while preserving unrelated list state.

Trigger list columns:

• Active - Toggle switch showing if the trigger is enabled/disabled
• Logs - Icon to access trigger execution logs
• Name - The trigger name (clickable link with arrow icon for code-defined triggers)
• Type - Trigger type (Schedule, API, or deprecated legacy Event)
• Frequency - How often the trigger runs (e.g., @once, @daily)
• Status - Current status of the trigger
• Next - Scheduled time for the next execution (with info icon for details)
• Runs - Total number of pipeline runs initiated by this trigger
• Tags - Associated tags for organization
• Created - Timestamp when the trigger was created (with info icon for details)

You can click on the logs icon to view detailed execution history and logs for each trigger.

​

Day calendar

Open Triggers → Calendar to review a day’s planned starts and inspect their execution status. Calendar is the default tab; List and Load open the existing trigger and load views.

Pick a day and filter starts

1. Select a date, type YYYY-MM-DD, or choose Today.
2. Search by pipeline or trigger name. Use All statuses to filter by execution status, or select the succeeded or failed summary count.
3. Open Calendar options (the three-dot menu) to search for a display timezone or filter by an existing trigger tag. Choose All trigger tags to remove the tag filter.
4. Select Refresh calendar to update the selected day. Today’s view also refreshes periodically.

Rows represent hours and horizontal positions represent minutes. Hover over the grid for an exact time. Entries mark scheduled starts; their width does not show run duration. Overlapping entries form groups: +N counts additional starts. Tag colors identify categories, while separate status badges keep failures visible inside groups. A branch icon and count indicate verified downstream runs when available.

Changing the display timezone changes the displayed day and times, not the trigger’s UTC schedule. The default follows your local-time display preference; an explicit calendar selection takes precedence. Daylight-saving transitions can add or remove hour rows.

Inspect a scheduled start

Select an entry to open its details. For a group, choose an occurrence from the list; Back returns to that group. Different triggers remain separate occurrences even when they belong to the same pipeline.

Open Timing details to see the interval or cron, UTC schedule timezone, display timezone, trigger ID, and available execution start and end times. Select the pipeline name, trigger name, or run button to open its page in a new tab. The panel toggle beside Refresh calendar hides or reopens details while retaining your selection.

Unconfirmed means a start is due but no execution is confidently matched; it does not mean failure. Unknown means execution data cannot be verified. All statuses also includes waiting, unconfirmed, and unknown states that have no individual filter option. A downstream failure does not change the selected execution’s own status.

Trace connected runs

For a matched execution, review the downstream summary, then select lineage ↗ to open its graph. Select a failed or blocked count to focus an affected execution directly.

Pan, zoom, or fit the graph; search for a pipeline and choose a matching run, or select a node to inspect it. View run opens the execution in a new tab. Close the modal to return to the same calendar selection. The modal keeps its opening snapshot; close and reopen it to see refreshed relationships.

Lineage shows recorded execution links, including serial and parallel branches, with each run counted once. It does not predict the next pipeline after success or dependencies for unstarted schedules. Downstream runs are not added as calendar starts unless independently scheduled. Results are bounded and permission-filtered: Some connections unavailable means verified counts are partial. Execution lineage unavailable offers a retry action; neither state establishes that no dependencies exist.

The calendar plots enabled, recurring time triggers using current settings. Past dates do not preserve historical schedule configurations. Disabled, run-once, always-on, landing-time, and streaming schedules are omitted. Known manual, API, backfill, and dependency-triggered runs are excluded from scheduled-run matching; older runs may lack origin information. Unsupported schedules, missing metadata, unavailable data, or result limits can make the calendar incomplete. An alert appears even when some starts remain visible; an incomplete empty day is identified separately. Summary counts describe the records shown.

Enabling/Disabling triggers:

• From trigger list: Use the toggle switch in the “Active” column to enable or disable a trigger
• From trigger detail page: Navigate to /pipelines/[pipeline_uuid]/triggers/[trigger_id] and use the “Disable trigger” button
• Disable all triggers in Mage Pro: On the global Triggers page, the current search and filters define the scope; only matching active triggers are disabled
• Disabled triggers: Will not run automatically but remain configured for future use
• Enabled triggers: Will run according to their configured schedule, API endpoint, or legacy event conditions

Testing triggers:

• Run once button: Click the “Run@once” button in the trigger list or detail page to manually trigger a pipeline run
• Testing purpose: Use this to test your trigger configuration before enabling it for automatic execution
• Immediate execution: The pipeline will start running immediately when you click the button

Backfill triggers:

• Historical data processing: Create backfill triggers to process historical data for a specific date range
• One-time execution: Backfill triggers run once and then disable themselves after completion
• Data gap filling: Use backfills to fill in missing data or reprocess data with updated logic

Deleting triggers:

• Mage OSS: Click the delete (trash) icon in the Actions column to delete a trigger
• Mage Pro: Right-click a trigger row to open its context menu. Select Copy URL to copy the trigger’s direct URL, or select Delete and confirm Yes to delete the trigger.

​

Advanced Trigger Configuration

​

Run Settings and Performance Optimization

​

Timeout configuration

• Set a timeout for each run of this trigger (optional)
  • Configure a timeout in seconds for pipeline runs initiated by this trigger
  • If a run exceeds the timeout, it will be marked with the specified timeout status
• Status for runs that exceed the timeout (default: failed)
  • Choose how to handle runs that exceed the configured timeout
  • Options typically include: failed, cancelled, or warning

​

SLA configuration

• Configure trigger SLA
  • Set up Service Level Agreement monitoring for your trigger
  • Define expected completion time with configurable time units (minutes, hours, days)
  • Helps monitor trigger performance against business requirements

​

Additional trigger settings

• Override runtime variables
  • You can override the runtime variables defined in the pipeline to use different values for different triggers.
  • This option only appears if you have runtime variables defined in the pipeline.
  • Use Form mode to add, edit, or delete runtime variables one at a time.
  • Switch to JSON mode to edit the full runtime variables object in a code editor. Click Format JSON to reformat valid JSON before saving.
  • Runtime variable JSON must be an object. Variable names cannot contain whitespace, and reserved Mage variables cannot be included.
  • Benefits: Allows you to run the same pipeline with different configurations for different use cases (e.g., different data sources, environments, or processing parameters) without creating separate pipelines.
• Keep running pipeline even if blocks fail
  • When enabled, the pipeline continues running even if individual blocks fail
  • The pipeline will continue running blocks that are not dependent on the failed blocks until completion
• (Schedule triggers only) Skip run if previous run still in progress
  • When enabled, the scheduler will not start a new run if a previous run for this trigger is still in progress.
• (Schedule triggers only)
  Available starting in v0.9.62
  Create initial pipeline run if start date is before current execution period
  • When enabled, this creates an initial pipeline run if the schedule’s execution date is after its start datetime but before the current time.
  • Example: If current time is 1/4/24 12:00 UTC and you create a daily trigger with start date 1/3/24 12:00 UTC, a pipeline run with execution date 2024-01-04 00:00 is immediately created.

​

Frequently Asked Questions

​

What is a trigger in Mage?

A trigger is a set of instructions that determine when or how a pipeline should run. New trigger creation supports Schedule and API triggers. Event triggers are deprecated and may only appear for existing legacy configurations.

• Schedule: Runs pipelines on a set interval (e.g., hourly, daily, weekly, monthly, or custom cron)
• API: Runs pipelines when a specific API endpoint is called, optionally with runtime variables

​

How do I create a trigger for my pipeline?

You can create triggers via the Mage UI or by defining them in a triggers.yaml file within your pipeline folder. In the UI, go to the pipeline editor, select “Triggers,” and click “New trigger.” In code, add your trigger configuration to pipelines/[pipeline_uuid]/triggers.yaml.

​

Can I configure triggers in code and in the UI? How are changes synced?

Yes, you can configure triggers in both the UI and code. Changes made in the UI are synced to the YAML file, and vice versa. However, deleting a trigger from the YAML file does not remove it from the database; you must delete it in the UI.

​

Why does my daily trigger run at 2AM instead of my specified time?

By default, @daily triggers run at midnight UTC. If you want your pipeline to run at a specific time, use the “custom” frequency option and enter a cron expression that matches your desired schedule.

​

What happens if I delete a pipeline—are its triggers deleted too?

No, deleting a pipeline does not automatically delete its associated triggers. You need to manually remove triggers in the UI to avoid orphaned triggers.

​

How do I prevent a pipeline from running if a previous run is still in progress?

There is a trigger setting called “Skip run if previous run still in progress.” Enabling this will prevent the scheduler from starting a new run if the previous one is still active.

​

Can I allow a pipeline to continue even if some blocks fail?

Yes, you can enable the “Allow blocks to fail” setting in your trigger configuration. This allows the pipeline to continue running blocks that are not dependent on failed blocks.

​

How do I trigger a pipeline via API?

You can set up an API-type trigger, which allows you to start a pipeline by making a POST request to a specific endpoint. You can also pass runtime variables in your request payload.