Troubleshoot Workflow Automation

This page provides solutions to common issues you might encounter when using Workflow Automation. For additional support, see the New Relic Support Forum or contact New Relic Support.

AWS credentials and authentication

"Invalid token" or expired credentials

Problem: Workflows fail with token expiration errors.

Solutions:

1. For session tokens: Check the expiration timestamp—session tokens typically last 1-12 hours. Generate new ones before they expire using:
   bash

   Copia

   $
   aws sts assume-role \
   >
     --role-arn "arn:aws:iam::YOUR_ACCOUNT:role/YOUR_ROLE" \
   >
     --role-session-name "WorkflowAutomationSession"

2. For access keys: Verify both the Access Key ID and Secret Access Key are correctly stored in secrets manager. Re-check the values in NerdGraph GraphiQL explorer.
3. Check secrets syntax: Ensure you're using ${{ :secrets:keyName }} format, not ${{ secrets.keyName }}. The colon prefix (:secrets:) is required.

Can't find my role ARN

Problem: You need the ARN but can't locate it in AWS.

Solution:

1. Sign in to the AWS IAM Console
2. Choose Roles from the navigation
3. Search for your role name (e.g., NewRelicWorkflowAutomationRole)
4. Select the role—the ARN appears in the Summary section
5. The ARN format is: arn:aws:iam::<YOUR_AWS_ACCOUNT>:role/<ROLE_NAME>

Workflows can't access specific AWS resources

Problem: Your role has permissions, but workflows still can't access certain resources.

Solutions:

1. Resource-level permissions: Check if your IAM policy restricts access to specific resource ARNs. You may need to add wildcard (*) for testing, then lock down to specific resources.

   {
     "Effect": "Allow",
     "Action": "ec2:DescribeInstances",
     "Resource": "*" // Change from specific ARN to * for testing
   }

   Copia
2. Service control policies (SCPs): If you're in an AWS Organization, SCPs might block certain actions. Contact your AWS administrator to review organization policies.
3. Region mismatch: Ensure your IAM permissions specify the correct AWS region where your resources exist. Update the workflow's awsRegion parameter to match your resource location.

Workflow execution issues

Quick troubleshooting reference

Use this table for fast diagnosis of common workflow problems:

For detailed instructions on viewing logs and execution history, see Monitor workflow runs.

Workflow shows as "Failed" status

Problem: A workflow appears with Failed status in the dashboard.

Solutions:

1. Check the run history:
   • Go to one.newrelic.com > All Capabilities > Workflow Automation
   • Click the workflow name
   • View Run history to see which execution failed
2. Review execution logs:
   • Click View logs on the failed run to identify the specific action that failed
   • Look for error messages that indicate the root cause
3. Common causes:
   • Invalid credentials: Expired tokens, incorrect secrets syntax, or wrong secret keys
   • Missing permissions: Insufficient AWS IAM permissions, Slack bot token scopes, or API access
   • Changes to target resources: Deleted EC2 instances, removed Slack channels, or renamed databases
   • API rate limits exceeded: Too many requests to external services (AWS, Slack, etc.)
   • Network timeouts: External APIs taking too long to respond
4. Fix and rerun: After resolving the issue, manually trigger the workflow using the StartWorkflowRun API to verify it works.

Workflow execution takes too long

Problem: Workflow runs exceed expected duration or timeout.

Solutions:

1. Review action sequence: Check logs to identify slow actions. Look for actions with execution times > 30 seconds.
2. Optimize queries: If using NRQL queries, optimize them for better performance:
   • Add specific time ranges instead of querying all data
   • Use LIMIT to reduce result set size
   • Filter early with WHERE clauses
3. Check external APIs: Slow responses from integrated services (AWS, Slack) can delay execution. Test API response times separately.
4. Consider workflow limits: Review workflow limits for timeout constraints (typically 15 minutes per workflow).
5. Break into smaller workflows: Split complex workflows into smaller, focused automations that can run in parallel.

Changes to workflow don't take effect

Problem: You edited a workflow but the changes don't apply when it runs.

Solutions:

1. Verify you saved: Check that you clicked Save after editing the workflow configuration.
2. Check the version:
   • Go to workflow details
   • Click Version history tab
   • Ensure your latest changes appear as a new version
   • Verify this version is marked as active
3. Update scheduled runs: If the workflow runs on a schedule, update the schedule to use the new version:
   • Go to one.newrelic.com > All Capabilities > Workflow Automation
   • Find scheduled runs
   • Update the schedule to reference the new workflow version
4. Clear cache: Browser caching might show old configuration—hard refresh the page (Ctrl+Shift+R or Cmd+Shift+R).

Workflow dashboard issues

Can't find a workflow in the dashboard

Problem: A workflow you created doesn't appear in the dashboard list.

Solutions:

1. Check filters:
   • Click the filter dropdown
   • Select "All" for status filters
   • Clear the search bar
2. Verify account: Confirm you're logged into the correct New Relic account where the workflow was created. Check the account selector in the top-right corner.
3. Check permissions: Ensure your user role has access to view workflows. Contact your New Relic admin if you need workflow access permissions.
4. Refresh the page: Browser caching can sometimes hide recent changes. Try a hard refresh (Ctrl+Shift+R or Cmd+Shift+R).

Cannot delete a workflow

Problem: Delete option is grayed out or deletion fails.

Solutions:

1. Check permissions: Verify your user role has delete permissions for workflows. Contact your account admin if needed.
2. Stop scheduled runs: Cancel any active scheduled runs before deleting:
   • Navigate to the workflow details
   • Go to Scheduled runs tab
   • Cancel all active schedules
3. Check dependencies: Some workflows cannot be deleted if other automations depend on them. Check if the workflow is referenced by:
   • Other workflows that call it
   • Alert policies that trigger it
   • External systems that start it via API
4. Contact support: If the issue persists after trying the above, contact New Relic Support for assistance.

Integration-specific issues

Slack notifications not appearing

Problem: Workflow completes successfully but Slack messages don't appear.

Solutions:

1. Verify channel ID: Ensure you're using the Slack channel ID (e.g., C01234ABCD), not the channel name. Find the ID in Slack:
   • Right-click the channel name
   • Select View channel details
   • Copy the Channel ID at the bottom
2. Check bot permissions: Verify your Slack bot has the following scopes:
   • chat:write - Post messages
   • channels:read - View public channels
   • groups:read - View private channels (if applicable)
3. Verify bot is in channel: Add the bot to the target channel:
   • Type /invite @YourBotName in the channel
   • Confirm the bot appears in the member list
4. Check token in secrets: Verify the Slack token stored in secrets manager is correct and hasn't expired.

AWS Systems Manager operations fail

Problem: SSM automation documents or commands fail to execute.

Solutions:

1. Verify SSM permissions: Ensure the IAM role has these permissions:

   {
     "Effect": "Allow",
     "Action": [
       "ssm:CreateDocument",
       "ssm:DeleteDocument",
       "ssm:StartAutomationExecution",
       "ssm:GetAutomationExecution"
     ],
     "Resource": "*"
   }

   Copia
2. Check SSM agent: Verify the target EC2 instances have the SSM agent installed and running:
   bash

   Copia

   $
   aws ssm describe-instance-information --region us-east-1

3. Verify instance profile: EC2 instances need an IAM instance profile with SSM permissions to execute commands.
4. Check document exists: If using an existing SSM document, verify it exists in your AWS account and region.

Template-specific issues

Entity GUID not found

Problem: Templates fail with Entity not found errors.

Solutions:

1. Find correct entity GUID:
   • Go to the monitored resource page in New Relic
   • Check the URL or entity metadata for the GUID
   • Use entity search to locate entities
2. Verify entity exists in correct account: Confirm you're using the entity from the same account where the workflow is deployed
3. Confirm entity reporting data: Check that the entity shows Last seen recently—stale entities may have been removed
4. For AWS entities: Ensure integrations are active

NRQL query returns no results

Problem: Templates using NRQL queries complete but return empty datasets.

Solutions:

1. Test queries first: Use query builder to validate queries before adding to templates
2. Verify data type and event names:
   • Use FROM Transaction, not FROM Transactions
   • Check attribute names are exact matches (case-sensitive)
3. Adjust time ranges: For sparse data, expand the time window:
   • Add SINCE 1 hour ago for recent data
   • Use SINCE 1 day ago for trends
4. Simplify and test incrementally: Start with basic query, add filters one at a time
5. Check NRQL syntax: Verify query structure matches NRQL requirements

Template deployment fails

Problem: Template won't deploy or shows errors during deployment.

Solutions:

1. Check all required inputs: Ensure you've filled in all mandatory fields (credentials, GUIDs, queries)
2. Verify credential format: Secrets must use ${{ :secrets:keyName }} format with the colon prefix
3. Test credentials independently: Before deploying, test AWS credentials with AWS CLI and Slack tokens in Slack API
4. Review template requirements: Each template lists specific prerequisites—verify all are met
5. Check for special characters: Some fields (channel names, queries) may fail with special characters—use alphanumeric where possible

Approval workflows not responding

Problem: Templates requiring Slack approval (API Gateway rollback, EC2 resize) don't detect reactions.

Solutions:

1. Verify bot token scopes:
   • reactions:read - Required to detect emoji reactions
   • chat:write - Required to post approval messages
2. Check reaction format: Use the exact emoji specified in the workflow (typically 👍 for approve, 👎 for deny)
3. Verify timeout settings: Approval requests expire after configured timeout (usually 5-10 minutes)
4. Ensure bot can read messages: Bot must be in the channel and have permission to read message history
5. Test with simple approval: Create a test workflow with just approval logic to isolate the issue

EC2 instance management issues

Problem: EC2 template fails to resize or manage instances.

Solutions:

1. Verify CloudWatch Metric Streams configured: Required for real-time EC2 metrics
2. Check EC2 monitoring integration active: Ensures instance data flows to New Relic
3. Confirm instance state: Instances must be in running or stopped state—transitional states (pending, stopping) cause failures
4. Verify instance type compatibility: Not all instance types support all resize operations—check AWS instance type documentation
5. Check stop/start permissions: IAM role needs ec2:StopInstances and ec2:StartInstances in addition to ec2:ModifyInstanceAttribute

JSON parsing template not logging data

Problem: JSON parsing template runs successfully but no data appears in New Relic logs.

Solutions:

1. Verify API endpoint: Test the status page URL in a browser—ensure it returns valid JSON
2. Check JSON structure: Template expects specific format—verify component health fields match expected structure
3. Review log filters: Ensure logging action uses correct event type and attributes
4. Test with simple endpoint: Use a basic JSON endpoint (like https://httpbin.org/json) to verify template mechanics work
5. Check account permissions: Verify workflow has permission to write logs to New Relic

NRLens SQL query authentication fails

Problem: NRLens template fails with authentication or connection errors.

Solutions:

1. Verify NRLens credentials: Confirm SQL Query Gateway credentials are current and correctly stored in secrets
2. Check organization and account IDs: Ensure both IDs match your New Relic organization exactly
3. Test SQL query separately: Use NRLens interface to test query before adding to workflow
4. Verify SQL syntax: NRLens SQL differs from NRQL—review NRLens documentation for supported syntax
5. Check query complexity: Very complex joins or large datasets may timeout—simplify query or add time limits

Next steps

If you're still experiencing issues after trying these solutions:

• Workflow Automation community forum: Ask questions and get help from other users
• New Relic Support: Contact support for account-specific issues
• Workflow best practices: Learn patterns for reliable workflows
• Workflow limits: Understand system constraints and quotas