Skip to content

aws-iac-mcp-server: troubleshoot_cloudformation_deployment returns empty events (describe_events not scoped to failed OperationId) #4275

Description

@dbartholomae

Description

troubleshoot_cloudformation_deployment (in cloudformation_deployment_troubleshooter.py) always returns an empty cloudformation_events list — even for a stack that is clearly in a failed state (UPDATE_ROLLBACK_COMPLETE) — because it never scopes the describe_events call to the operation that actually failed.

Root cause

# src/aws-iac-mcp-server/awslabs/aws_iac_mcp_server/tools/cloudformation_deployment_troubleshooter.py

stacks = self.cfn_client.describe_stacks(StackName=stack_name)['Stacks']
...
response['raw_data']['stack_status'] = stacks[0].get('StackStatus')
...
cloudformation_events = self.cfn_client.describe_events(
    StackName=stack_name, Filters={'FailedEvents': True}
)['OperationEvents']

Per the DescribeEvents docs, CloudFormation groups events by Operation ID — every rollback, update, etc. is its own operation. DescribeStacks now returns a LastOperations array, e.g.:

"LastOperations": [
  {"OperationType": "ROLLBACK", "OperationId": "d0f12313-..."},
  {"OperationType": "UPDATE_STACK", "OperationId": "1c211b5a-..."}
]

The failed resource events live under the UPDATE_STACK (or CREATE_STACK) operation — not the ROLLBACK operation, since the rollback itself typically succeeds. The tool reads describe_stacks only for StackStatus, discards LastOperations, and then calls describe_events with just StackName and no OperationId. Without an explicit OperationId, FailedEvents=true appears to scope to the latest operation only (the successful ROLLBACK), so it finds nothing — even though the preceding UPDATE_STACK operation has failed events available via describe-events --operation-id <id> --filter FailedEvents=true.

Reproduced against 1.0.11 and confirmed the bug is still present at main (current 1.0.20) — the file has had no relevant changes since it was introduced in #1836.

Steps to reproduce

  1. Trigger a CloudFormation stack update that fails and rolls back to UPDATE_ROLLBACK_COMPLETE.
  2. Call troubleshoot_cloudformation_deployment(stack_name=..., region=...).
  3. Observe raw_data.stack_status correctly shows UPDATE_ROLLBACK_COMPLETE, but raw_data.cloudformation_events, matched_failures, and failed_event_count are all empty/zero, giving no indication of what actually failed.

Expected behavior

The tool should:

  1. Read LastOperations from the describe_stacks response.
  2. Identify the failed operation (the one preceding a ROLLBACK, or the operation itself if it's a CREATE/UPDATE that failed without rollback).
  3. Call describe_events(OperationId=<that operation id>, Filters={'FailedEvents': True}) to retrieve the actual failed resource events.

Environment

  • Package: awslabs.aws-iac-mcp-server
  • Version tested: 1.0.11 (bug also present at main / 1.0.20)
  • AWS CLI/SDK: boto3 CloudFormation describe_events/describe_stacks

Suggested fix

stacks = self.cfn_client.describe_stacks(StackName=stack_name)['Stacks']
stack = stacks[0]
response['raw_data']['stack_status'] = stack.get('StackStatus')

# Pick the most recent non-rollback operation to find what actually failed
last_operations = stack.get('LastOperations', [])
operation_id = next(
    (op['OperationId'] for op in last_operations if op.get('OperationType') != 'ROLLBACK'),
    last_operations[0]['OperationId'] if last_operations else None,
)

describe_events_kwargs = {'Filters': {'FailedEvents': True}}
if operation_id:
    describe_events_kwargs['OperationId'] = operation_id
else:
    describe_events_kwargs['StackName'] = stack_name

cloudformation_events = self.cfn_client.describe_events(**describe_events_kwargs)['OperationEvents']

Happy to submit a PR with this fix if that's useful — wanted to file the issue first in case there's context I'm missing about why StackName-only scoping was chosen.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status
    To triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions