Describe the bug

I call "openshift-install destroy cluster " command to delete OKD cluster on AWS account.
The command does not stop and after terminating the cluster ec2 instances is continously trying to describe the ec2 instances that do not exist. Repeating the command gives the same result.
Where those ec2 instance id's come from and how to cleanup account so that command is finished with success ?

The log from openshift-install destroy cluster :

time="2025-09-18T11:20:28Z" level=debug msg="OpenShift Installer 4.19.0-okd-scos.11"
time="2025-09-18T11:20:28Z" level=debug msg="Built from commit 249d7428c51fe85cabdab6394c4aed86fc24d398"
time="2025-09-18T11:20:28Z" level=info msg="Credentials loaded from the "default" profile in file "/home/python/.aws/credentials""
time="2025-09-18T11:20:28Z" level=debug msg="search for matching resources by tag in eu-central-1 matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}"
time="2025-09-18T11:20:29Z" level=debug msg="search for matching resources by tag in us-east-1 matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}"
time="2025-09-18T11:20:29Z" level=debug msg="search for IAM roles"
time="2025-09-18T11:20:30Z" level=debug msg="iterating over a page of 100 IAM roles"
time="2025-09-18T11:20:43Z" level=debug msg="iterating over a page of 100 IAM roles"
time="2025-09-18T11:20:57Z" level=debug msg="iterating over a page of 42 IAM roles"
time="2025-09-18T11:21:02Z" level=debug msg="search for IAM users"
time="2025-09-18T11:21:02Z" level=debug msg="iterating over a page of 1 IAM users"
time="2025-09-18T11:21:03Z" level=debug msg="search for IAM instance profiles"
time="2025-09-18T11:21:03Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}"
time="2025-09-18T11:21:04Z" level=debug msg=Terminating id=i-0197e26c1d04739c2 resourceType=instance
time="2025-09-18T11:21:04Z" level=debug msg=Terminating id=i-066abd4e41439ce50 resourceType=instance
time="2025-09-18T11:21:05Z" level=debug msg=Terminating id=i-0843ce9fb92fa81dc resourceType=instance
time="2025-09-18T11:21:05Z" level=debug msg=Terminating id=i-09c3c2bce07a29958 resourceType=instance
time="2025-09-18T11:21:06Z" level=debug msg=Terminating id=i-01b44051dfed10440 resourceType=instance
time="2025-09-18T11:21:06Z" level=debug msg=Terminating id=i-0c825358eedf1fc2b resourceType=instance
time="2025-09-18T11:21:07Z" level=debug msg=Terminating id=i-05ce9b35fe48d3b74 resourceType=instance
time="2025-09-18T11:21:13Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}"
time="2025-09-18T11:21:23Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}"
time="2025-09-18T11:21:33Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}"
time="2025-09-18T11:21:43Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}"
time="2025-09-18T11:21:53Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}"
time="2025-09-18T11:22:03Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}"
time="2025-09-18T11:22:13Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}"
time="2025-09-18T11:22:23Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}"
time="2025-09-18T11:22:33Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}"
time="2025-09-18T11:22:43Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}"
time="2025-09-18T11:22:53Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}"
time="2025-09-18T11:23:03Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}" time="2025-09-18T11:23:13Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}" time="2025-09-18T11:23:23Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}" time="2025-09-18T11:23:33Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}" time="2025-09-18T11:23:43Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}" time="2025-09-18T11:23:53Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}" time="2025-09-18T11:24:03Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}" time="2025-09-18T11:24:13Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}" time="2025-09-18T11:24:23Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}" time="2025-09-18T11:24:33Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}" time="2025-09-18T11:24:43Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}" time="2025-09-18T11:24:53Z" level=debug msg="search for instances by tag matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}" time="2025-09-18T11:24:53Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: 98d043f0-58cb-48af-b689-93eece570355, api error InvalidInstanceID.NotFound: The instance ID 'i-0d79d475669e04649' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-0d79d475669e04649" time="2025-09-18T11:24:54Z" level=debug msg="listing AWS hosted zones "okd01.mer22-dev.mercury.rbi.cloud." (page 0)" id=Z06531582WKTVMPHYAR2M time="2025-09-18T11:24:54Z" level=debug msg="listing AWS hosted zones "mer22-dev.mercury.rbi.cloud." (page 0)" id=Z06531582WKTVMPHYAR2M time="2025-09-18T11:24:54Z" level=info msg=Deleted id=Z06531582WKTVMPHYAR2M record set="A api-int.okd01.mer22-dev.mercury.rbi.cloud." time="2025-09-18T11:24:54Z" level=info msg=Deleted id=Z06531582WKTVMPHYAR2M record set="A api.okd01.mer22-dev.mercury.rbi.cloud." time="2025-09-18T11:24:55Z" level=info msg=Deleted id=Z06531582WKTVMPHYAR2M record set="A \052.apps.okd01.mer22-dev.mercury.rbi.cloud." time="2025-09-18T11:24:55Z" level=info msg=Deleted id=Z06531582WKTVMPHYAR2M
time="2025-09-18T11:24:55Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: 030f4557-8c65-4c43-b56c-51462e31a7ed, api error InvalidInstanceID.NotFound: The instance ID 'i-02f27e7db46641399' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-02f27e7db46641399" time="2025-09-18T11:24:55Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: 15e1401c-2622-4a72-91ea-c4966772243b, api error InvalidInstanceID.NotFound: The instance ID 'i-0279abf35766a5a2f' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-0279abf35766a5a2f" time="2025-09-18T11:24:56Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: fd08a5c6-b97a-4205-b791-b3e48726fbab, api error InvalidInstanceID.NotFound: The instance ID 'i-0bb3d6766c02f45c0' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-0bb3d6766c02f45c0" time="2025-09-18T11:24:56Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: fac5d85c-436d-4596-8f7a-6bcd9f443581, api error InvalidInstanceID.NotFound: The instance ID 'i-057df4826cd436d2b' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-057df4826cd436d2b" time="2025-09-18T11:24:57Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: 22f43ebb-7fef-44de-b432-439bcafd2c5b, api error InvalidInstanceID.NotFound: The instance ID 'i-03708a3d9a10f2842' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-03708a3d9a10f2842" time="2025-09-18T11:24:57Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: 3e06f51e-899f-4f38-a7c1-2fdf4012d049, api error InvalidInstanceID.NotFound: The instance ID 'i-034e4638b40884b2a' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-034e4638b40884b2a" time="2025-09-18T11:24:57Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: ed77d19e-b9b6-4fa0-8d15-f3a2e724cbe5, api error InvalidInstanceID.NotFound: The instance ID 'i-0c50e61174288bfea' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-0c50e61174288bfea" time="2025-09-18T11:24:57Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: cbf3a30c-fb47-4e03-b45c-008abda0fbd4, api error InvalidInstanceID.NotFound: The instance ID 'i-0d0fb1da2ea62b130' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-0d0fb1da2ea62b130" time="2025-09-18T11:24:58Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: d0a31a11-f30b-447a-8464-d767c1b6e78b, api error InvalidInstanceID.NotFound: The instance ID 'i-035dbaf838b54f083' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-035dbaf838b54f083" time="2025-09-18T11:24:59Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: ae9a1bc0-09f4-4451-b010-5d869b7d1ce4, api error InvalidInstanceID.NotFound: The instance ID 'i-022207997bb26abd3' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-022207997bb26abd3" time="2025-09-18T11:24:59Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: 838bccef-1e86-4581-8766-1dc615a0b758, api error InvalidInstanceID.NotFound: The instance ID 'i-08ade52046e97f487' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-08ade52046e97f487" time="2025-09-18T11:24:59Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: b1eb0baa-9571-4a65-9c06-b11ef97a6695, api error InvalidInstanceID.NotFound: The instance ID 'i-0af5ee30a8d4c8d26' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-0af5ee30a8d4c8d26" time="2025-09-18T11:24:59Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: 69e4fa0d-5d90-410f-9509-65b75617b9b7, api error InvalidInstanceID.NotFound: The instance ID 'i-079b947504022dca8' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-079b947504022dca8" time="2025-09-18T11:25:00Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: e8a28d4e-1eae-4704-b33f-a5ffe318f3f0, api error InvalidInstanceID.NotFound: The instance ID 'i-0ccbaba6c8fc81172' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-0ccbaba6c8fc81172" time="2025-09-18T11:25:00Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: 7d985e6c-d8a8-4b24-956e-dbc9619df6e2, api error InvalidInstanceID.NotFound: The instance ID 'i-04046d3f80729dbf1' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-04046d3f80729dbf1"
....
time="2025-09-18T11:33:51Z" level=debug msg="search for matching resources by tag in eu-central-1 matching aws.Filter{"kubernetes.io/cluster/okd01-l4rsp":"owned"}"
time="2025-09-18T11:33:52Z" level=debug msg="search for IAM roles"
time="2025-09-18T11:33:52Z" level=debug msg="iterating over a page of 100 IAM roles"
time="2025-09-18T11:33:52Z" level=debug msg="iterating over a page of 100 IAM roles"
time="2025-09-18T11:33:52Z" level=debug msg="iterating over a page of 40 IAM roles" time="2025-09-18T11:33:52Z" level=debug msg="search for IAM users"
time="2025-09-18T11:33:53Z" level=debug msg="iterating over a page of 1 IAM users" time="2025-09-18T11:33:53Z" level=debug msg="search for IAM instance profiles" time="2025-09-18T11:34:00Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: ea75269c-7338-4d8e-865b-e60336df539c, api error InvalidInstanceID.NotFound: The instance ID 'i-010ffc6d0c2650401' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-010ffc6d0c2650401"
time="2025-09-18T11:34:00Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: cd8aca35-4c36-404f-9c6b-a3e96819e3ff, api error InvalidInstanceID.NotFound: The instance ID 'i-06f922b38ba57cb88' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-06f922b38ba57cb88"
time="2025-09-18T11:34:00Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: 2760c82a-f0d1-40b5-8127-05163cb42183, api error InvalidInstanceID.NotFound: The instance ID 'i-09a38d47f85a8eece' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-09a38d47f85a8eece"
time="2025-09-18T11:34:00Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: db288683-967d-40e3-a790-8ba470ec0bcc, api error InvalidInstanceID.NotFound: The instance ID 'i-03708a3d9a10f2842' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-03708a3d9a10f2842"
time="2025-09-18T11:34:00Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: baf8aad8-0f7f-4712-9fa2-1d404af802fe, api error InvalidInstanceID.NotFound: The instance ID 'i-0423126c085710e3e' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-0423126c085710e3e"
time="2025-09-18T11:34:01Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: a5f23122-35e3-4f47-bf65-b97642e1e931, api error InvalidInstanceID.NotFound: The instance ID 'i-07da0237bfe174793' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-07da0237bfe174793"
time="2025-09-18T11:34:01Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: 402dec35-9adc-46f1-b3ce-c6f641bdff5c, api error InvalidInstanceID.NotFound: The instance ID 'i-0c54a9115d591d14f' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-0c54a9115d591d14f"
time="2025-09-18T11:34:01Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: f4123651-b0ca-4036-8e04-09cefc8471a6, api error InvalidInstanceID.NotFound: The instance ID 'i-0d79d475669e04649' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-0d79d475669e04649"
time="2025-09-18T11:34:01Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: 80d7427c-7589-44bc-9cb4-cea196bfe1dd, api error InvalidInstanceID.NotFound: The instance ID 'i-00239014544e11f9a' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-00239014544e11f9a"
time="2025-09-18T11:34:01Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: 5848b808-d624-443e-9e65-0811fc24bed4, api error InvalidInstanceID.NotFound: The instance ID 'i-048bffee6dc3dbe90' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-048bffee6dc3dbe90"
time="2025-09-18T11:34:01Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: 0d162329-6dc5-4e33-ac1b-0cccebdec0cb, api error InvalidInstanceID.NotFound: The instance ID 'i-01db32eda0197ed9c' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-01db32eda0197ed9c"
time="2025-09-18T11:34:01Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: ef35a9a9-249a-4cdb-8379-2d0ebce1e008, api error InvalidInstanceID.NotFound: The instance ID 'i-0dd3ab8c08a3d150a' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-0dd3ab8c08a3d150a"
time="2025-09-18T11:34:01Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: 97f92002-71e9-45de-b9dc-7bbbb3fbb5c2, api error InvalidInstanceID.NotFound: The instance ID 'i-022207997bb26abd3' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-022207997bb26abd3"
time="2025-09-18T11:34:01Z" level=debug msg="operation error EC2: DescribeInstances, https response error StatusCode: 400, RequestID: d423a0f2-5c4a-44a8-badf-cfc51d461d22, api error InvalidInstanceID.NotFound: The instance ID 'i-0269854e32a6acc07' does not exist" arn="arn:aws:ec2:eu-central-1:302263046320:instance/i-0269854e32a6acc07"

Version

4.19.0-okd-scos.11

Reproducibility

each time on single AWS account

Log Bundle

No response