Skip to content

attu b6 backup s3 failed #1013

Open
Open

Description

@cnangel

Describe the bug:
When I use S3 on AWS, I am unable to perform backups, for example:
UNKNOWN: backup: run collection task: backup: execute collection strategy: backup: execute dml task backup: wait task backup: execute task backup: execute dml task backup: backup segments backup: wait backup segments backup: copy segment backup: segment 464629288825733969 has no insert logs

Image

Attu version:
3.0.0-beta.6

Milvus version:
2.6.15

milvus-backup version:
0.5.16

Activity

  1. shanghaikid commented on Jun 10, 2026

    @shanghaikid
    Collaborator

    Thanks for reporting this.

    We tested the AWS S3 backup path with milvus-backup 0.5.16 and confirmed that S3 can work as the backup target through the full flow: check, create, list, restore, and delete.

    This specific error does not look like an AWS S3 connectivity or permission failure:

    segment 464629288825733969 has no insert logs
    

    At this point the backup task has already passed the storage connection/authentication stage and is copying collection segment data. The error means Milvus returned segment 464629288825733969, but that segment did not have insert logs for milvus-backup to copy.

    This usually points to the source collection or segment state, for example:

    • the collection was being written/imported/compacted during backup
    • data had not fully flushed or settled yet
    • the segment metadata exists, but its binlog/insert log metadata is empty or inconsistent

    Please try:

    1. Stop writes/imports/compaction for this collection.
    2. Flush the collection.
    3. Wait for the segment state to settle.
    4. Retry the backup.

    If it still fails, please share DataCoord logs around segment 464629288825733969 and the segment/collection state.

    We also agree the current Attu UI message is too raw here. We are improving the error classification so this will be shown as a source collection/segment data issue instead of the long UNKNOWN: backup: ... chain.

  2. cnangel commented on Jun 11, 2026

    @cnangel
    Author

    We cannot stop the write operation, and there are no import/compression tasks.
    I tried flushing the collection, but it didn't work. Retrying also had no effect.
    milvus-backup issue? Could you give them some feedback?

  3. shanghaikid commented on Jun 11, 2026

    @shanghaikid
    Collaborator

    can you run deep check, and show me the result ? thanks

    Image
  4. cnangel commented on Jun 11, 2026

    @cnangel
    Author
    Image Image
  5. cnangel commented on Jun 11, 2026

    @cnangel
    Author

    I think it's a timeout issue?

  6. cnangel commented on Jun 11, 2026

    @cnangel
    Author
    Image Image

    The other S3 had the same problem.

  7. cnangel commented on Jun 11, 2026

    @cnangel
    Author
    Image

    Sometimes it crashes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions