[Bug]: Bulk Insert failed during Milvus restore with Azure buckets using Milvus Backup tool

[gifi-siby]

Is there an existing issue for this?

• I have searched the existing issues

Environment

- Milvus version:
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Perform a Milvus restore operation using the Milvus Backup tool to restore data to a Milvus instance associated with an azure bucket. Below error is found:

[ERROR] [core/backup_impl_restore_backup.go:371] ["executeRestoreCollectionTask failed"] [TargetDBName=default] [TargetCollectionName=watch_Back] [error="bulk insert fail, info: misaligned binlog count, field453580697765378790:5, field0:1: importing data failed"] [errorVerbose="bulk insert fail, info: misaligned binlog count, field453580697765378790:5, field0:1: importing data failed

Expected Behavior

Restore operation using the MilvusBack tool should be success without any error.

Steps To Reproduce

1. Create a backup using MilvusBackup tool
2. Restore the data to milvus associated with azure bucket

Milvus Log

No response

Anything else?

As part of BulkInsert, when recursive is true, ListObjects function of AzureObjectStorage is called to list all the files inside a certain path (inside backupPath/binlogs/insert_log/collectionID/partitionID/segmentID/groupID).
Folders and files were returned from ListObjects and thus following logic was misleading and caused misaligned binlog count error.
This issue is found as part of the MilvusBackup tool enhancement:
Enhancement task: #446
PR raised: #496

[yanliang567]

/assign @huanghaoyuanhhy
/unassign

[sre-ci-robot]

@yanliang567: GitHub didn't allow me to assign the following users: huanghaoyuanhhy.

Note that only milvus-io members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign @huanghaoyuanhhy
/unassign

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[zhuwenxing]

/assign @huanghaoyuanhhy

[sre-ci-robot]

@zhuwenxing: GitHub didn't allow me to assign the following users: huanghaoyuanhhy.

Note that only milvus-io members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign @huanghaoyuanhhy

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[huanghaoyuanhhy]

/assign

[gifi-siby]

/assign @gifi-siby