Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: SIGABORT when load index with inverted index #39595

Open
1 task done
chyezh opened this issue Jan 25, 2025 · 0 comments
Open
1 task done

[Bug]: SIGABORT when load index with inverted index #39595

chyezh opened this issue Jan 25, 2025 · 0 comments
Assignees
Labels
kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.

Comments

@chyezh
Copy link
Contributor

chyezh commented Jan 25, 2025

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version: e61a841806e53d1e0471141d874630009a6632b8
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

2025-01-25 15:39:26.669	[2025-01-25T07:39:26Z INFO  tantivy::directory::file_watcher] Meta file "/var/lib/milvus/data/indexnode/index_files/455551270650402252/1/meta.json" was modified
2025-01-25 15:39:26.669	[2025-01-25T07:39:26Z INFO  tantivy::directory::file_watcher] Meta file "/var/lib/milvus/data/indexnode/index_files/455551270650402241/1/meta.json" was modified
2025-01-25 15:39:26.682	[2025-01-25T07:39:26Z INFO  tantivy::directory::file_watcher] Meta file "/var/lib/milvus/data/indexnode/index_files/455551270650402235/1/meta.json" was modified
2025-01-25 15:39:26.731	[2025-01-25T07:39:26Z INFO  tantivy::directory::file_watcher] Meta file "/var/lib/milvus/data/indexnode/index_files/455551270650402225/1/meta.json" was modified
2025-01-25 15:39:26.744	[2025-01-25T07:39:26Z INFO  tantivy::directory::file_watcher] Meta file "/var/lib/milvus/data/indexnode/index_files/455551270650402196/1/meta.json" was modified
2025-01-25 15:39:26.744	[2025-01-25T07:39:26Z ERROR tantivy::reader] Error while loading searcher after commit was detected. OpenReadError(FileDoesNotExist("meta.json"))
2025-01-25 15:39:26.744	terminate called after throwing an instance of 'milvus::SegcoreError'
2025-01-25 15:39:26.744	  what():   => remove local directory:/var/lib/milvus/data/indexnode/index_files/455551270650402196/1/ failed, error: Directory not empty, files: /var/lib/milvus/data/indexnode/index_files/455551270650402196/1/.tantivy-meta.lock at /workspace/source/internal/core/src/storage/LocalChunkManager.cpp:228
2025-01-25 15:39:26.744	
2025-01-25 15:39:26.745	[2025-01-25T07:39:26Z INFO  tantivy::directory::file_watcher] Meta file "/var/lib/milvus/data/indexnode/index_files/455551270650402238/1/meta.json" was modified
2025-01-25 15:39:26.746	SIGABRT: abort
2025-01-25 15:39:26.746	PC=0x7f956ff9d9fc m=600 sigcode=18446744073709551610
2025-01-25 15:39:26.746	signal arrived during cgo execution
2025-01-25 15:39:26.746	
2025-01-25 15:39:26.746	goroutine 204779 gp=0xc001f27880 m=600 mp=0xc001a08008 [syscall, locked to thread]:

2025-01-25 15:39:45.605	non-Go function
2025-01-25 15:39:45.605		pc=0x7f956ff9d9fc
2025-01-25 15:39:45.605	non-Go function
2025-01-25 15:39:45.605		pc=0x7f956ff49475
2025-01-25 15:39:45.605	non-Go function
2025-01-25 15:39:45.605		pc=0x7f956ff2f7f2
2025-01-25 15:39:45.605	non-Go function
2025-01-25 15:39:45.605		pc=0x7f956fd7db9d
2025-01-25 15:39:45.605	non-Go function
2025-01-25 15:39:45.605		pc=0x7f956fd8920b
2025-01-25 15:39:45.605	non-Go function
2025-01-25 15:39:45.605		pc=0x7f956fd881e8
2025-01-25 15:39:45.605	non-Go function
2025-01-25 15:39:45.605		pc=0x7f956fd88958
2025-01-25 15:39:45.605	non-Go function
2025-01-25 15:39:45.605		pc=0x7f9570146883
2025-01-25 15:39:45.605	runtime.cgocall(0x61cb580, 0xc0028b9ed8)
2025-01-25 15:39:45.605		/go/pkg/mod/golang.org/[email protected]/src/runtime/cgocall.go:157 +0x4b fp=0xc0028b9eb0 sp=0xc0028b9e78 pc=0x20a1eab
2025-01-25 15:39:45.605	github.com/milvus-io/milvus/internal/querynodev2/segments._Cfunc_DeleteLoadIndexInfo(0x7f92f6c60300)
2025-01-25 15:39:45.605		_cgo_gotypes.go:353 +0x3f fp=0xc0028b9ed8 sp=0xc0028b9eb0 pc=0x5f0a37f
2025-01-25 15:39:45.605	github.com/milvus-io/milvus/internal/querynodev2/segments.deleteLoadIndexInfo.func1.1(0x7f9566c70060?)
2025-01-25 15:39:45.605		/workspace/source/internal/querynodev2/segments/load_index_info.go:65 +0x34 fp=0xc0028b9f10 sp=0xc0028b9ed8 pc=0x5f0b634
2025-01-25 15:39:45.605	github.com/milvus-io/milvus/internal/querynodev2/segments.deleteLoadIndexInfo.func1()
2025-01-25 15:39:45.605		/workspace/source/internal/querynodev2/segments/load_index_info.go:65 +0x17 fp=0xc0028b9f28 sp=0xc0028b9f10 pc=0x5f0b5d7
2025-01-25 15:39:45.605	github.com/milvus-io/milvus/pkg/util/conc.(*Pool[...]).Submit.func1()
2025-01-25 15:39:45.605		/workspace/source/pkg/util/conc/pool.go:82 +0xb3 fp=0xc0028b9f88 sp=0xc0028b9f28 pc=0x5882b73
2025-01-25 15:39:45.605	github.com/panjf2000/ants/v2.(*goWorker).run.func1()
2025-01-25 15:39:45.605		/go/pkg/mod/github.com/panjf2000/ants/[email protected]/worker.go:67 +0x8d fp=0xc0028b9fe0 sp=0xc0028b9f88 pc=0x4960c2d
2025-01-25 15:39:45.605	runtime.goexit({})
2025-01-25 15:39:45.605		/go/pkg/mod/golang.org/[email protected]/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0028b9fe8 sp=0xc0028b9fe0 pc=0x21199c1
2025-01-25 15:39:45.605	created by github.com/panjf2000/ants/v2.(*goWorker).run in goroutine 204769
2025-01-25 15:39:45.605		/go/pkg/mod/github.com/panjf2000/ants/[email protected]/worker.go:48 +0x5c

Expected Behavior

No response

Steps To Reproduce

Milvus Log

No response

Anything else?

No response

@chyezh chyezh added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jan 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.
Projects
None yet
Development

No branches or pull requests

2 participants