Fix docs #56
xUnit Tests ❌
❌ src/FastBertTokenizer.Tests/TestResults/test-results-0.4.26.5664.trx
1582 tests were completed in 46s with 1575 passed, 6 failed and 1 skipped.
Test suite | Passed | Failed | Skipped | Time |
---|---|---|---|---|
FastBertTokenizer.Tests.CompareBatched | 1✅ | 20s | ||
FastBertTokenizer.Tests.CompareToHuggingfaceTokenizer | 1572✅ | 1⚪ | 19s | |
FastBertTokenizer.Tests.LoadTokenizer | 2✅ | 6❌ | 331ms |
❌ FastBertTokenizer.Tests.LoadTokenizer
✅ LoadBaaiBgeTokenizerFromJson(path: "data/baai-bge-small-en/tokenizer.json")
❌ LoadBaaiBgeTokenizerFromJson(path: "data/bert-base-chinese/tokenizer.json")
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-chinese/tokenizer.json'.
❌ LoadBaaiBgeTokenizerFromJson(path: "data/bert-base-multilingual-cased/tokenizer.json")
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-multilingual-cased/tokenizer.json'.
❌ LoadBaaiBgeTokenizerFromJson(path: "data/bert-base-uncased/tokenizer.json")
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-uncased/tokenizer.json'.
✅ LoadTokenizerFromVocab(path: "data/baai-bge-small-en/vocab.txt", lowercase: True)
❌ LoadTokenizerFromVocab(path: "data/bert-base-chinese/vocab.txt", lowercase: True)
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-chinese/vocab.txt'.
❌ LoadTokenizerFromVocab(path: "data/bert-base-multilingual-cased/vocab.txt", lowercase: True)
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-multilingual-cased/vocab.txt'.
❌ LoadTokenizerFromVocab(path: "data/bert-base-uncased/vocab.txt", lowercase: True)
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-uncased/vocab.txt'.
Annotations
Check failure on line 29 in src/FastBertTokenizer/BertTokenizer.LoadTokenizerJson.cs
github-actions / xUnit Tests
FastBertTokenizer.Tests.LoadTokenizer ► LoadBaaiBgeTokenizerFromJson(path: "data/bert-base-chinese/tokenizer.json")
Failed test found in:
src/FastBertTokenizer.Tests/TestResults/test-results-0.4.26.5664.trx
Error:
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-chinese/tokenizer.json'.
Raw output
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-chinese/tokenizer.json'.
at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirError)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode, Func`4 createOpenException)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, UnixFileMode openPermissions, Int64& fileLength, UnixFileMode& filePermissions, Func`4 createOpenException)
at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
at System.IO.File.OpenRead(String path)
at FastBertTokenizer.BertTokenizer.LoadTokenizerJsonAsync(String tokenizerJsonFilePath) in /_/src/FastBertTokenizer/BertTokenizer.LoadTokenizerJson.cs:line 29
at FastBertTokenizer.Tests.LoadTokenizer.LoadBaaiBgeTokenizerFromJson(String path) in /home/runner/work/FastBertTokenizer/FastBertTokenizer/src/FastBertTokenizer.Tests/LoadTokenizer.cs:line 16
--- End of stack trace from previous location ---
Check failure on line 29 in src/FastBertTokenizer/BertTokenizer.LoadTokenizerJson.cs
github-actions / xUnit Tests
FastBertTokenizer.Tests.LoadTokenizer ► LoadBaaiBgeTokenizerFromJson(path: "data/bert-base-multilingual-cased/tokenizer.json")
Failed test found in:
src/FastBertTokenizer.Tests/TestResults/test-results-0.4.26.5664.trx
Error:
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-multilingual-cased/tokenizer.json'.
Raw output
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-multilingual-cased/tokenizer.json'.
at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirError)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode, Func`4 createOpenException)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, UnixFileMode openPermissions, Int64& fileLength, UnixFileMode& filePermissions, Func`4 createOpenException)
at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
at System.IO.File.OpenRead(String path)
at FastBertTokenizer.BertTokenizer.LoadTokenizerJsonAsync(String tokenizerJsonFilePath) in /_/src/FastBertTokenizer/BertTokenizer.LoadTokenizerJson.cs:line 29
at FastBertTokenizer.Tests.LoadTokenizer.LoadBaaiBgeTokenizerFromJson(String path) in /home/runner/work/FastBertTokenizer/FastBertTokenizer/src/FastBertTokenizer.Tests/LoadTokenizer.cs:line 16
--- End of stack trace from previous location ---
Check failure on line 29 in src/FastBertTokenizer/BertTokenizer.LoadTokenizerJson.cs
github-actions / xUnit Tests
FastBertTokenizer.Tests.LoadTokenizer ► LoadBaaiBgeTokenizerFromJson(path: "data/bert-base-uncased/tokenizer.json")
Failed test found in:
src/FastBertTokenizer.Tests/TestResults/test-results-0.4.26.5664.trx
Error:
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-uncased/tokenizer.json'.
Raw output
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-uncased/tokenizer.json'.
at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirError)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode, Func`4 createOpenException)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, UnixFileMode openPermissions, Int64& fileLength, UnixFileMode& filePermissions, Func`4 createOpenException)
at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
at System.IO.File.OpenRead(String path)
at FastBertTokenizer.BertTokenizer.LoadTokenizerJsonAsync(String tokenizerJsonFilePath) in /_/src/FastBertTokenizer/BertTokenizer.LoadTokenizerJson.cs:line 29
at FastBertTokenizer.Tests.LoadTokenizer.LoadBaaiBgeTokenizerFromJson(String path) in /home/runner/work/FastBertTokenizer/FastBertTokenizer/src/FastBertTokenizer.Tests/LoadTokenizer.cs:line 16
--- End of stack trace from previous location ---
Check failure on line 34 in src/FastBertTokenizer/BertTokenizer.LoadVocab.cs
github-actions / xUnit Tests
FastBertTokenizer.Tests.LoadTokenizer ► LoadTokenizerFromVocab(path: "data/bert-base-chinese/vocab.txt", lowercase: True)
Failed test found in:
src/FastBertTokenizer.Tests/TestResults/test-results-0.4.26.5664.trx
Error:
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-chinese/vocab.txt'.
Raw output
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-chinese/vocab.txt'.
at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirError)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode, Func`4 createOpenException)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, UnixFileMode openPermissions, Int64& fileLength, UnixFileMode& filePermissions, Func`4 createOpenException)
at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
at System.IO.StreamReader.ValidateArgsAndOpenPath(String path, Encoding encoding, Int32 bufferSize)
at System.IO.StreamReader..ctor(String path)
at FastBertTokenizer.BertTokenizer.LoadVocabularyAsync(String vocabTxtFilePath, Boolean convertInputToLowercase, String unknownToken, String clsToken, String sepToken, String padToken, NormalizationForm normalization) in /_/src/FastBertTokenizer/BertTokenizer.LoadVocab.cs:line 34
at FastBertTokenizer.Tests.LoadTokenizer.LoadTokenizerFromVocab(String path, Boolean lowercase) in /home/runner/work/FastBertTokenizer/FastBertTokenizer/src/FastBertTokenizer.Tests/LoadTokenizer.cs:line 27
--- End of stack trace from previous location ---
Check failure on line 34 in src/FastBertTokenizer/BertTokenizer.LoadVocab.cs
github-actions / xUnit Tests
FastBertTokenizer.Tests.LoadTokenizer ► LoadTokenizerFromVocab(path: "data/bert-base-multilingual-cased/vocab.txt", lowercase: True)
Failed test found in:
src/FastBertTokenizer.Tests/TestResults/test-results-0.4.26.5664.trx
Error:
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-multilingual-cased/vocab.txt'.
Raw output
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-multilingual-cased/vocab.txt'.
at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirError)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode, Func`4 createOpenException)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, UnixFileMode openPermissions, Int64& fileLength, UnixFileMode& filePermissions, Func`4 createOpenException)
at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
at System.IO.StreamReader.ValidateArgsAndOpenPath(String path, Encoding encoding, Int32 bufferSize)
at System.IO.StreamReader..ctor(String path)
at FastBertTokenizer.BertTokenizer.LoadVocabularyAsync(String vocabTxtFilePath, Boolean convertInputToLowercase, String unknownToken, String clsToken, String sepToken, String padToken, NormalizationForm normalization) in /_/src/FastBertTokenizer/BertTokenizer.LoadVocab.cs:line 34
at FastBertTokenizer.Tests.LoadTokenizer.LoadTokenizerFromVocab(String path, Boolean lowercase) in /home/runner/work/FastBertTokenizer/FastBertTokenizer/src/FastBertTokenizer.Tests/LoadTokenizer.cs:line 27
--- End of stack trace from previous location ---
Check failure on line 34 in src/FastBertTokenizer/BertTokenizer.LoadVocab.cs
github-actions / xUnit Tests
FastBertTokenizer.Tests.LoadTokenizer ► LoadTokenizerFromVocab(path: "data/bert-base-uncased/vocab.txt", lowercase: True)
Failed test found in:
src/FastBertTokenizer.Tests/TestResults/test-results-0.4.26.5664.trx
Error:
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-uncased/vocab.txt'.
Raw output
System.IO.DirectoryNotFoundException : Could not find a part of the path '/home/runner/work/FastBertTokenizer/FastBertTokenizer/bin/FastBertTokenizer.Tests/Release/net7.0/data/bert-base-uncased/vocab.txt'.
at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirError)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode, Func`4 createOpenException)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, UnixFileMode openPermissions, Int64& fileLength, UnixFileMode& filePermissions, Func`4 createOpenException)
at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
at System.IO.StreamReader.ValidateArgsAndOpenPath(String path, Encoding encoding, Int32 bufferSize)
at System.IO.StreamReader..ctor(String path)
at FastBertTokenizer.BertTokenizer.LoadVocabularyAsync(String vocabTxtFilePath, Boolean convertInputToLowercase, String unknownToken, String clsToken, String sepToken, String padToken, NormalizationForm normalization) in /_/src/FastBertTokenizer/BertTokenizer.LoadVocab.cs:line 34
at FastBertTokenizer.Tests.LoadTokenizer.LoadTokenizerFromVocab(String path, Boolean lowercase) in /home/runner/work/FastBertTokenizer/FastBertTokenizer/src/FastBertTokenizer.Tests/LoadTokenizer.cs:line 27
--- End of stack trace from previous location ---