Skip to content

Commit

Permalink
* Changed type of truncated result error from `StreamExecuteScanQuery…
Browse files Browse the repository at this point in the history
…` to retryable error
  • Loading branch information
asmyasnikov committed Sep 7, 2022
1 parent 510ab4b commit 11cfae5
Show file tree
Hide file tree
Showing 5 changed files with 23 additions and 12 deletions.
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
* Marked the truncated result as retryable error
* Changed type of truncated result error from `StreamExecuteScanQuery` to retryable error
* Added closing sessions if node removed from discovery results
* Moved session status type from `table/options` package to `table`
* Changed session status source type from `uint32` to `string` alias
Expand Down
6 changes: 3 additions & 3 deletions MIGRATION_v2_v3.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,8 +146,8 @@

## About truncated result <a name="truncated"></a>

Call of `session.Execute` may returns a result with a flag `Truncated` because `YDB` have a default limit of rows is a 1000.
In this case query must be changed for supporting pagination. Trucated flag in result must be checks explicitly.
Call of `session.Execute` may return a result with a flag `Truncated` because `YDB` have a default limit of rows is a 1000.
In this case query must be changed for supporting pagination. Truncated flag in result must be checks explicitly.
- in `v2`:
```go
var res *table.Result
Expand Down Expand Up @@ -178,7 +178,7 @@ In this case query must be changed for supporting pagination. Trucated flag in r
}
```
- in `v3`:
By default, truncated result wraps as non-retryable error
By default, truncated result wraps as non-retryable error for `session.Execute` and retryable error for `session.StreamExecuteScanQuery`
```go
import (
"github.com/ydb-platform/ydb-go-sdk/v3/table/result"
Expand Down
6 changes: 6 additions & 0 deletions internal/table/scanner/result.go
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,12 @@ func WithIgnoreTruncated(ignoreTruncated bool) option {
}
}

func WithMarkTruncatedAsRetryable() option {
return func(r *baseResult) {
r.scanner.markTruncatedAsRetryable = true
}
}

func NewStream(
recv func(ctx context.Context) (*Ydb.ResultSet, *Ydb_TableStats.QueryStats, error),
onClose func(error) error,
Expand Down
20 changes: 12 additions & 8 deletions internal/table/scanner/scanner.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,16 +23,17 @@ import (
"github.com/ydb-platform/ydb-go-sdk/v3/table/types"
)

var errTruncated = xerrors.Retryable(errors.New("truncated result"))
var errTruncated = xerrors.Wrap(errors.New("truncated result"))

type scanner struct {
set *Ydb.ResultSet
row *Ydb.Value
converter *rawConverter
stack scanStack
nextRow int
nextItem int
ignoreTruncated bool
set *Ydb.ResultSet
row *Ydb.Value
converter *rawConverter
stack scanStack
nextRow int
nextItem int
ignoreTruncated bool
markTruncatedAsRetryable bool

columnIndexes []int

Expand Down Expand Up @@ -226,6 +227,9 @@ func (s *scanner) Err() error {
return s.err
}
if !s.ignoreTruncated && s.truncated() {
if s.markTruncatedAsRetryable {
return xerrors.WithStackTrace(xerrors.Retryable(errTruncated))
}
return xerrors.WithStackTrace(errTruncated)
}
return nil
Expand Down
1 change: 1 addition & 0 deletions internal/table/session.go
Original file line number Diff line number Diff line change
Expand Up @@ -1084,6 +1084,7 @@ func (s *session) StreamExecuteScanQuery(
return err
},
scanner.WithIgnoreTruncated(s.config.IgnoreTruncated()),
scanner.WithMarkTruncatedAsRetryable(),
), nil
}

Expand Down

0 comments on commit 11cfae5

Please sign in to comment.