Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix shape and reformat free tensor handling in the input byte size check #125

Merged
merged 8 commits into from
Jul 27, 2024
Merged

Fix shape and reformat free tensor handling in the input byte size check #125

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
ad6136e
Update
pskiran1 Jul 12, 2024
1137ebd
Update flag name
pskiran1 Jul 13, 2024
1c58260
Update
pskiran1 Jul 13, 2024
a514e87
Update
pskiran1 Jul 13, 2024
3b0a96a
Fix pre-commit error
pskiran1 Jul 14, 2024
a4bfde4
Fix pre-commit error
pskiran1 Jul 14, 2024
f22e498
Update model_config.proto
pskiran1 Jul 16, 2024
6eda37b
Merge branch 'main' of https://github.com/triton-inference-server/com…
pskiran1 Jul 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions include/triton/common/triton_json.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,15 @@ class TritonJson {
return TRITONJSON_STATUSSUCCESS;
}

// Set/overwrite a boolean in a value. This changes the
// type of the value to boolean.
TRITONJSON_STATUSTYPE SetBool(const bool value)
{
rapidjson::Value& v = AsMutableValue();
v.SetBool(value);
return TRITONJSON_STATUSSUCCESS;
}

// Set/overwrite a signed integer in a value. This changes the
// type of the value to signed int.
TRITONJSON_STATUSTYPE SetInt(const int64_t value)
Expand Down
34 changes: 27 additions & 7 deletions protobuf/model_config.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -392,15 +392,15 @@ message ModelInput
//@@
bool is_shape_tensor = 6;

//@@ .. cpp:var:: bool allow_ragged_batch
//@@ .. cpp:var:: bool is_non_linear_format_io
//@@
//@@ Whether or not the input is allowed to be "ragged" in a dynamically
//@@ created batch. Default is false indicating that two requests will
//@@ only be batched if this tensor has the same shape in both requests.
//@@ True indicates that two requests can be batched even if this tensor
//@@ has a different shape in each request.
//@@ Indicates whether the input tensor uses a non-linear IO format. This
//@@ field is currently supported only for TensorRT models. An error will
//@@ be generated if this specification does not comply with the
// underlying
//@@ model.
//@@
bool allow_ragged_batch = 7;
bool is_non_linear_format_io = 7;

//@@ .. cpp:var:: bool optional
//@@
Expand All @@ -409,6 +409,16 @@ message ModelInput
//@@ Default value is false.
//@@
bool optional = 8;

//@@ .. cpp:var:: bool allow_ragged_batch
//@@
//@@ Whether or not the input is allowed to be "ragged" in a dynamically
//@@ created batch. Default is false indicating that two requests will
//@@ only be batched if this tensor has the same shape in both requests.
//@@ True indicates that two requests can be batched even if this tensor
//@@ has a different shape in each request.
//@@
bool allow_ragged_batch = 9;
pskiran1 marked this conversation as resolved.
Show resolved Hide resolved
}

//@@
Expand Down Expand Up @@ -461,6 +471,16 @@ message ModelOutput
//@@ model.
//@@
bool is_shape_tensor = 6;

//@@ .. cpp:var:: bool is_non_linear_format_io
//@@
//@@ Indicates whether the output tensor uses a non-linear IO format. This
//@@ field is currently supported only for TensorRT models. An error will
//@@ be generated if this specification does not comply with the
// underlying
//@@ model.
//@@
bool is_non_linear_format_io = 7;
}

//@@ .. cpp:var:: message BatchInput
Expand Down
Loading