Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pydantic contrib module #757

Open
wants to merge 96 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
96 commits
Select commit Hold shift + click to select a range
5ccd08e
Copy of pydantic v1 converter from samples-python
dandavison Jan 28, 2025
afa8184
Cleanup
dandavison Feb 2, 2025
5ccd203
update pydantic to v2
dandavison Jan 30, 2025
e8b6f01
Drive-by: use zip_longest
dandavison Jan 31, 2025
33b7980
Get rid of pydantic sandbox hack
dandavison Jan 31, 2025
7af32c1
Expand test coverage
dandavison Feb 3, 2025
f071373
Implement type converter and __get_pydantic_core_schema__
dandavison Feb 3, 2025
32b74da
Rename
dandavison Feb 3, 2025
24f89de
Clean up
dandavison Feb 3, 2025
cc1d8f5
Extend test coverage
dandavison Feb 3, 2025
adea382
Add test of mixed type inputs
dandavison Feb 3, 2025
0a9e3f6
Use v2 API to_jsonable_python
dandavison Feb 3, 2025
3c0a857
Use JSONEncoder
dandavison Feb 4, 2025
b853549
Cleanup
dandavison Feb 4, 2025
4e95208
README
dandavison Feb 3, 2025
e769fe8
Retain hack for backwards compatiblity
dandavison Feb 4, 2025
aa4affb
Hack: unbreak instantiation of pydantic models
dandavison Feb 4, 2025
9e4d0a1
Test pydantic usage in workflow
dandavison Feb 4, 2025
62bdfcc
Don't restrict datetime instances
dandavison Feb 4, 2025
941065d
Only use pydantic in sandbox if it can be imported
dandavison Feb 4, 2025
bd22c07
Expand tests
dandavison Feb 4, 2025
781e080
Fix __get_pydantic_core_schema__ implementation
dandavison Feb 6, 2025
eb60fe6
TEST FAILURE: Refactor tests and use two models
dandavison Feb 6, 2025
ccdbe0f
One model
dandavison Feb 6, 2025
c1ba55b
Refactor tests
dandavison Feb 6, 2025
2811d64
test date
dandavison Feb 6, 2025
825fec4
Don't restrict date instances
dandavison Feb 6, 2025
e1da664
Test timedelta
dandavison Feb 6, 2025
d39ce6e
Revert "Test timedelta"
dandavison Feb 6, 2025
3287edf
Rename
dandavison Feb 6, 2025
5d0a3e4
test timedelta
dandavison Feb 6, 2025
c31569b
test union field
dandavison Feb 6, 2025
3f44ffa
test type hints
dandavison Feb 6, 2025
1928274
Test dunder methods
dandavison Feb 7, 2025
11ee007
Organize tests
dandavison Feb 6, 2025
dbba4ce
Test pathlib.Path
dandavison Feb 7, 2025
c92486d
Fix pathlib.Path usage
dandavison Feb 7, 2025
f52c2e8
Delete redundant fix
dandavison Feb 7, 2025
298ee64
Remove redundant isinstance check
dandavison Feb 7, 2025
d2a956a
Deduplicate tests
dandavison Feb 7, 2025
725792f
Clean up
dandavison Feb 7, 2025
7032a70
Expand
dandavison Feb 7, 2025
4005a65
Clean up
dandavison Feb 7, 2025
815fa84
Disable field types that don't work
dandavison Feb 7, 2025
aa4ad22
Expand
dandavison Feb 7, 2025
6b1bfd1
Test special instead of standard model
dandavison Feb 7, 2025
7534a7d
datetime variants
dandavison Feb 7, 2025
2f1e17e
Add date datetime
dandavison Feb 7, 2025
747454b
Time fields
dandavison Feb 7, 2025
f94acf0
Revert "Test special instead of standard model"
dandavison Feb 7, 2025
3959a83
namedtuple
dandavison Feb 7, 2025
9ec4c34
sequence field
dandavison Feb 7, 2025
8508be5
iterable field
dandavison Feb 7, 2025
3df3c92
TypedDict
dandavison Feb 7, 2025
15ba77d
Expand tests
dandavison Feb 7, 2025
9fc5d77
Always pass through typing_extensions
dandavison Feb 8, 2025
5c6e44c
Make activities generic
dandavison Feb 8, 2025
1b2b731
Revert "Make activities generic"
dandavison Feb 8, 2025
634fe80
Reduce tests
dandavison Feb 8, 2025
899b92c
Expand tests
dandavison Feb 8, 2025
be5f569
Test union fields
dandavison Feb 8, 2025
61e75af
Complex custom type
dandavison Feb 8, 2025
02d5d98
Test complex union
dandavison Feb 8, 2025
706f5f2
Rename
dandavison Feb 9, 2025
0ce76b7
Use non-list input
dandavison Feb 10, 2025
0470592
Fix lint
dandavison Feb 10, 2025
e1080c7
Reorganize
dandavison Feb 10, 2025
62b2ee5
Define some models inside the sandbox and some outside
dandavison Feb 10, 2025
b2e3487
lint
dandavison Feb 10, 2025
a544224
Warn pydantic users who are not using contrib.pydantic
dandavison Feb 10, 2025
4d109c7
Cleanup
dandavison Feb 10, 2025
90ce448
Support pydantic v1 and v2
dandavison Feb 10, 2025
be3b8f8
Drop v1 support in data converter
dandavison Feb 10, 2025
d3b525a
Drop v2 support outside contrib
dandavison Feb 10, 2025
89758fd
README
dandavison Feb 11, 2025
24d489c
Include pydantic extra
dandavison Feb 11, 2025
7e7c6d4
date instances are no longer proxied
dandavison Feb 11, 2025
3de8f28
Fix tests on Windows
dandavison Feb 11, 2025
b2f62de
doctoc
dandavison Feb 11, 2025
975a1fc
Restrict pydantic range to v2 in pyproject.toml
dandavison Feb 11, 2025
17c06ec
Fix warning message
dandavison Feb 11, 2025
d45bbcd
Failing test for strict mode
dandavison Feb 11, 2025
5cf962f
Hand over entirely to pydantic
dandavison Feb 12, 2025
b6d3336
Test round-trip misc top-level objects
dandavison Feb 12, 2025
051b003
Define special types model in the sandbox
dandavison Feb 13, 2025
b61d06e
Test strict versions of some models
dandavison Feb 13, 2025
7c7529c
Fix link
dandavison Feb 13, 2025
23c88e8
Add more misc standard lib types to test
dandavison Feb 13, 2025
b3c7d73
Failing test
dandavison Feb 13, 2025
0e13cfa
Handle absence of type hint
dandavison Feb 13, 2025
512bd7b
Allow pydantic ValidationError to be thrown out
dandavison Feb 13, 2025
e91c90d
Test ValidationError
dandavison Feb 13, 2025
f5ee416
Fixup
dandavison Feb 13, 2025
616c78f
Cleanup
dandavison Feb 13, 2025
a8fb520
Test defaultdict support
dandavison Feb 13, 2025
771de5c
Fix docstrings
dandavison Feb 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 31 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ informal introduction to the features and their implementation.
- [Usage](#usage)
- [Client](#client)
- [Data Conversion](#data-conversion)
- [Pydantic Support](#pydantic-support)
- [Custom Type Data Conversion](#custom-type-data-conversion)
- [Workers](#workers)
- [Workflows](#workflows)
Expand Down Expand Up @@ -84,7 +85,6 @@ informal introduction to the features and their implementation.
- [Extending Restricted Classes](#extending-restricted-classes)
- [Certain Standard Library Calls on Restricted Objects](#certain-standard-library-calls-on-restricted-objects)
- [is_subclass of ABC-based Restricted Classes](#is_subclass-of-abc-based-restricted-classes)
- [Compiled Pydantic Sometimes Using Wrong Types](#compiled-pydantic-sometimes-using-wrong-types)
- [Activities](#activities)
- [Definition](#definition-1)
- [Types of Activities](#types-of-activities)
Expand Down Expand Up @@ -298,10 +298,10 @@ other_ns_client = Client(**config)
#### Data Conversion

Data converters are used to convert raw Temporal payloads to/from actual Python types. A custom data converter of type
`temporalio.converter.DataConverter` can be set via the `data_converter` client parameter. Data converters are a
combination of payload converters, payload codecs, and failure converters. Payload converters convert Python values
to/from serialized bytes. Payload codecs convert bytes to bytes (e.g. for compression or encryption). Failure converters
convert exceptions to/from serialized failures.
`temporalio.converter.DataConverter` can be set via the `data_converter` parameter of the `Client` constructor. Data
converters are a combination of payload converters, payload codecs, and failure converters. Payload converters convert
Python values to/from serialized bytes. Payload codecs convert bytes to bytes (e.g. for compression or encryption).
Failure converters convert exceptions to/from serialized failures.

The default data converter supports converting multiple types including:

Expand All @@ -312,22 +312,39 @@ The default data converter supports converting multiple types including:
* Anything that [`json.dump`](https://docs.python.org/3/library/json.html#json.dump) supports natively
* [dataclasses](https://docs.python.org/3/library/dataclasses.html)
* Iterables including ones JSON dump may not support by default, e.g. `set`
* Any class with a `dict()` method and a static `parse_obj()` method, e.g.
[Pydantic models](https://pydantic-docs.helpmanual.io/usage/models)
* The default data converter is deprecated for Pydantic models and will warn if used since not all fields work.
See [this sample](https://github.com/temporalio/samples-python/tree/main/pydantic_converter) for the recommended
approach.
* [IntEnum, StrEnum](https://docs.python.org/3/library/enum.html) based enumerates
* [UUID](https://docs.python.org/3/library/uuid.html)

This notably doesn't include any `date`, `time`, or `datetime` objects as they may not work across SDKs.
To use pydantic model instances, see [](#pydantic-support).
dandavison marked this conversation as resolved.
Show resolved Hide resolved

Users are strongly encouraged to use a single `dataclass` for parameter and return types so fields with defaults can be
easily added without breaking compatibility.
`datetime.date`, `datetime.time`, and `datetime.datetime` can only be used with the Pydantic data converter.

Although workflows, updates, signals, and queries can all be defined with multiple input parameters, users are strongly
encouraged to use a single `dataclass` or Pydantic model parameter, so that fields with defaults can be easily added
without breaking compatibility. Similar advice applies to return values.

Classes with generics may not have the generics properly resolved. The current implementation does not have generic
type resolution. Users should use concrete types.

##### Pydantic Support

To use Pydantic model instances, install Pydantic and set the Pydantic data converter when creating client instances:

```python
from temporalio.contrib.pydantic import pydantic_data_converter

client = Client(data_converter=pydantic_data_converter, ...)
```

This data converter supports conversion of all types supported by Pydantic to and from JSON.

In addition to Pydantic models, these include all `json.dump`-able types, various non-`json.dump`-able standard library
types such as dataclasses, types from the datetime module, sets, UUID, etc, and custom types composed of any of these.

Pydantic v1 is not supported by this data converter. If you are not yet able to upgrade from Pydantic v1, see
https://github.com/temporalio/samples-python/tree/main/pydantic_converter/v1 for limited v1 support.


##### Custom Type Data Conversion

For converting from JSON, the workflow/activity type hint is taken into account to convert to the proper type. Care has
Expand Down Expand Up @@ -1133,15 +1150,6 @@ Due to [https://bugs.python.org/issue44847](https://bugs.python.org/issue44847),
checked to see if they are subclasses of another via `is_subclass` may fail (see also
[this wrapt issue](https://github.com/GrahamDumpleton/wrapt/issues/130)).

###### Compiled Pydantic Sometimes Using Wrong Types

If the Pydantic dependency is in compiled form (the default) and you are using a Pydantic model inside a workflow
sandbox that uses a `datetime` type, it will grab the wrong validator and use `date` instead. This is because our
patched form of `issubclass` is bypassed by compiled Pydantic.

To work around, either don't use `datetime`-based Pydantic model fields in workflows, or mark `datetime` library as
passthrough (means you lose protection against calling the non-deterministic `now()`), or use non-compiled Pydantic
dependency.

### Activities

Expand Down Expand Up @@ -1341,7 +1349,7 @@ async def check_past_histories(my_client: Client):
OpenTelemetry support requires the optional `opentelemetry` dependencies which are part of the `opentelemetry` extra.
When using `pip`, running

pip install temporalio[opentelemetry]
pip install 'temporalio[opentelemetry]'

will install needed dependencies. Then the `temporalio.contrib.opentelemetry.TracingInterceptor` can be created and set
as an interceptor on the `interceptors` argument of `Client.connect`. When set, spans will be created for all client
Expand Down
Loading
Loading