Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Use datetime.datetime.fromisoformat to parse replication keys #74

Merged
merged 1 commit into from
Apr 10, 2024
Merged

fix: Use datetime.datetime.fromisoformat to parse replication keys #74

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 42 additions & 2 deletions poetry.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,11 @@ packages = [

[tool.poetry.dependencies]
python = "<3.12,>=3.7.1"
singer-sdk = { version="^0.33.1" }
backports-datetime-fromisoformat = {version = "^2.0.1", python = "<3.11"}
cached-property = "^1" # Remove after Python 3.7 support is dropped
fs-s3fs = { version = "^1.1.1", optional = true }
requests = "^2.28.2"
cached-property = "^1" # Remove after Python 3.7 support is dropped
singer-sdk = { version="^0.33.1" }

[tool.poetry.group.dev.dependencies]
pytest = "^7.2.1"
Expand Down
9 changes: 6 additions & 3 deletions tap_hubspot/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
import sys
import requests
import datetime
import pytz

from typing import Any, Callable

Expand All @@ -23,6 +22,11 @@
from singer_sdk.streams.core import REPLICATION_INCREMENTAL
from tap_hubspot.auth import HubSpotOAuthAuthenticator

if sys.version_info < (3, 11):
from backports.datetime_fromisoformat import MonkeyPatch

MonkeyPatch.patch_fromisoformat()

_Auth = Callable[[requests.PreparedRequest], requests.PreparedRequest]


Expand Down Expand Up @@ -304,8 +308,7 @@ def prepare_request_payload(
if self._is_incremental_search(context):
# Only filter in case we have a value to filter on
# https://developers.hubspot.com/docs/api/crm/search
ts = datetime.datetime.strptime(self.get_starting_replication_key_value(context), "%Y-%m-%dT%H:%M:%S.%fZ")
ts = pytz.utc.localize(ts)
ts = datetime.datetime.fromisoformat(self.get_starting_replication_key_value(context))
if next_page_token:
# Hubspot wont return more than 10k records so when we hit 10k we
# need to reset our epoch to most recent and not send the next_page_token
Expand Down
Loading