Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

patch for get_data, for when there are 0 rows to fetch and has_more tag is still present #47

Open
lag21392 opened this issue Feb 27, 2023 · 0 comments

Comments

@lag21392
Copy link

lag21392 commented Feb 27, 2023

def get_data(self, endpoint, max_recs=None, offset=0):
    """
    get data from a given endpoint
    this simplifies the looping process that would otherwise be repeated

    :param str endpoint: endpoint to be appended to bulk_base
    :param int max_recs: Max number of records to return
    """

    url_base = self.bulk_base + endpoint + '?limit={limit}&offset={offset}'

    limit, has_more = self._set_limit(max_recs, 0, True)

    return_data = []
    cant_rows = 1
    while has_more and cant_rows>0:

        url = url_base.format(limit=limit, offset=offset)
        
        req = requests.get(url=url, auth=self.auth)

        _elq_error_(req)

        if 'items' in req.json().keys():
            return_data.extend(req.json()['items'])
            cant_rows=len(req.json()['items'])
        else:
            cant_rows = 0
        offset += 1000

        if max_recs is None:

            has_more = req.json()['hasMore']

        limit, has_more = self._set_limit(max_recs, len(return_data),
                                          req.json()['hasMore'])

    return return_data
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant