Response detections #7

steinitzu · 2024-03-20T13:39:43Z

So far addresses some of the points in #5

Detect payload schema by comparing response schema with all other endpoints: https://github.com/dlt-hub/dlt-init-openapi/blob/response-detections/openapi_python_client/parser/responses.py#L32

Works pretty well so far, but found some cases where it picks the wrong path
Some improvements detecting list/single item endpoints. Lot of that comes from 1.
Currently have basic detection for offset and cursor pagination. Only using property name and types for this atm, looking for the least nested property matching "cursor", etc. https://github.com/dlt-hub/dlt-init-openapi/blob/response-detections/openapi_python_client/parser/pagination.py#L1
Transformers with multiple path params supported. Only when all path params resolve to corresponding properties in parent schema
Fix resource/transformer arguments order, remove pagination args

steinitzu added 18 commits March 19, 2024 15:45

Attempts to detect the payload path of a response

2417f32

Payload detect hacks

0c37672

Fix ranking, fix find transformer param, pokeapi test

67ca2ff

Fix root path

1c63412

Handle multiple path params in transformers

b13024f

Remove bundled rest client

20401bf

Fixed dependency

2e417c3

Detect cursor and offset pagination

88918c0

Todo pagination in template

84aebd1

paging client data selector

b6616b1

Fix transformer params finder

c9e8458

Fix nullable types in params, transformer arg order

ebb859f

offset and cursor paginator code gen

816be61

Fix resource arguments

84b5dc0

Detect primay key, slightly improve payload detect and table name

0875eaf

Primary payload with pk

1282484

Schema name from component ref only, update dlt, cleanup

f8ac60f

Func generate endpoint info for prompts

482958f

Provide feedback