Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: use --slurp with --paginate to concat pages into valid json #820

Merged
merged 1 commit into from
Jan 28, 2025

Conversation

ldelossa
Copy link
Contributor

@ldelossa ldelossa commented Jan 25, 2025

Fixes: #819

The original arguments to gh api for certain commands are problematic.

For instance:

@@ -85,13 +95,14 @@ end
 function PullRequest:get_changed_files(callback)
   local url = string.format("repos/%s/pulls/%d/files", self.repo, self.number)
   gh.run {
-    args = { "api", "--paginate", url, "--jq", "." },

In the above arguments the returned results, if exceeds one pages, will be two separate JSON arrays. E.g. [{...}][{...}].

You cannot feed this directly to vim.fn.jsondecode since its not actually valid json, its two json arrays in a single string.

Luckily, gh includes the --slurp argument to the api command which works similiarly to jq's slurp command, concating sequential arrays.

We can use this, with one catch.
The results of any API call with --slurp is "an array of pages" and thus the data we want is nested one level deep.

We simply write a merge_pages function which extracts all items into a 1d array, which is what Octo currently expects.

Describe what this PR does / why we need it

Does this pull request fix one issue?

Describe how you did it

Describe how to verify it

Special notes for reviews

Checklist

  • Passing tests and linting standards
  • Documentation updates in README.md and doc/octo.txt

Fixes: pwntester#819

The original arguments to `gh api` for certain commands are problematic.

For instance:
@@ -85,13 +95,14 @@ end
 function PullRequest:get_changed_files(callback)
   local url = string.format("repos/%s/pulls/%d/files", self.repo, self.number)
   gh.run {
-    args = { "api", "--paginate", url, "--jq", "." },

In the above arguments the returned results, if exceeds one pages, will
be two separate JSON arrays. E.g. [{page1..}][{page2...}].

You cannot feed this directly to `vim.fn.jsondecode` since its not
actually valid json, its two json arrays in a single string.

Luckily, `gh` includes the `--slurp` argument to the `api` command which
works similiarly to `jq`'s slurp command, concating sequential arrays.

We can use this, with one catch.
The results of any API call with `--slurp` is "an array of pages" and
thus the data we want is nested one level deep.

We simply write a `merge_pages` function which extracts all items into
a 1d array, which is what Octo currently expects.

Signed-off-by: ldelossa <[email protected]>
@ldelossa
Copy link
Contributor Author

hmm, this could be seen as a breaking change, but only if you're using a gh cli older then when the --slurp argument was introduced: cli/cli#8620

I'll look for advice on how to handle that.

@wd60622
Copy link
Collaborator

wd60622 commented Jan 25, 2025

Thanks for the PR @ldelossa

Let me take a look at the code. As for the gh cli version, --slurp is being used in many places. I wouldn't concern about that

Copy link
Collaborator

@wd60622 wd60622 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if this function already exists

@wd60622
Copy link
Collaborator

wd60622 commented Jan 28, 2025

This ready to go? @ldelossa

@wd60622 wd60622 added the bug Something wrong in the code label Jan 28, 2025
@ldelossa
Copy link
Contributor Author

ldelossa commented Jan 28, 2025

As a quick fix yes. I would like to make aggregate_pages work so we dont have this one off function like you mentioned, but I can do this as a follow up PR if you think thats good.

@wd60622
Copy link
Collaborator

wd60622 commented Jan 28, 2025

As a quick fix yes. I would like to make aggregate_pages work so we dont have this one off function, but I can do this as a follow up PR if you think thats good.

Perfect. I will create an issue for this. There is no rush

Copy link
Collaborator

@wd60622 wd60622 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @ldelossa !

@wd60622 wd60622 merged commit 1d2982f into pwntester:master Jan 28, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something wrong in the code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GH API CLI commands with "--paginate" cannot be fed directly into vim.fn.jsondecode
2 participants