Web form for submitting new sites

[saqimtiaz]

To facilitate the process for contributors adding their site to TiddlyWikLinks, we should consider a form that generates a PR to add the given site to sites.json

I allowed myself to get distracted and spend some time looking at this yesterday with a Google form that writes to a Google Sheet. It is fairly straight forward with one sticky area: how to remember between Github workflow runs which rows in the Google sheet are new since the previous run.

I tested a few different alternative Github workflows:

Option 1 (with the problem of identifying new rows unsolved)

• make the Google sheet public
• in a cron based workflow run in Github actions, fetch the JSON data from the sheet over HTTP
• run a nodejs based script to update sites.json
• Use a workflow action to create a PR

The problem here is how to remember between runs which rows have already been processed without checking temporary files into the repo.

Option 2

• A script in Google Sheets to mirror new rows to a separate worksheet, so that we can modify it and have the original as a log.
• Use a Github action with read & write access to the Google sheet to remove rows already processed.
  • requires a Google service account that has access to the sheet (and nothing else) with the relevant secrets stored in Github repository settings as secrets.
• run a nodejs script to update sites.json
• Use a workflow action to create a PR

Works but a few too many moving parts, could be refined.

Option 3

• Use a script in Google sheets which triggers on new form submissions and does a POST request to trigger a Github workflow using repository_dispatch event.
  • Requires that you create a Github personal access token with repo scope (read and write) and provide that to the script in Sheets.
• The POST request provides the new form submission data and it is processed using a nodejs script to create a new PR.

This is the cleanest in terms of flow but I am uncomfortable saving that PAT in the Google script with regards to security.

[Jermolene]

Thanks @saqimtiaz.

The problem here is how to remember between runs which rows have already been processed without checking temporary files into the repo.

The TiddlyWiki5 build scripts commit progressive changes to the jermolene.github.io repository (rather than as is normal completely overwriting the existing content). We may be able to use the same technique.

It's annoying that options are all so messy, particularly because I'd like to be able to make more of techniques like this. While we could write a little Lambda to do the necessary, I wonder if we might be better off using an off-the-shelf service that's tuned for this kind of thing:

https://zapier.com/apps/github/integrations/google-forms

[saqimtiaz]

It's annoying that options are all so messy, particularly because I'd like to be able to make more of techniques like this. While we could write a little Lambda to do the necessary, I wonder if we might be better off using an off-the-shelf service that's tuned for this kind of thing:

https://zapier.com/apps/github/integrations/google-forms

Yes that is worth considering. I saw the third party service options but was hoping to find a good alternative that avoided such a dependency. However none of the alternatives seem clean or robust enough as you have noted.

[saqimtiaz]

It occurred to me today that there is a way to resolve the security issue with option 3 above.

1. Create a new user account (machine account) and fork the TiddlyWikiLinks repository in that user account.
2. Have a Google Sheet script that has a PAT for the machine account, and does a POST request on every form submission to trigger a workflow for that fork creating a new branch with the submitted data incorporated. (The branch is created in the machine account's fork).
3. In the main TiddlyWiki links repository have a cron workflow that looks for new branches in the machine account fork that do not have corresponding PRs in the upstream repository, and creates PRs for them.

I tested the various parts briefly today and it seems the premise is sound.

[Jermolene]

Hi @saqimtiaz that sounds like it should work, very cunning.

[linonetwo]

Why not use a tiddlyhost with public access (I'm not sure if it can do this), and you can fetch the HTML and filter tiddlers from it...Just like how links site works.

The only requirement is a public editable tiddlyhost site.