More robust replace_origin

[eelco]

We generate PR previews for each commit to a PR and generally have ~25-40 PRs open. Using replace_origin which temporarily updates the URLs is therefore not really a workable strategy for us, as PRs might interfere.

[josephwynn-sc]

Thanks for opening this issue! The long-term solution to this is that we'll be adding the ability to run on-demand tests against any URL. It might be some time before this is implemented, so I'll have a think about potential short-term solutions and get back to you.

[jasikpark]

Has there been any change wrt this caveat? It looks like the action still works this way

[josephwynn-sc]

Hey @jasikpark! We have some ideas around how to improve this, but our engineering effort has been focused elsewhere. We are aiming to pick up a big CI/CD project soon, and one of our primary goals is to seamlessly support a PR-based workflow.

[jasikpark]

awesome! it's been great to see improvements per diff after setting up this action, but I'm sure it'll be better once we can ensure that we can separate deploys on main/ prod from PR deploys, similar to Chromatic testing or Percy snapshot testing