New command to easily compare deploys or tests

[wildlyinaccurate]

What problem is this trying to solve?

It's a common scenario to want to run performance regression testing against a feature branch, to make sure it hasn't introduced any issues compared to the main branch. You can do this in the speedcurve.com web app relatively easily using the bookmark/compare feature, but it's cumbersome to do this comparison in other environments e.g. a build process.

What is the proposed solution?

All of the building blocks already exist to do this in the CLI. You could set up a site in SpeedCurve with two URLs: one for the main branch and one for feature branches:

When a new feature branch has been built to an environment, you can update the feature branch URL to point to the new environment and kick off a deploy:

speedcurve update-url 12345 https://feature-branch-b.dev.speedcurve.com/
speedcurve deploy --site 444

(It probably works better if you run the deploy command once for each URL. I think we need to add a --json flag to the deploy command so that you can more easily grab the test IDs for each deploy)

You'd then run this new command to compare the results. It could have the concept of a failure threshold, where the command exits with an error status if any metrics got worse by more than a certain percentage.

speedcurve compare-tests <mainTestId> <featureTestId> --failure-threshold 10