-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[REG-1579] add initial Automated Testing package docs #55
Changes from 2 commits
d5068c8
bf93195
cbdfecf
d609bfb
c46eeae
ab83e34
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,125 @@ | ||
--- | ||
sidebar_label: Getting Started | ||
--- | ||
|
||
# Getting Started with Automated Testing | ||
|
||
This quick walk-through will show you how to integrate the Regression Games Automated Testing package into your game, | ||
and how to use the built-in "Monkey Bot", | ||
abeizer marked this conversation as resolved.
Show resolved
Hide resolved
|
||
which is a bot that randomly clicks on UI elements in your game, | ||
recording state along the way. | ||
|
||
## A sample game to start with | ||
|
||
If you have a game you want to automate, you can skip this step. | ||
But if you just want to check the package out and don't have an active game project, | ||
we recommend using the Boss Room Co-Op Sample Project from Unity. | ||
You can download it from GitHub: [https://github.com/Unity-Technologies/com.unity.multiplayer.samples.coop](https://github.com/Unity-Technologies/com.unity.multiplayer.samples.coop) | ||
abeizer marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Once downloaded, open the project in Unity. | ||
Navigate to the **Scenes** folder in the Project window and open the **MainMenu** scene. | ||
Use this scene for the rest of the tutorial. | ||
|
||
 | ||
|
||
## Installing the package | ||
|
||
Our package is available at [https://github.com/Regression-Games/RegressionGames.Unity.Testing](https://github.com/Regression-Games/RegressionGames.Unity.Testing). | ||
Import it into your game by clicking **Window > Package Manager** in the Unity menu, | ||
and then clicking the **+** button in the top left corner of the Package Manager window. | ||
Select **Add package from git URL** and paste in the following URL: | ||
|
||
``` | ||
https://github.com/Regression-Games/RegressionGames.Unity.Testing.git?path=src/gg.regression.unity.testing#v0.0.1 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For testing the doc, I currently recommend using this Git URL instead:
When Regression-Games/RegressionGames.Unity.Testing#6 lands, you could also use:
To get the latest There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This needs to be the SSH link, git@github.com:Regression-Games/RegressionGames.Unity.Testing.git, rather than the https link. I found out (through my own config) that if the user only ever uses SSH (as is the case with my computer), I have not http config for git, and even for public repos, it will fail. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ohhh, I keep forgetting I have this in my global git config 🤣
(It forces any Good call, I'll post an updated URL soon. The doc is still fine though, since when the package goes live, the HTTPS URL will work because the repo will be public. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. After resolving some issues @abeizer found, please use this URL instead:
This is necessary until Regression-Games/RegressionGames.Unity.Testing#8 lands There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
We can provide the SSH URL too, but currently we only provide the HTTPS link in our docs for the existing SDK There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh my apologies! I confused myself and it was actually the reverse I experienced, duh... we need to use the https one and not the ssh one, THAT was the issue I originally had. So yes we can just keep the HTTPS one |
||
``` | ||
|
||
* **NOTE**: The package uses [TextMeshPro](https://docs.unity3d.com/Manual/com.unity.textmeshpro.html) to provide UI elements. If you are prompted by Unity to add TextMeshPro assets to your project, please add them. | ||
* **NOTE**: The package uses [Newtonsoft Json](https://docs.unity3d.com/Packages/com.unity.nuget.newtonsoft-json@3.1/manual/index.html) to serialize state for recordings. | ||
abeizer marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
 | ||
|
||
## Adding the Automation Controller and components to your scene | ||
|
||
Once the package is installed, you need to add an Automation Controller to the scene you wish to automate. | ||
analogrelay marked this conversation as resolved.
Show resolved
Hide resolved
|
||
The Automation Controller is the hub for all automation in a scene. | ||
Bots are spawned as children of the Automation Controller, | ||
and Entity Discoverers (see below) should be added as children of the Automation Controller. | ||
analogrelay marked this conversation as resolved.
Show resolved
Hide resolved
|
||
The Controller also provides APIs for spawning bots and managing recordings (via a separate, Automation Recorder component attached to the same object). | ||
|
||
To add the Automation Controller to your scene, click **GameObject > Regression Games > Automation Controller** in the Unity menu. | ||
vontell marked this conversation as resolved.
Show resolved
Hide resolved
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What is the Unity menu? I went to the Unity menu in the toolbar. Might be good to clarify right clicking on the scene object. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ah, I meant the main menu bar, but right clicking also provides the same menu. I'll clarify! |
||
|
||
 | ||
|
||
**By default**, the Automation Controller and all it's children will mark themselves as [DontDestroyOnLoad](https://docs.unity3d.com/ScriptReference/Object.DontDestroyOnLoad.html). This ensures that bots continue to run when you load a new scene in your game. You can disable this behavior by unchecking the "Dont Destroy On Load" checkbox on the Automation Controller component. | ||
analogrelay marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
 | ||
|
||
abeizer marked this conversation as resolved.
Show resolved
Hide resolved
|
||
## Adding Entity Discoverers to your scene | ||
|
||
In order to automate entities in your game, the Automation Controller needs to be able to find them. | ||
The Automation Controller uses Entity Discoverers to find entities in the scene that can be automated. | ||
analogrelay marked this conversation as resolved.
Show resolved
Hide resolved
|
||
The package provides a few built-in Entity Discoverers, but you can also create your own. | ||
To add one of the built-in discoverers, open the **GameObject > Regression Games > Discovery** menu and select the discoverer you want to add: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Here, also be sure to add that you should do this on the automation controller, unless it can be done at the top level as well (but we should at least tell them somewhere to click to get that menu) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see... this comes after the picture. I'd put it before the picture so that we avoid the scenario I just had where I didn't scroll further to see more and got stuck on that There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This "where to right click to get the menu" applies to further parts in these docs, so I'll avoid making that comment again and leave this as the last one. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
If you click that menu item without a node selected, it automatically targets the automation controller. Otherwise, it should give you a fairly clear error telling you the problem.
I didn't want to put it the image above since the text that describes the image should generally precede it. I can do that if you think it's clearer though. |
||
|
||
 | ||
|
||
As long as you have an Automation Controller in your scene, the Entity Discoverer will be **automatically** be added as a child of the Automation Controller, no matter what object is selected in the Hierarchy window. | ||
analogrelay marked this conversation as resolved.
Show resolved
Hide resolved
vontell marked this conversation as resolved.
Show resolved
Hide resolved
abeizer marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
 | ||
|
||
For this example, we recommend spawning the "UI Element Discoverer", which will find all the UI elements in your scene. | ||
abeizer marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Elements will provide various states that can be recorded, such as the opacity and interactability of a Canvas Group. | ||
Some elements provide actions that the bot can take, such as clicking on a Button. | ||
|
||
## Adding the Overlay UI | ||
|
||
For simple scenarios, we provide a UI Overlay that can be used to spawn bots in the scene. | ||
This allows you to quickly try out automations without having to write code to manage the bots. | ||
To add the Overlay UI, click **GameObject > Regression Games > UI Overlay** in the Unity menu. | ||
|
||
 | ||
|
||
The UI Overlay provides a small button in the bottom-right corner of the game. | ||
When clicked, this button will open a dialog that shows you a list of available bots that can be spawned. | ||
vontell marked this conversation as resolved.
Show resolved
Hide resolved
|
||
You can customize which bots appear in this list using the "Available Bots" property on the UI Overlay component. | ||
|
||
 | ||
|
||
The UI Overlay includes the built-in "Monkey Bot", which is a bot that randomly clicks on UI elements in your game and records a screenshot of the game after each click. | ||
|
||
**By default**, the Overlay UI will mark itself as [DontDestroyOnLoad](https://docs.unity3d.com/ScriptReference/Object.DontDestroyOnLoad.html). This allows you to retain control over starting and stopping bots when you load a new scene in your game. You can disable this behavior by unchecking the "Dont Destroy On Load" checkbox on the Overlay UI component. | ||
|
||
 | ||
|
||
## Running the Monkey Bot | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Before this or at the beginning, make sure to mention that you should save the scene There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated! |
||
|
||
Now that you have the Automation Controller, UI Element Discoverer, and Overlay UI in your scene, you can run an example with the Monkey Bot. | ||
|
||
1. Click the **Play** button in the Unity Editor to start the game. | ||
1. Click the button in the bottom-right corner of the game to open the Overlay UI. | ||
1. Choose the "Monkey Bot" from the list of available bots and click **Start Bot**. | ||
1. The overlay will close and you'll see the Monkey Bot activating UI elements randomly in your game. | ||
1. After a few seconds, open the overlay again and click **Stop** next to the Monkey Bot. | ||
|
||
TODO: GIF of Monkey Bot running | ||
|
||
## Viewing recordings | ||
|
||
Whenever you spawn a bot using the Overlay UI, a recording is automatically started. | ||
When that bot stops, the recording is automatically stopped. | ||
You can view the recordings by clicking on the "Automation Controller" object in the Hierarchy window. | ||
On the inspector, find the "Automation Recorder" component and click the **View Recordings...** button. | ||
The directory in which the recordings are stored will open in your file explorer. | ||
|
||
 | ||
|
||
 | ||
|
||
Each recording is an `.rgrec.zip` file. | ||
Inside these ZIP files are: | ||
|
||
1. A `recording.json` file which contains global metadata about the recording (when it was made, who made it, etc.) | ||
1. A series of `snapshot.[frameNumber].json` files that contain all the state captured for each frame of the recording. Frames with no state changes are not included in the recording, so if a given frame `N` is missing, it means that the state of the game did not change between frames `N-1` and `N`. | ||
1. A series of `screenshot.[frameNumber].png` files that contain a screenshot of the game for each frame that a screenshot was requested. Bots, and your game code, can use an API on the Automation Controller to request a screenshot at any time. For example, the Monkey Bot requests a screenshot for the frame immediately after it clicks on a UI element. | ||
|
||
 |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
--- | ||
sidebar_label: Overview | ||
--- | ||
|
||
# Automated Testing for Unity | ||
|
||
The Regression Games Automated Testing Package for Unity is a set of tools that help you test your game with less code. The package combines several powerful features to help you test your game: | ||
|
||
1. Automatic State Capture and Recording - Regression Games automatically discovers components in your game and allows you to make recordings of their state throughout a gameplay session. These recordings can be analyzed to determine if your game is behaving as expected. | ||
|
||
1. Simple Automation Framework - Regression Games provides a simple framework for automating gameplay, and provides built-in automations for common tasks such as navigating menus. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since right now this only encapsulate the UI-related components, I was wondering if we might call it "Automated UI Testing"? We can always change it, but I don't want it to get confused with some of our other docs and site that also claim automated testing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍🏻
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this was changed yet?