Skip to content

Commit

Permalink
- Ignore gh-pages leftovers when switching branches.
Browse files Browse the repository at this point in the history
- Update instructions, comments, readme.
- Added CodeQL review action for javascript.
  • Loading branch information
raycardillo committed Mar 19, 2022
1 parent 22d07fc commit d03cf5f
Show file tree
Hide file tree
Showing 9 changed files with 137 additions and 75 deletions.
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,8 @@
# Misc
.DS_Store
*.log
*.swp

# gh-pages branch leftovers
_site
Gemfile.lock
6 changes: 6 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Contributing
============

Get in touch if you're interested in contributing, but you would need to sign a [Contributor License Agreement](https://en.wikipedia.org/wiki/Contributor_License_Agreement) before any contributions could be considered.

More generally, you agree that any contributions or correspondence are donated to this project's maintainers. If you don't agree to that then do not communicate with us or submit any code. However, that's not sufficient to protect the open source license when more significant contributions are submitted as Pull Requests, etc.
108 changes: 75 additions & 33 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,77 @@
# automatic-gbg-texture-creator

Automatic Game Builder Garage Texture Creator.

This is based upon some great work that has been done by **VideoDojo** community
members (mainly **Scrubz** and **Borri** to my knowledge).
There are two main parts to the solution: **Image Analysis** and **Mouse Contoller**.

Their solution was developed on an **Arduino Leonardo** board, but I had some spare
**Seeeduino XIAO** boards sitting around and wanted to use them for this purpose.
They're inexpensive but very capable tiny little microprocessor boards.
However, it did not turn out to be as simple a task as I originally thought because
it requires the use of a special TinyUSB library so I almost had to start over.
See the `agbgtc_tinyusb` description below. I may add more variants later.

### Image Analysis
The image analysis utility is required to create the texture data CSV representing the color of each pixel.

Right now, I've just copied their latest HTML helper utils for convenience here.
If you want to see the latest they're working on, see the VideoDojo YouTube
channel or visit the [VideoDojo community Discord](https://discord.com/channels/851450528944357437/941124085595910174)
and look at the pinned messages for the latest developments.

### Mouse Controller
The mouse controller program simulates mouse clicks required to create the textures.

- **[agbgtc_tinyusb](./agbgtc_tinyusb)** - This is a variant I created with the
intention of supporting any TinyUSB device but it currently depends on the
`Adafruit_TinyUSB` library (version `0.10.5`). It also includes a special mouse
utility helper that I created specifically for this project.

## Useful References:
- USB HID related specifications
- https://www.usb.org/sites/default/files/documents/hid1_11.pdf
- https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf
Automatic Game Builder Garage Texture Creator

[Visit the GitHub page instead](https://raycardillo.github.io/automatic-gbg-texture-creator/) if you just want to **get going quickly**. It includes a quick overview, a YouTube video demonstrating the process (and includes product links), and has everything you need to get started quickly all in one place for convenience.

<figure>
<img alt="AGBGTC Examples" src="./images/AGBGTC-Examples.jpg">
<figcaption style="font-style:italic;font-weight:300;">Photo Credit: Ray Cardillo / LongToothGamer</figcaption>
</figure>

-----

_If you want more details about the internals, or if you're a Software Developer, then this README is the best place to start._

-----

## Background
_This is based upon some great work that has been done by **VideoDojo** community members (mainly **Scrubz** and **Borri** to my knowledge). Their solution was originally developed for an **Arduino Leonardo** board._

I had some spare **Seeeduino XIAO** boards sitting around and the original code does not work for TinyUSB based boards. I love the tiny Arduino variants because they're inexpensive and very capable little microprocessors, but they are not "standard" Arduino boards so you have to use special board libraries for them. Also, there aren't as many special purpose utility helper libraries available.

I spent many hours understanding their solution, understanding the limitations of the TinyUSB devices, experimenting, and investigating options. Ultimately, because of various aspects of device compatibility and library availability, I was able to develop a TinyUSB variant of the mouse controller program with a special TinyUSB mouse utility helper class. See my [other research](#other-research) at the bottom of the page if you're interested in other improvements I experimented with.

## Solution Space
The are basically two parts of the solution. First, the image must be converted and pixel texture data extracted. Next, the pixel data is used by the mouse controller to draw a Texture Nodon.

### Image Extraction Utilities
The image extraction utilities convert an image into pixel data that is used by the mouse controller program. It produces a CSV file that is copy and pasted into the mouse controller program.

With permission from **Scrubz**, I have made the latest versions of the two most popular tools in the `utils` directory for convenience. If you want to see the latest they're working on, then connect with them on the [VideoDojo community Discord](https://discord.com/channels/851450528944357437/941124085595910174). The latest versions are in the pinned posts in `Community Texture Tools` under `help-and-tutorials` messages.

#### Pixel Data Format
The extracted data is simply a series of 8-bit (`uint8_t`) values representing colors that are supported in the Texture Nodon editor palette. The first color is the "most common color" and can be used for an initial bucket fill operation or a similar shortcut approach. Texture Nodons are `64` pixels square, so there are `4096` values that follow (`64` rows x `64` columns of pixels) and each represent the pixel color at that location.

| INDEX | 00 | 01 | ... | 64 | ... | N |
| ----- | -- | ------- | --- | ------- | --- | ----------------------------- |
| COLOR | BG | (00,00) | ... | (63,00) | ... | ( (N-1) % 64) , (N-1) / 64) ) |


### Mouse Controller Program
The mouse controller program is uploaded to a microprocessor board which is then connected to a Nintendo Switch to simulate mouse movements over the USB port. The program goes through the pixel data and sends mouse movements and clicks to automatically create Texture Nodons. Not only is this faster than manually creating them, but it's more accurate and less frustrating as well.

#### Seeeduino XIAO / TinyUSB
Follow the instructions in the **[agbgtc_tinyusb](./agbgtc_tinyusb)** directory (or watch the YouTube video from the GitHub page link above). In theory, it should work on any TinyUSB device but in practice it currently depends on the `Adafruit_TinyUSB` library (version `0.10.5`). I also had to create a special mouse utility helper specifically for this project. I've currently only tested it on the Seeeduino XIAO board.

<figure>
<img alt="Seeeduino XIAO" src="./images/Seeeduino-XIAO.jpg" style="width:400px;height:300px;">
<figcaption style="font-style:italic;font-weight:300;">Photo Credit: Ray Cardillo / LongToothGamer</figcaption>
</figure>

#### Arduino Leonardo
Visit the [VideoDojo community Discord](https://discord.com/channels/851450528944357437/941124085595910174) and see the pinned posts in `Community Texture Tools` under `help-and-tutorials` for more information about **Borri**'s version.

<figure>
<img alt="Arduino Leonardo" src="./images/Arduino-Leonardo.jpg" style="width:400px;height:300px;">
<figcaption style="font-style:italic;font-weight:300;">Photo Credit: Becky Stern - Flickr: Arduino Leonardo</figcaption>
</figure>

-----

## Other Research
While working on this project, I pursued a couple other ideas, but ran out of time and interest in pursuing them further. Capturing some notes here for future reference in case someone else wants to contribute.

### Absolute Mouse Position
I experimented with sending custom HID mouse reports that use absolute mouse positioning instead of a series of delta position updates. I still have the code laying around. The key to this solution is that the mouse reports have to indicate they are ABSOLUTE and not RELATIVE updates. However, as a result of that, it requires (2) 16-bit position parameters instead of (2) 8-bit delta position parameters.

The code actually worked well on my Mac but did not work at all on the Switch. Apparently it is considered non-standard so the Nintendo Switch mouse driver did not recognize the data properly.

### Digitizer HID Devices
I also experimented with sending "digitizer" input instead of "mouse" input by simulating a stylus or touchpad. However, I was unable to get that working and lost interest in pursuing that approach. I think the reason was that most drivers require a fairly robust implementation of several types of input, and I was only trying to implement the bare minimum for this project.

If we can determine what type of devices the Switch supports (and the specific protocol it expects) the speed could be drastically improved by sending direct touch point reports instead of delta mouse movements and clicks (and the extra delay that the drivers require to recognize the location and click reports).

### USB HID Specifications
- https://www.usb.org/sites/default/files/documents/hid1_11.pdf
- https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf
49 changes: 26 additions & 23 deletions agbgtc_tinyusb/README.md
Original file line number Diff line number Diff line change
@@ -1,39 +1,42 @@
# Usage Notes
# Seeeduino XIAO Mouse Controller

This was specifically developed for and tested on a **Seeeduino XIAO** microprocessor board.
It is currently dependent upon the **Adafruit_TinyUSB** library, but should be easy to adapt to any **TinyUSB** based environment.
This variant was specifically developed for and tested on a **Seeeduino XIAO** microprocessor board.

It is currently dependent upon the **Adafruit_TinyUSB** library, but should be easy to adapt to any **TinyUSB** based environment in the future.

## Setup and Build:

Abbreviated instructions for installing and setting up Arduino IDE **v2.0.0-rc5** or greater for XIAO development.
Abbreviated instructions for installing and setting up Arduino IDE **v2.0.0-rc5** or greater for development.

1. Install Arduino IDE `v2.0.0-rc5` (`v1.8.x` may also work but you will not see the `src` folder helper class).
1. Install [Arduino IDE](https://www.arduino.cc/en/software) `v2.0.0-rc5` or greater (`v1.8.x` may also work but you will not see the `src` folder that contains the mouse helper utility class).
1. Copy this `agbgtc_tinyusb` folder into your `Arduino` project folder.
1. Open the `agbgtc_tinyusb.ino` file in the Arduino IDE. Notice that the `src` files must be retained in tact because there are relative references to it.
1. Open the `agbgtc_tinyusb.ino` file in the Arduino IDE. Notice that the `src` files must be retained in tact because there are relative references to it in the program.
1. Add the [Seeed Studio](https://wiki.seeedstudio.com/Seeeduino-XIAO/) board package definitions using `File ⮕ Preferences` and adding the URL: `https://files.seeedstudio.com/arduino/package_seeeduino_boards_index.json`
1. Download the `Seeeduino` board drivers using `Tools ⮕ Board Manager`.
1. Download the `Adafruit_TinyUSB` version `0.10.5` using `Tools ⮕ Manage Libraries`. You must select version `0.10.5` until Seeeduino releases an update due to a breaking change.
1. Download the `Adafruit_TinyUSB` version `0.10.5` using `Tools ⮕ Manage Libraries`. You must select version `0.10.5` until Seeeduino releases an update (due to a breaking change in the `1.x` release).

## Convert Image Data
1. _You can skip this step and verify your initial setup using the default example images instead._
1. Use the `GBG_Texture_Builder` utility to convert your image to image data CSV.
1. Copy the CSV numbers into the image data array in the program.
1. Delete any image data you don't want.

## Upload Program to XIAO Board
1. Plugin the device to the computer and select the port.
1. Select the `Seeeduino XIAO` *Board* and *Port* using `Tools` menu.
1. Select the `TinyUSB` stack using `Tools ⮕ USB Stack`.
1. Upload this program to the board.
1. Unplug the XIAO board
_For your first run, you can skip this step and use the default example texture. This allows you to verify your initial setup before creating your own textures._
1. Use the `GBG_Texture_Builder` utility to convert your image to a CSV representing the texture pixel data.
1. Copy the CSV numbers into the image data array (`images[][IMAGE_DATA_LEN]`) in the program. If you're on a Mac the `pbcopy` utility is a quick way to copy the CSV data into your pasteboard (e.g., `pbcopy < file.csv` and then `⌘ Command + V` to paste).
1. Delete any example image data that you don't want.
1. I've created more than 10 images in one session so it's just limited to the amount of `PROGMEM` that is available on your board (and how long you want to wait).

## Upload Program to Board
1. Plugin the device to the computer.
1. Select the `Seeeduino XIAO` **Board** and **Port** using the `Tools` menu.
1. Select the `TinyUSB` **Stack** using `Tools ⮕ USB Stack`.
1. Upload the program to the board.
1. Unplug the XIAO board from the computer.

## Running on Device
1. Plug the XIAO Board into the Nintento Switch USB port.
1. Bring up a Game Builder Garage game in programming mode.
1. Trigger or jump the start pin to start running (pin 2 by default or can be jumped to a switch). Use the **3.3V** output pin for this because the inputs can only tolerate 3.3V.
1. Do not disturb the mouse or touch screen while it's working.
1. Unplug when complete.
1. Plug the Board into the Nintendo Switch USB-C port.
1. Bring up a Game Builder Garage game and open programming mode.
1. Trigger (or jump the start pin) to start creating the textures. By default this is pin 2. You can change this later or use a breadboard with a switch, etc. Be careful to only use **3.3V** to jump the input pin signal because the inputs on this board can only tolerate 3.3V even though they provide a 5v converter pin as well.
1. The light will stay solid while it's creating textures. Do not disturb the mouse or touch screen while it's working.
1. Unplug when complete and the LED is alternating.

## Useful References:
- https://www.arduino.cc/en/software
- https://wiki.seeedstudio.com/Seeeduino-XIAO/
- https://github.com/adafruit/Adafruit_TinyUSB_Arduino/releases/tag/0.10.5
Loading

0 comments on commit d03cf5f

Please sign in to comment.