Add a general-purpose autoencoder class AE to serve as a building block for other autoencoder models

[voidvoxel]

Description

Oh. My. Freaking. Goodness! There's somewhat of a "story" behind this, but here's your TL;DR:

• I tried to port my previous autoencoder implementation from @voidvoxel/auto-encoder which had a muddled commit history, as it originates from my older GitHub account. This attempt was futile, because it was a complicated implementation written outside of the brain.js codebase.
• I decided that not only would it be more time efficient to reimplement autoencoders from scratch directly within the brain.js library, but it would also improve readability as well as compatibility. This attempt was a success!
• I made sure to add a few unit tests for the new AutoEncoder class.
• I made sure to replace the temporary shallowClone method that I added to auto-encoder.ts with deepClone to support multi-dimensional input arrays.
• Wikipedia said that I'm illiterate (there is no space or hyphen in the word "autoencoder").
• OCD compelled me to rename the class Autoencoder to AE to match the other classes; feel free to revoke this change if you'd prefer the more-descriptive, yet more-verbose, Autoencoder over AE
• Add AE usage to README.md

Motivation and Context

I've been using my own home-rolled autoencoders that utilize brain.js for awhile now, so I wanted to share it with the community that made it possible to begin with 💖

How Has This Been Tested?

I added unit tests to test the basic functionality of the class. Further tests should be added in the future, which I'll be glad to contribute to 😊

To run the tests, I rebuilt and ran the test script by running the following command:

npm rebuild && npm run build && npm test

Screenshots (if appropriate):

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Author's Checklist:

• My code focuses on the main motivation and avoids scope creep.
• My code passes current tests and adds new tests where possible.
• My code is SOLID and DRY.
• I have updated the documentation as needed.

Reviewer's Checklist:

• I kept my comments to the author positive, specific, and productive.
• I tested the code and didn't find any new problems.
• I think the motivation is good for the project.
• I think the code works to satisfies the motivation.

[robertleeplummerjr]

Nice work! One of the most thorough PR's I've seen for brain.js. Can the autoencoder be used with things like strings easily such as with an LLM?

[voidvoxel]

The original implementation did have string support, but I noticed quite poor quality from the word embedding algorithm I wrote, so I'm reworking it right now. I can either add it in the next PR or I can add it to this one once its finished. The new embedding algorithm is designed to process new words that were not in the training data much more effectively, which is likely improve the quality of encodings produced by the string autoencoders by a significant amount 😊

[robertleeplummerjr]

I can either add it in the next PR or I can add it to this one once its finished.

Let's do this incrementally.

The new embedding algorithm is designed to process new words that were not in the training data much more effectively, which is likely improve the quality of encodings produced by the string autoencoders by a significant amount 😊

That has me super excited. Thank you for sharing. I'll go ahead and merge this now.

If anyone is watching this, this is how you contribute to a project.

[robertleeplummerjr]

I see I have some updates I should make prior to merging. I'll work on that next to get you in a state to merge.

[voidvoxel]

I see I have some updates I should make prior to merging. I'll work on that next to get you in a state to merge.

If you'd like any backup for delegation, feel free to ping me 💕

[robertleeplummerjr]

If you would like to handle upgrading the packages so that they run on Travis ci, by all means take a stab at it.

[voidvoxel]

If you would like to handle upgrading the packages so that they run on Travis ci, by all means take a stab at it.

On it friend 🥰

[voidvoxel]

A little weather-related hiccup required my system to be unplugged last night, but we’re back on track today 💞

[voidvoxel]

Sorry for the delays! Our house flooded in a couple of rooms and a hallway during that storm. Also, unrelated: I now have an unexpected surgery coming up. Wish me luck!

Commit a4257fd should fix the workflow errors, as they were relating to my usage of JavaScript private properties (#property syntax) instead of TypeScript private properties (private property syntax). My apologies!

[robertleeplummerjr]

I was off last week and had an injury, sorry for the delay. I hope you the best on your surgery. Looking at this now.

[robertleeplummerjr]

The inability to build in travis is unrelated to these fine changes. Let's go ahead and merge this, and the build issues can be handled elsewhere so as to prevent scope creep.

[seunafara]

Can we make AE support non-gpu Neural network? Or is there a specific reason why we are tying this to GPU NNs only?

[voidvoxel]

I'm not sure if it ever made it into mainstream or not, but I already wrote both CPU and GPU implementations of the autoencoder class (Autoencoder and AutoencoderGPU IIRC). If I forgot to create a PR, I'll make one soon 😊

Sorry for the lack of updates for awhile. Health got in the way of work, but I've mostly recovered and am back in good health so I've recently started to work full time again, so I'll try to make more progress with the autoencoder and loss function features 💖

[seunafara]

I made a PR that adds serialization (toJSON & fromJSON) to AE - #950

[seunafara]

AE as of today runs on only Neural Network GPU, do we want to pass an argument to the AE class telling it to use NeuralNetworkGPU or NeuralNetwork (cpu)?

[robert-lore]

Just for comparing benchmarks would be very interesting.