If you had a Teddy Ruxpin as a child, you probably thought the robot talking bear was amazing at first, but as you got older, the fact that the only thing the bear could do was tell cute stories made it, well, less interesting.
T-Rux is a project that aims to make Teddy, like us, WAY cooler now that we're older by creating new cassettes that let Teddy say–or sing–anything!
This project started because I wanted to fix up an old Teddy Ruxpin for my daughter, who wanted one after seeing an old commercial on YouTube. Once I got it working, I decided to try to learn how it works and to see if I could make Teddy read a book that my daughter and I read together when she was younger. The result is T-Rux – and using it is pretty simple!
Here's an example:
FIND A BOOK AND READ IT
Here's one of my daughter's voice on Teddy:
FIND A BOOK AND HAVE HER READ IT
Teddy Ruxpin worked using plain audio cassette tapes, but with the key difference that the right channel of the stereo recording held the audio track, while the left channel held Pulse Position Modulation (PPM) data that was decoded into motor movements by the hardware inside the bear. If you've ever played a Teddy Ruxpin cassette in a tape deck and heard the horrifying sound coming from the left speaker, you know what I mean. That's the sound of PPM servo data…or nightmares!
Originally, the PPM motor control data was generated by puppeteers using joysticks. (PPM is the same technology used to control RC airplanes, so the tech has been around for a long time.) The goal of T-Rux, though, is to replicate something that looks similar to a puppeteer-created movement, but to do it automatically by analyzing the recorded audio waveform from any source. It's not the same exact look, but it's pretty darn close without having to be or know a puppeteer.
T-Rux uses Python 3. Be sure to install all prerequisites (numpy, scipy) via pip.
python t-rux.py inputfile.wav outputfile.wav
To use T-Rux, you'll need: • A tape recorder • A cassette tape (You might want a splicing kit, too unless you plan on making a 60 or 90 minute long recording. Teddy doesn't have a rewind button.) • A USB to RCA interface (something that supports a high bitrate is best) • Audio editing software (Audacity, Adobe Audition, etc.) • A Teddy Ruxpin (currently this is only tested with the original 3-servo Teddy. It probably works with the 2-servo version, too, since the tapes were interchangeable.)
You can record your source audio however you want. I used a home recording setup that I have, but you could use your phone. The important thing is ultimately end up with a WAV file with your voice.
When you run T-Rux, it analyzes the amplitude of the waveform in the audio file and uses the relative loudness of your recording to make the mouth move. To get the most dramatic mouth movements, it's important that your recording is loud with lots of variability between the loudest and quietest parts. There are advanced techniques listed below to get the best performance.
(Teddy's eyes are also controlled by PPM data, and T-Rux simply introduces a 1 second blink at random intervals to simulate blinking, otherwise the bear gets a little creepy looking!)
The output of T-Rux is an audio file with the PPM data encoded inside that you'll use as the left channel of your final recording.
The last step is to build a new stereo audio file in your audio editor. In this file, you put the audio track you want to hear in the right channel, and the PPM motor control data in the left channel. Then, you record to tape, rewind, and pop it into Teddy! (If you want it on both sides, record Side A, let it run for a few seconds after, then stop the tape, flip it over to Side B, and record again. That way you can just flip the tape over in Teddy instead of rewinding or playing it through.)
I've found people are either creeped out by this or think it's really cool. Watching their reactions is awesome in both situations!
To get the best motion effects, I created a "mask" sound file just to generate the PPM data with more exaggerated movements. I used Adobe Audition's Match Loudness effect and set the target loudness to -6 LUFS, then went through and muted the areas where I didn't want the mouth to move at all. This file isn't the best sounding, but we're only using it to generate the motor commands. When combining the tracks on to the cassette, I used the original recording with the exaggerated PPM data generated by T-Rux.
Making Teddy read is one thing, but making Teddy sing any song is another! The way T-Rux works would have Teddy's mouth moving at ANY sound –- not just voice -- so we have to find a way to separate the vocals from the instrumental part of a song. That's where Spleeter comes in. (There are others, but I've only used Spleeter.)
The process, essentially, is to use Spleeter to analyze a song and output a file that just contains the vocals. This file will take some heavy editing to clean up well enough for use in Teddy, but once you've got just the parts you want Teddy to say isolated, you can save your edited vocal track and use it as a "mask" (just like above) to generate the PPM data using T-Rux. Once you have the PPM data, you simply record it with the original vocal track to the cassette.
Here is an example of Teddy singing Mr. Roboto –- the creepiest song I could think of making a robot teddy bear that was only meant to sing lullabies sing:
MR ROBOTO VIDEO
And I couldn't let this go without some Snoop Dogg:
SNOOP DOGG VIDEO
- The higher bitrate file you can supply, the better. If you can, try for 192000bps.
- The output ppm file will always be 192000bps to ensure the highest amount of accuracy
- Remember - after running t-rux, you'll still need to create an audio file that combines the source file and the PPM filetwo. Place the original audio voice recording in the right channel, and the PPM file in the left channel.
#todo
- generate finished wav file from code
- at the very least, make output.wav single channel. Doing that breaks the waveform somehow, so for now I just strip the empty channel and assemble the new audio track in Audition. (inputfile.wav on left ch, ppm channel of outputfile.wav on the right).