Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
NickTiny authored Dec 18, 2022
1 parent c2e0b32 commit 4f49a5e
Showing 1 changed file with 57 additions and 10 deletions.
67 changes: 57 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,68 @@
# This is an Work-in-progress fork of "Blender Rhubarb Lip Sync". Find [the original repo here](https://github.com/scaredyfish/blender-rhubarb-lipsync)
# Blender Rhubarb 2D Lipsync

Rhubarb Lip Sync is a command-line tool created by Daniel S. Wolf that automatically creates mouth animation from voice recordings. You can use it for characters in computer games, in animated cartoons, or in any other project that requires animating mouths based on existing recordings.

---------------------------------------
# Demo with GPencil Time Offset Modifier
![update-rhubarb](https://user-images.githubusercontent.com/86638335/172741045-61a8ed1d-a696-464a-a17b-3aca41205e08.gif)
Blender Rhubarb Lipsync is an addon for Blender that integrates Rhubarb Lip Sync and uses it to generate mouth-shape keyframes from a pose library.

For support using this addon in Blender, please report an issue at
https://github.com/NickTiny/blender-rhubarb-2d-lipsync/issues

# Installation
Download a release from https://github.com/NickTiny/blender-rhubarb-2d-lipsync/releases. If you download or clone the repository, the Rhubarb Lip Sync executable and data files will be missing - they can be downloaded separately from the Rhubarb Lip Sync repository, and set up as described below under Usage.

# Pre-Release Usage
Use version `4.0.0` with `Blender 3.0+`.

1. Install [latest release](https://github.com/NickTiny/blender-rhubarb-lipsync/releases/download/v4.0.0/blender-rhubarb-lipsync.4.0.0-ALPHA-Win10.zip)
2. Open the example.blend in [Test-Assets.zip](https://github.com/NickTiny/blender-rhubarb-lipsync/files/8865822/Test-Assets.zip)
3. Set a GPencil with a time offset or an Armature with a int prop as active
4. Set prop/modifier name using "Load Properties" dropdown
5. Hit "Rhubarb Lipsync" to run operation
Do not unzip the file.

In Blender, open Blender Preferences Edit -> Preferences select Add-ons and choose Install.... In the file dialog, select the .zip file. Once installed, enable the add-on with the checkbox.

# Usage:

![time_offset_demo](https://user-images.githubusercontent.com/86638335/208316400-8b3fd323-1936-4a85-909d-82fdf4af38d3.gif)


**Set Up Object**

For Time Offset types. Create a grease pencil object with the mouth shapes described in the Rhubarb Lip Sync documentation. You can set your poses on whatever frame you like. Add a Time Offset Modifier in Fixed mode to your grease pencil object.


**Set Executable**

Blender Rhubarb Lipsync includes the Rhubarb Lip Sync executable, but if you want to use a different executable, you can choose it in user preferences.

You can also set the recognizer here. PocketSphinx is recommended for English language, phonetic may give better results for other languages.

![image](https://user-images.githubusercontent.com/86638335/208315480-a42398e9-e0e2-4417-b5b8-14f2eed6b149.png)

**Select a Target Type**

- Object will keyframe an integer/float property on the Object's Data.
- Time Offset will directly keyframe a Time Offset Modifier's Frame Offset.
- Bone will keyframe an interger or float property on the active Pose Bone.

![image](https://user-images.githubusercontent.com/86638335/208315884-ff24dcd6-a558-4084-a57f-ebb63755d2fe.png)


**Set Target's Property**

Select the name of your Grease Pencil's Time Offset Modifier under Properties. Or the name of an integer/float property on the object/bone.

![image](https://user-images.githubusercontent.com/86638335/208316229-bff658a7-fdec-40e6-b501-8572d6bf4a13.png)

**Assign Mouth Values**

Set integers to match Rhubarb Lip Sync mouth shapes. Select your sound file, and dialog file (optional), and the start frame where your sound begins.

![image](https://user-images.githubusercontent.com/86638335/208316842-d28c383c-5115-4ede-9a79-682456620f4c.png)


**Run Rhubarb LipSync**

Click the Rhubarb LipSync button and wait for the process to complete. The mouse cursor will change to a progress indicator, and your keyframes will appear when the process is complete.


# Troubleshooting
This software is pre-release and provided without support. In the event of problems, you can use the system console (Window->Toggle System Console on Windows, or start Blender from a command line on Mac/Linux) to get more info on progress and error messages. When reporting an issue, please include any errors reported here.

------------------
This is a fork of "Blender Rhubarb Lip Sync". Find [the original repo here](https://github.com/scaredyfish/blender-rhubarb-lipsync)

0 comments on commit 4f49a5e

Please sign in to comment.