Skip to content

Latest commit

 

History

History

Tools

README.md

robot-speech-to-speech 🤖🗣️

Pacte Novation Internship Project - 2024

Tools 🛠️

In this folder, you will find the tools used in the project and theri evaluations. A tool is a module designed to run in a separated process and to perform a specific task. If you wish to have more information about the tools, please refer to the README.md file in the corresponding folder.

Tools structure

This folder is structured as follows:

Tools/
├── Evaluation/
├── ToolName/ for each tool
├── __init__.py
├── nao_ip.txt
├── parameters.yaml
├── README.md
├── start_subprocesses.py
└── utils.py
  • Evaluation/: Each tool evaluation is a study of the tool's performance and efficiency in a specific task. The evaluations are available in this folder.
  • ToolName/: Folder for each tool, containing the tool's code and documentation. (see below)
  • __init__.py: Empty file to make the folder a package.
  • nao_ip.txt: File containing the IP address of the NAO robot. (to be filled by the user, WARNING: you must add this file because it is in the .gitignore file)
  • parameters.yaml: File containing the parameters of the tools. This file is automatically generated by the start_subprocesses.py script, it is in fact copied from the StateMachine parameters file.
  • README.md: This file.
  • start_subprocesses.py: Script to start the tools as separate processes.
  • utils.py: Contains utility functions used by the tools.

Tools list

Tool Name Description Status
T0 Read Audio Reads an audio file on the NAO robot Done
T1 Person Recognition Recognizes a person in an audio file Done
T2 Text Generation Generates a text response with LLMs Done
T3 Text to Speech Converts text to speech Done
T4 Action Selection Selects an action to perform given a text input Done
T5 Perform Action Performs an action Not started
T6 Record Audio Records audio from the NAO robot Done
T7 Capture Images Captures images from the NAO robot Done
T8 Speech to Text Converts speech to text, transcription Done
T9 LEDS Controls the LEDs on the NAO robot Done
T10 Retrieve and Augment Retrieves information from some stored documents and make it a prompt Done
T11 Gesture Performs gestures on the NAO robot Done
T12 Rights Management Manages the rights of the users in terms of actions and documents Done

Each Tool structure

Each tool is structured as follows:

ToolName/
├── README.md
├── __init__.py
├── ToolName.py
├── fast_app.py
├── requirements.txt
├── venv/
└── fast_com/
    ├── README.md
    ├── command.txt
    └── status.txt
  • README.md: Contains the documentation of the tool.
  • __init__.py: Empty file to make the folder a package.
  • ToolName.py: Contains the code of the tool, where its core functionalities are implemented. (python class)
  • fast_app.py: File to run the tool as a separate process.
  • requirements.txt: Contains the dependencies of the tool, it is recommended to install them in a virtual environment for each tool.
  • venv/: Folder containing the virtual environment of the tool. (ignored by git)
  • fast_com/: Folder containing the communication files.
    • README.md: Contains the documentation of the communication protocol.
    • command.txt: File to send commands to the tool.
    • status.txt: File to get the status of the tool.