Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OpenAI compatible Chat module #1278

Merged
merged 3 commits into from
Dec 13, 2023
Merged

Conversation

rmackay9
Copy link
Contributor

@rmackay9 rmackay9 commented Dec 9, 2023

This adds a "chat" module that can interface with an OpenAI chatGPT-4 Assistant and control an Ardupilot vehicle (Copter, Plane, Rover, Sub). Note that running this requires either using my OpenAI API key or you must have your own and then run the provided setup_assistant.py script.

Below are some screenshots showing a typical interaction
chat-screen
rtl-home
console

There are some known issue we may want to resolve before merging:

  • add MAVProxy package dependency on "openai"
  • how should we deal with the need for the OpenAI API key? ArduPilot will likely fund at least the core devs for now.
  • allow more permanent storing of OpenAI API key
  • reduce debug output to console
  • remove mavlink xml files from "assistant_setup" directory
  • improve voice recording (currently always 5sec, should always "listen" or use threshold to turn off)

Corresponding MAVProxy wiki PR ArduPilot/ardupilot_wiki#5651

Some references:

@stephendade
Copy link
Contributor

Very cool! I'd be interested if it could help users diagnose any issues with their configuration. Something like "why is my copter not arming"

@rmackay9 rmackay9 force-pushed the chat-module branch 2 times, most recently from c18a86b to 1cd4e89 Compare December 11, 2023 11:24
@tpwrules
Copy link

tpwrules commented Dec 11, 2023

I think this is a neat demo (though it does have its own problems) but I don't think the additional requirements justify including it into the official build. (I think you forgot some too). This would mean that every SITL run, every distro package, every CI invocation, etc. will need the additional packages. It's a virtue that this is very lightweight.

Is there a supported framework for external modules?

tridge
tridge previously requested changes Dec 11, 2023
requirements.txt Outdated Show resolved Hide resolved
@tridge
Copy link
Contributor

tridge commented Dec 11, 2023

Is there a supported framework for external modules?

there is, but its a bit clunky

Copy link
Contributor

@peterbarker peterbarker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So cool!

@peterbarker peterbarker merged commit 034c6e0 into ArduPilot:master Dec 13, 2023
1 check passed
@ShiauweiZhao
Copy link

cool

@yatink-ci
Copy link

I guess we are half close to being ironmen now. lol, Thank you All :D

@rmackay9 rmackay9 deleted the chat-module branch December 13, 2023 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants