Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Real world ball and cup environment #139

Merged
merged 15 commits into from
Nov 13, 2023
Merged

Real world ball and cup environment #139

merged 15 commits into from
Nov 13, 2023

Conversation

nkumar-bdai
Copy link

@nkumar-bdai nkumar-bdai commented Nov 3, 2023

This PR is a start on a real world BallAndCupStickyTable environment. Some notes:

  1. Since placement needs to be super precise, we really want to get the position of the cup to be as accurate as possible. For this to happen, we need to observe the cup from one of the cameras right before we go to place (depth readings tend to be super unreliable from far away). The main thing this PR does is change around a bunch of the operators/predicates to enable this.
  2. This env has different objects, etc. than the base env, so I copied the entire base map and made a new folder with a different metadata file. Probably redundant/wasteful, so maybe we'll want to do something nicer.

Rough edges with this PR/things to note:

  1. currently, the robot always keeps a forward orientation when picking the ball; this isn't great
  2. Sometimes, the position of the bowl is ~50cm off, because of a faulty depth reading from far away. This makes the robot believe the bowl isn't on the table, and that returns a planning failure issue (since the bowl is neither on the floor, nor on the table). I fixed this by making the table HUGE, but this is definitely a hack
  3. The robot sometimes fails to see the ball inside the bowl when it succeeds: probably will need to adjust the pose of the final look action.

Next steps:

  1. Figure out how to pick up the bowl after it falls on the floor and reliably put it back on the table
  2. Get a reasonably-working oracle
  3. Modify domain so that planning graph is fully closed
  4. Figure out how we want to parameterize all our samplers
  5. Start running experiments, and deal with deal with all rough edges that prevent us from getting good results
  6. Profit

@nkumar-bdai nkumar-bdai self-assigned this Nov 3, 2023
@nkumar-bdai nkumar-bdai marked this pull request as ready for review November 3, 2023 22:32
Copy link

@tsilver-bdai tsilver-bdai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Overall, I'm pleased with the relatively small number of changes, considering that it's a whole new real robot environment. Great work!

@nkumar-bdai nkumar-bdai merged commit 277d746 into master Nov 13, 2023
@nkumar-bdai nkumar-bdai deleted the spot-ball-and-cup branch November 13, 2023 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants