Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[unitree/cross] make userを久しぶりに実行すると/bin/bashが無いと怒られて失敗する #61

Open
tkmtnt7000 opened this issue Jul 1, 2022 · 4 comments
Open

[unitree/cross] make userを久しぶりに実行すると/bin/bashが無いと怒られて失敗する #61

tkmtnt7000 opened this issue Jul 1, 2022 · 4 comments

Comments

@tkmtnt7000
Copy link
Collaborator

tkmtnt7000 commented Jul 1, 2022 •
edited
Loading

cc:@iory @a-ichikura
Linux環境でmake userを久しぶりに実行すると/bin/bashがないと出てエラーになります．
(M1 macではqemuを使わないので起こりません)
一時的な解決法としては

docker rmi multiarch/qemu-user-static
./prepare_requirements_ros1.sh # or docker pull multiarch/qemu-user-static

として，multiarch/qemu-user-staticを入れ直すと直ります．

実行時の挙動は以下のとおりです．

$ make user
./build_user.sh
WARNING: Source directory is found arm64v8_User
WARNING: It might contain old data. We recommend to remove arm64v8_User and start again
WARNING: Are you sure to continue [y/N] ? y
+ mkdir -p arm64v8_User/src/jsk_robot
+ rsync -avzh --delete --exclude 'jsk_unitree_robot/cross*' --exclude go1.l --exclude go1-simple.l ../../../jsk_robot arm64v8_User/src/
sending incremental file list
deleting jsk_robot/.git/CATKIN_IGNORE
deleting jsk_robot/.github/CATKIN_IGNORE
deleting jsk_robot/.travis/CATKIN_IGNORE
deleting jsk_robot/doc/CATKIN_IGNORE
deleting jsk_robot/jsk_aero_robot/CATKIN_IGNORE
deleting jsk_robot/jsk_baxter_robot/CATKIN_IGNORE
deleting jsk_robot/jsk_denso_robot/CATKIN_IGNORE
deleting jsk_robot/jsk_fetch_robot/CATKIN_IGNORE
jsk_robot/.git/
jsk_robot/.git/COMMIT_EDITMSG
jsk_robot/.git/index
jsk_robot/.git/logs/HEAD
jsk_robot/.git/logs/refs/heads/unitree-add-accel
jsk_robot/.git/logs/refs/remotes/tkmtnt7000/unitree-add-accel
jsk_robot/.git/modules/.travis/
jsk_robot/.git/objects/
jsk_robot/.git/objects/a6/
jsk_robot/.git/objects/a6/fafb40a73d31eb34c34e15129944c99dac4f2b
jsk_robot/.git/objects/aa/
jsk_robot/.git/objects/aa/02dbeb4cb2f89edd25344cffb2fff91b344185
jsk_robot/.git/objects/ac/
jsk_robot/.git/objects/ac/acd77435756d268356df4b017d2418c5720c28
jsk_robot/.git/objects/b5/
jsk_robot/.git/objects/b5/7954f59bcce6c582b4ea94d686274ec70b9072
jsk_robot/.git/objects/ba/
jsk_robot/.git/objects/ba/05882d40b11a7ace663e66d3defe84686b9e67
jsk_robot/.git/objects/d3/
jsk_robot/.git/objects/d3/d0ae8b5b84c7a4d4dc2b971b5975b1778e78ad
jsk_robot/.git/refs/heads/
jsk_robot/.git/refs/heads/unitree-add-accel
jsk_robot/.git/refs/remotes/tkmtnt7000/
jsk_robot/.git/refs/remotes/tkmtnt7000/unitree-add-accel
jsk_robot/.github/
jsk_robot/.travis/
jsk_robot/doc/
jsk_robot/jsk_aero_robot/
jsk_robot/jsk_baxter_robot/
jsk_robot/jsk_denso_robot/
jsk_robot/jsk_fetch_robot/
deleting jsk_robot/jsk_kinova_robot/CATKIN_IGNORE
deleting jsk_robot/jsk_magni_robot/CATKIN_IGNORE
deleting jsk_robot/jsk_naoqi_robot/CATKIN_IGNORE
deleting jsk_robot/jsk_pr2_robot/CATKIN_IGNORE
deleting jsk_robot/jsk_robot/CATKIN_IGNORE
jsk_robot/jsk_kinova_robot/
jsk_robot/jsk_magni_robot/
jsk_robot/jsk_naoqi_robot/
jsk_robot/jsk_pr2_robot/
jsk_robot/jsk_robot/
jsk_robot/jsk_robot_common/jsk_robot_startup/scripts/
jsk_robot/jsk_robot_common/jsk_robot_startup/scripts/head_teleop_joy_completion.py
jsk_robot/jsk_unitree_robot/unitreeeus/

sent 129.51K bytes  received 1.29K bytes  261.60K bytes/sec
total size is 441.19M  speedup is 3,373.07
++ find arm64v8_User/src/jsk_robot -maxdepth 1 -mindepth 1 -type d
+ for dir in $(find ${SOURCE_ROOT}/src/jsk_robot -maxdepth 1 -mindepth 1 -type d)
+ [[ ! arm64v8_User/src/jsk_robot/jsk_magni_robot =~ jsk_unitree_robot|jsk_robot_common ]]
+ touch arm64v8_User/src/jsk_robot/jsk_magni_robot/CATKIN_IGNORE
+ for dir in $(find ${SOURCE_ROOT}/src/jsk_robot -maxdepth 1 -mindepth 1 -type d)
+ [[ ! arm64v8_User/src/jsk_robot/doc =~ jsk_unitree_robot|jsk_robot_common ]]
+ touch arm64v8_User/src/jsk_robot/doc/CATKIN_IGNORE
+ for dir in $(find ${SOURCE_ROOT}/src/jsk_robot -maxdepth 1 -mindepth 1 -type d)
+ [[ ! arm64v8_User/src/jsk_robot/.travis =~ jsk_unitree_robot|jsk_robot_common ]]
+ touch arm64v8_User/src/jsk_robot/.travis/CATKIN_IGNORE
+ for dir in $(find ${SOURCE_ROOT}/src/jsk_robot -maxdepth 1 -mindepth 1 -type d)
+ [[ ! arm64v8_User/src/jsk_robot/jsk_naoqi_robot =~ jsk_unitree_robot|jsk_robot_common ]]
+ touch arm64v8_User/src/jsk_robot/jsk_naoqi_robot/CATKIN_IGNORE
+ for dir in $(find ${SOURCE_ROOT}/src/jsk_robot -maxdepth 1 -mindepth 1 -type d)
+ [[ ! arm64v8_User/src/jsk_robot/.git =~ jsk_unitree_robot|jsk_robot_common ]]
+ touch arm64v8_User/src/jsk_robot/.git/CATKIN_IGNORE
+ for dir in $(find ${SOURCE_ROOT}/src/jsk_robot -maxdepth 1 -mindepth 1 -type d)
+ [[ ! arm64v8_User/src/jsk_robot/jsk_robot =~ jsk_unitree_robot|jsk_robot_common ]]
+ touch arm64v8_User/src/jsk_robot/jsk_robot/CATKIN_IGNORE
+ for dir in $(find ${SOURCE_ROOT}/src/jsk_robot -maxdepth 1 -mindepth 1 -type d)
+ [[ ! arm64v8_User/src/jsk_robot/jsk_denso_robot =~ jsk_unitree_robot|jsk_robot_common ]]
+ touch arm64v8_User/src/jsk_robot/jsk_denso_robot/CATKIN_IGNORE
+ for dir in $(find ${SOURCE_ROOT}/src/jsk_robot -maxdepth 1 -mindepth 1 -type d)
+ [[ ! arm64v8_User/src/jsk_robot/.github =~ jsk_unitree_robot|jsk_robot_common ]]
+ touch arm64v8_User/src/jsk_robot/.github/CATKIN_IGNORE
+ for dir in $(find ${SOURCE_ROOT}/src/jsk_robot -maxdepth 1 -mindepth 1 -type d)
+ [[ ! arm64v8_User/src/jsk_robot/jsk_baxter_robot =~ jsk_unitree_robot|jsk_robot_common ]]
+ touch arm64v8_User/src/jsk_robot/jsk_baxter_robot/CATKIN_IGNORE
+ for dir in $(find ${SOURCE_ROOT}/src/jsk_robot -maxdepth 1 -mindepth 1 -type d)
+ [[ ! arm64v8_User/src/jsk_robot/jsk_unitree_robot =~ jsk_unitree_robot|jsk_robot_common ]]
+ for dir in $(find ${SOURCE_ROOT}/src/jsk_robot -maxdepth 1 -mindepth 1 -type d)
+ [[ ! arm64v8_User/src/jsk_robot/jsk_aero_robot =~ jsk_unitree_robot|jsk_robot_common ]]
+ touch arm64v8_User/src/jsk_robot/jsk_aero_robot/CATKIN_IGNORE
+ for dir in $(find ${SOURCE_ROOT}/src/jsk_robot -maxdepth 1 -mindepth 1 -type d)
+ [[ ! arm64v8_User/src/jsk_robot/jsk_robot_common =~ jsk_unitree_robot|jsk_robot_common ]]
+ for dir in $(find ${SOURCE_ROOT}/src/jsk_robot -maxdepth 1 -mindepth 1 -type d)
+ [[ ! arm64v8_User/src/jsk_robot/jsk_fetch_robot =~ jsk_unitree_robot|jsk_robot_common ]]
+ touch arm64v8_User/src/jsk_robot/jsk_fetch_robot/CATKIN_IGNORE
+ for dir in $(find ${SOURCE_ROOT}/src/jsk_robot -maxdepth 1 -mindepth 1 -type d)
+ [[ ! arm64v8_User/src/jsk_robot/jsk_pr2_robot =~ jsk_unitree_robot|jsk_robot_common ]]
+ touch arm64v8_User/src/jsk_robot/jsk_pr2_robot/CATKIN_IGNORE
+ for dir in $(find ${SOURCE_ROOT}/src/jsk_robot -maxdepth 1 -mindepth 1 -type d)
+ [[ ! arm64v8_User/src/jsk_robot/jsk_kinova_robot =~ jsk_unitree_robot|jsk_robot_common ]]
+ touch arm64v8_User/src/jsk_robot/jsk_kinova_robot/CATKIN_IGNORE
+ '[' 0 -eq 0 ']'
+ tee arm64v8_build_user.log
++ id -u tsukamoto
+ docker run -it --rm -u 1000 -e INSTALL_ROOT=System -v /home/tsukamoto/ros/unitree_ws/src/jsk-ros-pkg/jsk_robot/jsk_unitree_robot/cross/arm64v8_System/ros1_dependencies:/opt/jsk/System/ros1_dependencies:ro -v /home/tsukamoto/ros/unitree_ws/src/jsk-ros-pkg/jsk_robot/jsk_unitree_robot/cross/arm64v8_System/Python:/opt/jsk/System/Python:ro -v /home/tsukamoto/ros/unitree_ws/src/jsk-ros-pkg/jsk_robot/jsk_unitree_robot/cross/arm64v8_System/ros1_inst:/opt/jsk/System/ros1_inst:ro -v /home/tsukamoto/ros/unitree_ws/src/jsk-ros-pkg/jsk_robot/jsk_unitree_robot/cross/arm64v8_System/ros1_dependencies_setup.bash:/opt/jsk/System/ros1_dependencies_setup.bash:ro -v /home/tsukamoto/ros/unitree_ws/src/jsk-ros-pkg/jsk_robot/jsk_unitree_robot/cross/arm64v8_System/system_setup.bash:/opt/jsk/System/system_setup.bash:ro -v /home/tsukamoto/ros/unitree_ws/src/jsk-ros-pkg/jsk_robot/jsk_unitree_robot/cross/arm64v8_User:/opt/jsk/User:rw -v /home/tsukamoto/ros/unitree_ws/src/jsk-ros-pkg/jsk_robot/jsk_unitree_robot/cross/rosinstall_generator_unreleased.py:/home/user/rosinstall_generator_unreleased.py:ro ros1-unitree:arm64v8 bash -c '    source /opt/jsk/System/system_setup.bash &&     env &&     set -xeuf -o pipefail &&     cd /opt/jsk/User &&     [ 0 -eq 0 ] || ROS_PACKAGE_PATH=src:${ROS_PACKAGE_PATH} /home/user/rosinstall_generator_unreleased.py jsk_unitree_startup unitreeeus --rosdistro melodic --exclude RPP --exclude mongodb_store | tee user.repos &&     [ 0 -eq 0 -o -z "$(cat user.repos)" ] || PYTHONPATH= vcs import src < user.repos &&     catkin build jsk_unitree_startup unitreeeus -s -vi         --cmake-args -DCATKIN_ENABLE_TESTING=FALSE     '
WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64) and no specific platform was requested
exec /bin/bash: no such file or directory
Makefile:18: recipe for target 'user' failed
make: *** [user] Error 1
@k-okada
Copy link
Owner

k-okada commented Jul 3, 2022

次起こったら

/proc/sys/fs/binfmt_misc/qemu-aarch64

があるかとか

docker images -a

の結果，とかも調べておいてください。

@tkmtnt7000
Copy link
Collaborator Author

再現したので貼っておきます．
qemu-aarch64はちゃんとありそうです

$ ls /proc/sys/fs/binfmt_misc/qemu-aarch64 
/proc/sys/fs/binfmt_misc/qemu-aarch64

$ docker images -a
REPOSITORY                                 TAG                        IMAGE ID       CREATED         SIZE
ros1-unitree                               arm64v8                    89945ffd73c0   5 weeks ago     2.77GB
multiarch/qemu-user-static                 latest                     cd504ee34e34   6 weeks ago     301MB
tkmtnt7000/ubuntu-ros                      ros_melodic_cuda9.2        1aec6de092ec   4 months ago    6.51GB
ros_noetic_cuda11.3_torch1.10              latest                     2c8fd243c242   4 months ago    14.4GB
tkmtnt7000/ros_noetic_cuda11.3_torch1.10   latest                     2c8fd243c242   4 months ago    14.4GB
<none>                                     <none>                     0f9de5d9e769   4 months ago    14.4GB
<none>                                     <none>                     117a49eb5ecf   4 months ago    14.4GB
<none>                                     <none>                     ed11106bf757   4 months ago    14.4GB
<none>                                     <none>                     e70ede7373a8   4 months ago    14.4GB
<none>                                     <none>                     efbd169186de   4 months ago    14.4GB
<none>                                     <none>                     e198f2b12a3f   4 months ago    14.4GB
<none>                                     <none>                     993d06f48e93   4 months ago    14.4GB
<none>                                     <none>                     4d294c317b71   4 months ago    14.4GB
<none>                                     <none>                     159fad071c7b   4 months ago    14.4GB
<none>                                     <none>                     15531ed98210   4 months ago    14.4GB
<none>                                     <none>                     8663dc003013   4 months ago    14.4GB
<none>                                     <none>                     cca2a3a2a2d7   4 months ago    14.4GB
<none>                                     <none>                     4aa8887b76b0   4 months ago    14.4GB
<none>                                     <none>                     e4656f036332   4 months ago    14.4GB
<none>                                     <none>                     6ccbe1f9de24   4 months ago    14.4GB
<none>                                     <none>                     335394988b1a   4 months ago    14.3GB
<none>                                     <none>                     8993168083e9   4 months ago    14.3GB
<none>                                     <none>                     6c17b66c7c85   4 months ago    8.63GB
<none>                                     <none>                     90568d46361e   4 months ago    8.26GB
<none>                                     <none>                     470fb25a1a66   4 months ago    4.77GB
<none>                                     <none>                     99e0a235a141   4 months ago    4.77GB
<none>                                     <none>                     cee2899b9a57   4 months ago    4.77GB
<none>                                     <none>                     4ce30a935d76   4 months ago    4.77GB
<none>                                     <none>                     a06f2069134e   4 months ago    4.73GB
<none>                                     <none>                     e4a25984ed91   4 months ago    4.73GB
<none>                                     <none>                     8b9f2fe57eb3   4 months ago    4.73GB
<none>                                     <none>                     3520306235a0   4 months ago    4.73GB
<none>                                     <none>                     8f3b9c4c7f7f   4 months ago    4.55GB
<none>                                     <none>                     51092acdc88a   4 months ago    4.55GB
cyberbotics/webots                         latest                     4ecfa15db757   7 months ago    5.5GB
ubuntu                                     20.04                      ba6acccedd29   9 months ago    72.8MB
nvidia/cudagl                              11.3.1-devel-ubuntu20.04   3fe4ef68781b   10 months ago   4.55GB
wkentaro/labelme                           latest                     67b1864c03f4   13 months ago   848MB
hello-world                                latest                     d1165f221234   16 months ago   13.3kB
nvidia/cuda                                11.1-devel-ubuntu20.04     0185bd6d8ed9   19 months ago   4.76GB

@k-okada
Copy link
Owner

k-okada commented Jul 29, 2022

なるほど，スクリプトに

docker run -it --rm ros1-unitree:arm64v8 bash -c 'exit' ||  docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

とか追加したらいいのかな？

@tkmtnt7000
Copy link
Collaborator Author

tkmtnt7000 commented Jul 29, 2022 •
edited
Loading

追加すると良さそうでした．プルリク出します．
簡単に自分のメモも兼ねてmultiarch/qemu-user-staticの--resetオプションについて書いておくのですが，

  1. この--resetオプションをつけることで/proc/sys/fx/binfmt_misc/qemu-*を実行時に登録し直すことができる．(エラーが出たら--resetをつけてみるとよさそうという雰囲気を出した説明が書かれている．)
    https://github.com/multiarch/qemu-user-static#getting-started
    https://github.com/multiarch/qemu-user-static#multiarchqemu-user-static-images

  2. なぜ登録し直すかというと...（まだよく理解しているわけではありませんが）

  • PCをrebootすると/proc/sys/fs/binfmt_misc/qemu*が書き換わる(flags: F ==> flags: OC)
  • flags: OCとなっているときはインタプリタの実行ができない...？？(怪しい)
# /proc/sys/fs/binfmt_misc/qemu-arm
enabled
interpreter /usr/bin/qemu-arm-static
flags: OC
offset 0
magic 7f454c4601010100000000000000000002002800
mask ffffffffffffff00fffffffffffffffffeffffff

(参考: https://docs.kernel.org/admin-guide/binfmt-misc.html, https://stackoverflow.com/questions/55115177/usage-of-reset-and-register-tag-in-docker-run-command)

tkmtnt7000 added a commit to tkmtnt7000/jsk_robot that referenced this issue Jul 29, 2022
@tkmtnt7000
[unitree/cross] check binfmt_misc related to k-okada#61 (comment)
6c5e1e3
@tkmtnt7000 tkmtnt7000 mentioned this issue Jul 29, 2022
Merged
k-okada added a commit that referenced this issue Jul 29, 2022
@k-okada
Merge pull request #67 from tkmtnt7000/PR-fix-qemu
e1ba95e 
[unitree/cross] check binfmt_misc #61
iory pushed a commit to iory/jsk_robot that referenced this issue Aug 2, 2022
@tkmtnt7000 @iory
[unitree/cross] check binfmt_misc related to k-okada#61 (comment)
d3f2c70
iory pushed a commit to iory/jsk_robot that referenced this issue Sep 7, 2022
@tkmtnt7000 @iory
[unitree/cross] check binfmt_misc related to k-okada#61 (comment)
4687716
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@k-okada @tkmtnt7000