libcrypt-2-*.12.so not found during auditwheel repair

[myselfhimself]

Hello,
This was possibly dealt with in #305 on this tracker, though I do not understand how to fix/mitigate the issue.
In the manylinux2010_x86_64 python 3.7 auditwheel repair step after building a cpython/c++ binding module wheel, I have the following log:
https://github.com/dtschump/gmic-py/runs/358967116

..with especially this non-funny failing auditwheel repair linking error:

  File "/usr/local/bin/auditwheel", line 10, in <module>
    sys.exit(main())
  File "/opt/_internal/cpython-3.7.5/lib/python3.7/site-packages/auditwheel/main.py", line 50, in main
    rval = args.func(args, p)
  File "/opt/_internal/cpython-3.7.5/lib/python3.7/site-packages/auditwheel/main_repair.py", line 83, in execute
    update_tags=args.UPDATE_TAGS)
  File "/opt/_internal/cpython-3.7.5/lib/python3.7/site-packages/auditwheel/repair.py", line 101, in repair_wheel
    needed = elf_read_dt_needed(path)
  File "/opt/_internal/cpython-3.7.5/lib/python3.7/site-packages/auditwheel/elfutils.py", line 16, in elf_read_dt_needed
    raise ValueError('Could not find soname in %s' % fn)
ValueError: Could not find soname in .libsgmic/libcrypt-2-a5d5a997.12.so
gmic-2.8.1.dev0-cp34-cp34m-linux_x86_64.whl

I am already pulling the :latest Docker image tag.
Should I yum install libxcrypt or libcrypt-compat or so first?
Should I consider manylinux2014 instead? (my project does not plan to support very old linuxes for now).

Here is the original issue link on our side:
https://github.com/dtschump/gmic-py/issues/9

Thank you for your help :)

[myselfhimself]

Using manylinux2014 does not help, with the following auditwheel repair error:

 auditwheel repair wheelhouse/gmic-2.8.1.dev1-cp35-cp35m-linux_x86_64.whl --plat manylinux2014_x86_64 -w /io/wheelhouse/
INFO:auditwheel.main_repair:Repairing gmic-2.8.1.dev1-cp35-cp35m-linux_x86_64.whl
usage: auditwheel [-h] [-V] [-v] command ...
auditwheel: error: cannot repair "wheelhouse/gmic-2.8.1.dev1-cp35-cp35m-linux_x86_64.whl" to "manylinux2014_x86_64" ABI because of the presence of too-recent versioned symbols. You'll need to compile the wheel on an older toolchain.
gmic-2.8.1.dev1-cp35-cp35m-linux_x86_64.whl
gmic-2.8.1.dev1-cp36-cp36m-linux_x86_64.whl
gmic-2.8.1.dev1-cp37-cp37m-linux_x86_64.whl
gmic-2.8.1.dev1-cp38-cp38-linux_x86_64.whl

The manylinux2014 build/repair log is here :
https://github.com/dtschump/gmic-py/runs/360777824

[myselfhimself]

I have removed the dependency on libcurl so libcrzpt is not linked any more and the auditwheel repair operation continues and succeeds

[myselfhimself]

Closing, anyone in need may reopen this.

[dralley]

I believe the issue is that libcurl inside this environment is dynamically linked against libcrypt.so.1

Installed:
  libcurl-devel.x86_64 0:7.29.0-54.el7_7.2                                      

Dependency Updated:
  curl.x86_64 0:7.29.0-54.el7_7.2       libcurl.x86_64 0:7.29.0-54.el7_7.2      

Complete!
+ ls -la /lib64/
+ grep libcurl
lrwxrwxrwx.  1 root root      16 Feb 24 18:39 libcurl.so -> libcurl.so.4.3.0
lrwxrwxrwx.  1 root root      16 Feb 24 18:39 libcurl.so.4 -> libcurl.so.4.3.0
-rwxr-xr-x.  1 root root  439320 Feb  4 17:00 libcurl.so.4.3.0
+ ldd /lib64/libcurl.so.4
+ grep libcrypt
        libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f41742d8000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f4173428000)

(Taken from inside the manylinux 2014 docker image)

So as a packager, I can't avoid this problem without avoiding libcurl or building it from source myself such that it links against libcrypt.so.2.

When I run ldd on the .so inside my own package on Fedora, the host system, it correctly shows that libcrypt.so.2 is being loaded. In the docker environment, it shows libcrypt.so.1. But because it's just a transitive dependency, it doesn't "break" on either system, because my library doesn't care about libcrypt, just libcurl, and it's the same on both systems.

This makes sense because libcrypt doesn't show as a direct dependency via. readelf.

[myselfhimself]

This is now much clearer to me thank you. So... compiling from source would be the way possibly.. po 24. 2. 2020 v 19:37 odesílatel Daniel Alley <[email protected]> napsal:

…

I believe the issue is that libcurl inside this environment is dynamically linked against libcrypt.so.1 Installed: libcurl-devel.x86_64 0:7.29.0-54.el7_7.2 Dependency Updated: curl.x86_64 0:7.29.0-54.el7_7.2 libcurl.x86_64 0:7.29.0-54.el7_7.2 Complete! + ldd /lib64/libcurl.so.4 + grep libcrypt libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f41742d8000) libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f4173428000) (Taken from inside the manylinux 2014 docker image) So as a packager, I can't avoid this problem without avoiding libcurl or building it from source. The .so inside my own package, which was built inside the docker image, shows different results for ldd when I run ldd inside the docker image, or on the Fedora host system. Because on my host system, libcurl dynamically links against libcrypt.so.2. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#411?email_source=notifications&email_token=AAJU5QXT7PMF7PDFEDNR6C3REQHXLA5CNFSM4J6MKBD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMZBDBQ#issuecomment-590483846>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJU5QQ2CBATT6I44AZJMMDREQHXLANCNFSM4J6MKBDQ> .

[dralley]

Hopefully this will resolve the problem properly: pypa/auditwheel#230

[myselfhimself]

Should I reopen this issue until pypa/auditwheel#230 is resolved?

[dralley]

That would be reasonable, but do whatever you like!

[vapier]

i hit this kind of error when trying to build a wheel that utilized libcrypt.so.1. i traced it back to Centos 6's glibc shipping a libcrypto.so.1 that patchelf does not handle correctly and ends up corrupting it in a way that tools get confused when they try to process the ELF later on.

details of the patchelf bug can be found at NixOS/patchelf#197.

this impacts the manylinux2010 container only. the manylinux2014 container doesn't seem to have a weird libcrypto.so.1 file.

i hacked around it by slightly modifying the librypt.so.1 in the image. this makes patchelf pick the correct section header table when it rewrites the soname.

$ printf '\x1a' | dd of=/lib64/libcrypt-2.12.so count=1 bs=1 conv=notrunc seek=$((0x9f68))

still experimenting with this.