Error installing on python 3.9?

[rderidder-lda]

Hello.. maybe i'm missing a dependency? Sorry.. I'm not incredibly great at environment setup..
I had this up and running, in my python 3.8 environment, and just tried to move to 3.9.
..I'm assuming the error ' error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/ ' is a bit of a red herring (i did try and install it anyway, but didn't make any difference)..
I couldn't find any install document on dedupe.io 's site that mentioned anything special.

Could it be related to my pycharm IDE / any other dependency the package has? I found a few similar issues online, but none related to python 3.9..

Collecting dedupe==2.0.13
Using cached dedupe-2.0.13.tar.gz (70 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: numpy>=1.13 in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from dedupe==2.0.13) (1.22.3)
Collecting Levenshtein-search==1.4.5
Using cached Levenshtein_search-1.4.5.tar.gz (7.8 kB)
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Collecting dedupe-hcluster
Using cached dedupe_hcluster-0.3.9-cp39-cp39-win_amd64.whl (170 kB)
Collecting haversine>=0.4.1
Using cached haversine-2.5.1-py2.py3-none-any.whl (6.1 kB)
Requirement already satisfied: typing-extensions in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from dedupe==2.0.13) (4.1.1)
Requirement already satisfied: zope.index in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from dedupe==2.0.13) (5.1.0)
Requirement already satisfied: rlr>=2.4.3 in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from dedupe==2.0.13) (2.4.6)
Collecting doublemetaphone
Using cached DoubleMetaphone-1.1-cp39-cp39-win_amd64.whl (28 kB)
Collecting categorical-distance>=1.9
Using cached categorical_distance-1.9-py3-none-any.whl (3.3 kB)
Requirement already satisfied: BTrees>=4.1.4 in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from dedupe==2.0.13) (4.10.0)
Collecting fastcluster
Using cached fastcluster-1.2.6-cp39-cp39-win_amd64.whl (36 kB)
Collecting dedupe-variable-datetime
Using cached dedupe_variable_datetime-0.1.5-py3-none-any.whl (4.8 kB)
Collecting affinegap>=1.3
Using cached affinegap-1.12-cp39-cp39-win_amd64.whl (16 kB)
Requirement already satisfied: simplecosine>=1.2 in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from dedupe==2.0.13) (1.2)
Collecting highered>=0.2.0
Using cached highered-0.2.1-py2.py3-none-any.whl (3.3 kB)
Requirement already satisfied: persistent>=4.1.0 in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from BTrees>=4.1.4->dedupe==2.0.13) (4.9.0)
Requirement already satisfied: zope.interface>=5.0.0 in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from BTrees>=4.1.4->dedupe==2.0.13) (5.4.0)
Requirement already satisfied: pyhacrf-datamade>=0.2.0 in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from highered>=0.2.0->dedupe==2.0.13) (0.2.6)
Requirement already satisfied: pylbfgs in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from rlr>=2.4.3->dedupe==2.0.13) (0.2.0.14)
Requirement already satisfied: future in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from dedupe-variable-datetime->dedupe==2.0.13) (0.18.2)
Requirement already satisfied: datetime-distance in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from dedupe-variable-datetime->dedupe==2.0.13) (0.1.3)
Requirement already satisfied: six in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from zope.index->dedupe==2.0.13) (1.16.0)
Requirement already satisfied: setuptools in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from zope.index->dedupe==2.0.13) (57.0.0)
Requirement already satisfied: cffi in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from persistent>=4.1.0->BTrees>=4.1.4->dedupe==2.0.13) (1.15.0)
Requirement already satisfied: python-dateutil>=2.6.0 in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from datetime-distance->dedupe-variable-datetime->dedupe==2.0.13) (2.8.2)
Requirement already satisfied: pycparser in c:\users\xxx\pycharmprojects\adm-backend\venv\lib\site-packages (from cffi->persistent>=4.1.0->BTrees>=4.1.4->dedupe==2.0.13) (2.21)

**

Building wheels for collected packages: dedupe, Levenshtein-search
Building wheel for dedupe (pyproject.toml): started
Building wheel for dedupe (pyproject.toml): finished with status 'error'
Building wheel for Levenshtein-search (setup.py): started
Building wheel for Levenshtein-search (setup.py): finished with status 'error'
Running setup.py clean for Levenshtein-search
Failed to build dedupe Levenshtein-search

**

error: subprocess-exited-with-error

Building wheel for dedupe (pyproject.toml) did not run successfully.
exit code: 1

[42 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.9
creating build\lib.win-amd64-3.9\dedupe
copying dedupe\api.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe\backport.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe\blocking.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe\canonical.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe\canopy_index.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe\clustering.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe\convenience.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe\core.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe\datamodel.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe\index.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe\labeler.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe\levenshtein.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe\predicates.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe\sampling.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe\serializer.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe\tfidf.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe\training.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe_init.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe_typing.py -> build\lib.win-amd64-3.9\dedupe
copying dedupe_init_.py -> build\lib.win-amd64-3.9\dedupe
creating build\lib.win-amd64-3.9\dedupe\variables
copying dedupe\variables\base.py -> build\lib.win-amd64-3.9\dedupe\variables
copying dedupe\variables\categorical_type.py -> build\lib.win-amd64-3.9\dedupe\variables
copying dedupe\variables\exact.py -> build\lib.win-amd64-3.9\dedupe\variables
copying dedupe\variables\exists.py -> build\lib.win-amd64-3.9\dedupe\variables
copying dedupe\variables\interaction.py -> build\lib.win-amd64-3.9\dedupe\variables
copying dedupe\variables\latlong.py -> build\lib.win-amd64-3.9\dedupe\variables
copying dedupe\variables\price.py -> build\lib.win-amd64-3.9\dedupe\variables
copying dedupe\variables\set.py -> build\lib.win-amd64-3.9\dedupe\variables
copying dedupe\variables\string.py -> build\lib.win-amd64-3.9\dedupe\variables
copying dedupe\variables_init_.py -> build\lib.win-amd64-3.9\dedupe\variables
warning: build_py: byte-compiling is disabled, skipping.

**

running build_ext
building 'dedupe.cpredicates' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
[end of output]

**

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for dedupe
error: subprocess-exited-with-error

python setup.py bdist_wheel did not run successfully.
exit code: 1

[5 lines of output]
running bdist_wheel
running build
running build_ext
building 'Levenshtein_search' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for Levenshtein-search
ERROR: Could not build wheels for dedupe, which is required to install pyproject.toml-based projects

[rderidder-lda]

in case it makes a difference, i am using numpty 1.22.3
should i downgrade that version to something like 1.19? (just saw some hints on other issues that maybe this is it)

[fgregg]

it doesn't have anything to do with numpy, as the pieces that are not building don't depend on numpy.

what is the output if you install the Microsoft Visual C++ 14.0 tools?

[rderidder-lda]

Thanks for getting back to me! appreciate the time.

To be precise, I ran the installation using "visualcppbuildtools_full.exe" which I downloaded from MS.
Inside that installation application, I selected 2 components.

I restarted the machine and reran the dedupe install. I get the identical output as above.

What is more curious is that it ran with python 3.8 fine.. So am I missing something around the fact that this Microsoft install needs to somehow be referenced in the specific virtual environment that runs in my IDE?
I use pycharm.. and change my interpreter from python 3.8 to 3.9.
Re install all libraries.
but this one is failing. I did have to upgrade Flask to the latest for it to work.
And I upgraded pip itself to the latest just incase.

I suppose this is specific to the Levenshtien-search library, and not dedupe itself. I have tried just installing that library only, and I get the same error..
Thanks for any tips at all...
R

[rderidder-lda]

finding https://wiki.python.org/moin/WindowsCompilers#Microsoft_Visual_C.2B-.2B-_14.2_standalone:_Build_Tools_for_Visual_Studio_2019_.28x86.2C_x64.2C_ARM.2C_ARM64.29
i selected a couple more components and restarted.

Same error appears. I'm thinking since it worked for python 3.8, i shouldn't have to install anything outside the venv just because i changed to python 3.9.. strange.

[fgregg]

you need to use the same compiler that was used to compile your version of python. that might have changed between 3.8 and 3.9

[rderidder-lda]

lol.. i'm a bit of a newbie.. i'm reading that sentence over and over trying to understand it.. my 'compiler' is pycharm.. in pycharm i can switch between 2 interpreters being used (python 3.8 and 3.9). When i'm in 3.8, the environment has all the libraries installed and working great. When i switch to 3.9, I can't get past the install of the dedupe library (apparently due to the levenshtein-search dependency having this install issue)... so i'm not sure where exactly you are mentioning something i can control..
I've tried uninstalling python 3.9, and reinstalling / re-setting up the pycharm 3.9 interpreter and environment.. but no change..

[fgregg]

i'm sorry, you are having trouble getting everything set up, but i think i can't help you further get your environment set up.

this is a good argument for #976, so that we can provide binaries for more platforms.

[rderidder-lda]

if i understand it right.. levenshtein-search provided a wheel for python 3.8, and so no compiling was needed when it was installed.. but they do not supply a wheel for 3.9.. so in order to compile it, we need a certain number of things installed for MS C++. I'm not sure what exactly those things are.. as above, I tried 4 of the modules per the MS website.. but maybe more are required? Maybe other folks do not report this issue because they already had these MS items installed and so the wheel is compiled without them knowing? However after much head-banging, i do believe this is not really a local issue, but an issue with the fact that levenshtein-search does not have a 3.9 wheel.. please do correct me if i'm wrong here. I will now try and install more MS modules to see if i can get this library to compile..

[rderidder-lda]

if anyone does have a 3.9 wheel for windows 10 64bit, i'd love to download it.. thanks

[rderidder-lda]

Success.. i added a bunch more modules from MS.. and it can now compile. Likely can narrow this list further, but its better than the entire VS: