Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TotalSegmentator command HTTP request failure, network unreachable #353

Open
Fjrm-297 opened this issue Aug 11, 2024 · 4 comments
Open

TotalSegmentator command HTTP request failure, network unreachable #353

Fjrm-297 opened this issue Aug 11, 2024 · 4 comments

Comments

@Fjrm-297
Copy link

How do I get around the error below that I get when I try to run the ‘TotalSegmentator -i ct.nii.gz -o segmentations’ command on a SLURM-managed HPC cluster/a system without internet access?
The main issue clearly is that the script attempts to make the HTTP request for the ‘download_url_and_unpack’ function, but the network connection fails because the system cannot access the internet.
Has anyone experienced the same?
Please can someone help?
Thanks.

Error message is:
Traceback (most recent call last):
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
OSError: [Errno 101] Network is unreachable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 793, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 491, in _make_request
raise new_e
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 467, in _make_request
self._validate_conn(conn)
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1099, in _validate_conn
conn.connect()
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/urllib3/connection.py", line 616, in connect
self.sock = sock = self._new_conn()
^^^^^^^^^^^^^^^^
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/urllib3/connection.py", line 213, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x2ade890d33e0>: Failed to establish a new connection: [Errno 101] Network is unreachable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/requests/adapters.py", line 589, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 847, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/urllib3/util/retry.py", line 515, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='github.com', port=443): Max retries exceeded with url: /wasserth/TotalSegmentator/releases/download/v2.0.0-weights/Dataset291_TotalSegmentator_part1_organs_1559subj.zip (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x2ade890d33e0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/xxxx/.conda/envs/myenvr/bin/TotalSegmentator", line 8, in
sys.exit(main())
^^^^^^
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/totalsegmentator/bin/TotalSegmentator.py", line 138, in main
totalsegmentator(args.input, args.output, args.ml, args.nr_thr_resamp, args.nr_thr_saving,
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/totalsegmentator/python_api.py", line 392, in totalsegmentator
download_pretrained_weights(tid)
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/totalsegmentator/libs.py", line 333, in download_pretrained_weights
download_url_and_unpack(WEIGHTS_URL, config_dir)
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/totalsegmentator/libs.py", line 129, in download_url_and_unpack
raise e
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/totalsegmentator/libs.py", line 112, in download_url_and_unpack
with requests.get(url, stream=True) as r:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/.conda/envs/myenvr/lib/python3.12/site-packages/requests/adapters.py", line 622, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='github.com', port=443): Max retries exceeded with url: /wasserth/TotalSegmentator/releases/download/v2.0.0-weights/Dataset291_TotalSegmentator_part1_organs_1559subj.zip (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x2ade890d33e0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
srun: error: indigo51: task 0: Exited with exit code 1

@wasserth
Copy link
Owner

You can manually download the weights using totalseg_download_weights on your local machine with internet. Then all weights will get downloaded to ~/.totalsegmentator. Then you can copy this folder to the same location on your HPC cluster. If you want to set a different location you can set the environment variable: TOTALSEG_HOME_DIR, e.g. export TOTALSEG_HOME_DIR=/mnt/nor/software/.totalsegmentator

@Echo-Kkk
Copy link

Echo-Kkk commented Dec 3, 2024

I set the pretrained weight file in the computing equipment to be used, which path is like '/Projects/TotalSegmentator/.totalsegmentator/nnunet/results', I set all the zip and unzip files(because i dont know whether it need to be zipped, so i put both), and i export the env path: 'export TOTALSEG_HOME_DIR=/Projects/TotalSegmentator/.totalsegmentator', but it occur the same error which shows 'requests.exceptions.ConnectionError: HTTPSConnectionPool', it looks like it doesnt find my manually download pretrain weight file, do i need to do anything else except putting them to target location and doing export env path operation?
Looking forward to your answer, THX :)

@wasserth
Copy link
Owner

wasserth commented Dec 3, 2024

The files need to be unzipped. But if you did this it is hard to say what the problem might without seeing your system.

@Echo-Kkk
Copy link

Echo-Kkk commented Dec 5, 2024

Nice to see your reply! I find the mistake not because of the env path, just when I need to use task_id=258(lung_vessel), the manually downloaded pretained weights filename did not match its require, it needs the lunng_vessel's weight path named 'Dataset258_lung_vessels....', but the manually downloaded one named 'Task258_lung_vessels...', just a little problem. And now i meet another problem that when i finish the inference(maybe have not finished),it showed me the progress bar'100%|██████████| 1/1 [00:06<00:00, 6.86s/it]', and jump a error telling me that it can not find the dataset.json file, I find other my manually download weight files, in dataset008(HepaticVessel), it has datast.json file and have a correct file path, even in dataset258, the target file path is not matched with needed one, so just a suggestion, there are some tasks pretrain file, its name or file in it or file path do not match with the codes, if you still mainntain this project, maybe check all these tasks file.
Best wishes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants