Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Solved] Error parsing lftp job status [v0.8.6] #106

Open
casperse opened this issue Sep 30, 2021 · 7 comments
Open

[Solved] Error parsing lftp job status [v0.8.6] #106

casperse opened this issue Sep 30, 2021 · 7 comments

Comments

@casperse
Copy link

casperse commented Sep 30, 2021

FYI: NOT SOLVED FOR THE DOCKER - MOVED SEEDSYNC TO VM!

Running the latest docker on unraid
Extra parameters: --cpus=2 --memory=3G --restart unless-stopped --user 99:100

The docker stopped before - so I added the : --restart unless-stopped
I can see it connects and start downloading (Great speed by the way!)
But then it stops.

I am using the SSH user and password.
And I have settings:

[General]
debug = True
verbose = False

[Lftp]
remote_address = domain.seedhost
remote_username = username
remote_password = password
remote_port = 22
remote_path = /home/username/downloads
local_path = /downloads
remote_path_to_scan_script = /tmp
use_ssh_key = False
num_max_parallel_downloads = 2
num_max_parallel_files_per_download = 4
num_max_connections_per_root_file = 4
num_max_connections_per_dir_file = 4
num_max_total_connections = 16
use_temp_file = True

[Controller]
interval_ms_remote_scan = 30000
interval_ms_local_scan = 10000
interval_ms_downloading_scan = 1000
extract_path = /tmp
use_local_path_as_extract_path = True

[Web]
port = 8800

[AutoQueue]
enabled = True
patterns_only = True
auto_extract = False

`lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 322, in status
statuses = self.__job_status_parser.parse(out)
File "/app/python/lftp/job_status_parser.py", line 94, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")

lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status

2021-09-30 11:46:11,459 - INFO - seedsync.Controller (MainProcess/ControllerJob) - Exited controller

2021-09-30 11:46:34,495 - INFO - web_access (MainProcess/worker 1) - 192.168.0.14 - - [30/Sep/2021:11:46:34 +0200] "GET /server/config/get HTTP/1.1" 200 801 "http://192.168.0.6:8800/autoqueue" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36"
2021-09-30 11:46:34,496 - INFO - web_access (MainProcess/worker 2) - 192.168.0.14 - - [30/Sep/2021:11:46:34 +0200] "GET /server/autoqueue/get HTTP/1.1" 200 185 "http://192.168.0.6:8800/autoqueue" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36"
2021-09-30 11:46:53,422 - INFO - web_access (MainProcess/worker 3) - 192.168.0.14 - - [30/Sep/2021:11:46:53 +0200] "GET /settings HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36"
2021-09-30 11:47:46,404 - INFO - web_access (MainProcess/worker 4) - 192.168.0.14 - - [30/Sep/2021:11:47:41 +0200] "GET /server/stream HTTP/1.1" 200 - "http://192.168.0.6:8800/settings" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36"
2021-09-30 11:47:46,441 - INFO - web_access (MainProcess/worker 5) - 192.168.0.14 - - [30/Sep/2021:11:47:46 +0200] "GET /server/config/get HTTP/1.1" 200 801 "http://192.168.0.6:8800/dashboard" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36"
2021-09-30 11:54:39,724 - INFO - seedsync (MainProcess/MainThread) - Caught signal SIGTERM
2021-09-30 11:54:39,724 - INFO - seedsync (MainProcess/MainThread) - Exiting Seedsync
2021-09-30 11:54:50,887 - INFO - seedsync (MainProcess/MainThread) - Debug mode is disabled.
2021-09-30 11:54:50,887 - INFO - seedsync (MainProcess/MainThread) - Starting SeedSync
2021-09-30 11:54:50,890 - INFO - seedsync (MainProcess/MainThread) - Platform: x86_64
2021-09-30 11:54:51,398 - INFO - seedsync.WebApp (MainProcess/MainThread) - Html path set to: /app/html
2021-09-30 11:54:51,835 - INFO - RemoteScanner.RemoteScanner (RemoteScanner/RemoteScanner) - Skipping remote scanfs installation: already installed
2021-09-30 11:54:54,531 - INFO - web_access (MainProcess/worker 0) - 192.168.0.14 - - [30/Sep/2021:11:54:54 +0200] "GET /logs HTTP/1.1" 200 764 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36"
2021-09-30 11:54:54,893 - INFO - web_access (MainProcess/worker 1) - 192.168.0.14 - - [30/Sep/2021:11:54:54 +0200] "GET /server/stream HTTP/1.1" 200 - "http://192.168.0.6:8800/logs" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36"
2021-09-30 11:54:55,190 - INFO - web_access (MainProcess/worker 2) - 192.168.0.14 - - [30/Sep/2021:11:54:55 +0200] "GET /assets/favicon.png HTTP/1.1" 200 1000 "http://192.168.0.6:8800/logs" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36"
2021-09-30 11:55:01,943 - INFO - seedsync.AutoQueue (MainProcess/ControllerJob) - Auto queueing '#TVSHOWS-C' for pattern '#TVSHOWS-C'
2021-09-30 11:55:01,944 - INFO - seedsync.AutoQueue (MainProcess/ControllerJob) - Auto queueing '#MUSIC-SABnzbd' for pattern '#MUSIC-SABnzbd'
2021-09-30 11:55:02,444 - INFO - seedsync.Controller (MainProcess/ControllerJob) - Received command Action.QUEUE for file #TVSHOWS-C

2021-09-30 11:55:02,496 - INFO - seedsync.Controller (MainProcess/ControllerJob) - Received command Action.QUEUE for file #MUSIC-SABnzbd

2021-09-30 11:55:05,533 - INFO - web_access (MainProcess/worker 4) - 192.168.0.14 - - [30/Sep/2021:11:55:05 +0200] "GET /server/autoqueue/get HTTP/1.1" 200 185 "http://192.168.0.6:8800/autoqueue" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36"
2021-09-30 11:55:05,565 - INFO - web_access (MainProcess/worker 5) - 192.168.0.14 - - [30/Sep/2021:11:55:05 +0200] "GET /server/config/get HTTP/1.1" 200 801 "http://192.168.0.6:8800/autoqueue" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36"
2021-09-30 11:55:12,318 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser (MainProcess/ControllerJob) - LftpJobStateParser error: Unrecognized line 'pget: /downloads/#MUSIC-SABnzbd/Nelly.Furtado.-.Loose.(Expanded.Edition).(2021).Mp3.320kbps.[PMEDIA].��-xpost/10..Say.It.Right.mp3: Permission denied'

2021-09-30 11:55:12,323 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser (MainProcess/ControllerJob) - Status:

pget: /downloads/#MUSIC-SABnzbd/Nelly.Furtado.-.Loose.(Expanded.Edition).(2021).Mp3.320kbps.[PMEDIA].��-xpost/10..Say.It.Right.mp3: Permission denied
2021-09-30 11:55:12,328 - WARNING - seedsync.Controller.Lftp (MainProcess/ControllerJob) - Ignoring status error (count=1)

2021-09-30 11:55:19,193 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser (MainProcess/ControllerJob) - LftpJobStateParser error: Unrecognized line 'pget: /downloads/#MUSIC-SABnzbd/Nelly.Furtado.-.Loose.(Expanded.Edition).(2021).Mp3.320kbps.[PMEDIA].��-xpost/16..Te.Busque.(Spanish.Version).mp3: Permission denied'

2021-09-30 11:55:19,193 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser (MainProcess/ControllerJob) - Status:

pget: /downloads/#MUSIC-SABnzbd/Nelly.Furtado.-.Loose.(Expanded.Edition).(2021).Mp3.320kbps.[PMEDIA].��-xpost/16..Te.Busque.(Spanish.Version).mp3: Permission denied

2021-09-30 11:55:19,194 - WARNING - seedsync.Controller.Lftp (MainProcess/ControllerJob) - Ignoring status error (count=2)

2021-09-30 11:55:25,173 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser (MainProcess/ControllerJob) - LftpJobStateParser error: Unrecognized line 'mirror: /downloads/#MUSIC-SABnzbd/Nelly.Furtado.-.Mi.Plan.(2009).Flac-xpost/[PRiVATE]-[WtFnZb]-[about_album]-[13+15] - "" yEnc 851 (1+1): Permission denied'

2021-09-30 11:55:25,173 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser (MainProcess/ControllerJob) - Status:

mirror: /downloads/#MUSIC-SABnzbd/Nelly.Furtado.-.Mi.Plan.(2009).Flac-xpost/[PRiVATE]-[WtFnZb]-[about_album]-[13+15] - "" yEnc 851 (1+1): Permission denied
2021-09-30 11:55:25,173 - ERROR - seedsync.ControllerJob (MainProcess/ControllerJob) - Caught exception in job ControllerJob

Traceback (most recent call last):
File "/app/python/lftp/job_status_parser.py", line 89, in parse
statuses += self.__parse_queue(lines)
File "/app/python/lftp/job_status_parser.py", line 482, in __parse_queue
raise ValueError("Unrecognized line '{}'".format(lines[0]))
ValueError: Unrecognized line 'mirror: /downloads/#MUSIC-SABnzbd/Nelly.Furtado.-.Mi.Plan.(2009).Flac-xpost/[PRiVATE]-[WtFnZb]-[about_album]-[13+15] - "" yEnc 851 (1+1): Permission denied'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute

self.__controller.process()
File "/app/python/controller/controller.py", line 202, in process

self.__update_model()
File "/app/python/controller/controller.py", line 295, in __update_model

lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 322, in status
statuses = self.__job_status_parser.parse(out)
File "/app/python/lftp/job_status_parser.py", line 94, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")

lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status

2021-09-30 11:55:25,278 - ERROR - seedsync (MainProcess/MainThread) - Caught exception

Traceback (most recent call last):
File "/app/python/lftp/job_status_parser.py", line 89, in parse
statuses += self.__parse_queue(lines)
File "/app/python/lftp/job_status_parser.py", line 482, in __parse_queue
raise ValueError("Unrecognized line '{}'".format(lines[0]))
ValueError: Unrecognized line 'mirror: /downloads/#MUSIC-SABnzbd/Nelly.Furtado.-.Mi.Plan.(2009).Flac-xpost/[PRiVATE]-[WtFnZb]-[about_album]-[13+15] - "" yEnc 851 (1+1): Permission denied'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/python/seedsync.py", line 164, in run
controller_job.propagate_exception()
File "/app/python/common/job.py", line 77, in propagate_exception
raise exc_info[1].with_traceback(exc_info[2])
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute

self.__controller.process()
File "/app/python/controller/controller.py", line 202, in process

self.__update_model()
File "/app/python/controller/controller.py", line 295, in __update_model

lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 322, in status
statuses = self.__job_status_parser.parse(out)
File "/app/python/lftp/job_status_parser.py", line 94, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")

lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status

2021-09controller-30 11:55:28,491 - INFO - seedsync.Controller (MainProcess/ControllerJob) - Exited `

@quadcom
Copy link

quadcom commented Sep 30, 2021

What are your connection settings? Can you post a pic here (not your server address and password stuff, the thread counts and stuff on the right side)

@casperse
Copy link
Author

I posted The configuration file at The top
(All info is There except username and password (-(

@quadcom
Copy link

quadcom commented Sep 30, 2021

I posted The configuration file at The top
(All info is There except username and password (-(

Sorry, missed that

I have tested using different configs but I have learned that anything other than 1 file at a time with 10 connections causes LFTP to bug out.

Here are the settings that I've settled with.

image

@casperse
Copy link
Author

casperse commented Oct 1, 2021

Thanks - I just used the "default config" from the docker install (I thought these would be the safe ones)
but I have changed the settings now to match yours!
Its running but the UI is very unresponsive - maybe I have to many files to sync/DL?
If I move away from the UI and go back I have to wait for it to load (Long time) again it seems like its utillizing all power to do the transfers... RAM is just above 2G usage and CPU is really low 15%

@casperse
Copy link
Author

SOLVED: I found some comments about Ubuntu and version 20 problems?
So I took a chance and took my notes and did it all over again - but this time on Ubuntu 18.04.3 and here everything works!
I have the two VM's running side by side and so far I have not ben able to find any configuration or access differences on folder or paths?

One last Q. its now running very fast and without any slow UI or freeze on downloads
How many cores and Memory have you assigned for your VM (What is enough?)

@casperse casperse changed the title Error parsing lftp job status [v0.8.6] [Solved] Error parsing lftp job status [v0.8.6] Oct 12, 2021
@quadcom
Copy link

quadcom commented Oct 12, 2021

IMO, it doesn't need all that much as the functions are mostly throughput dependant.

On the other hand, I have gone overboard mainly because I have 12 cores in the CPU and 4 VM's of which are a MariaDB, a pihole and a Plex server so mostly low CPU applications (aside from Plex).

4 cores, 4GBon a 20GB HDD. It maxes out the RAM but never crashes or causes any issues with other VM's

Congrats on getting it going. It's a great feeling when you figure shit out and get the lab running.

Now you need to get filebot running.......

@casperse
Copy link
Author

casperse commented Oct 12, 2021

Actually I haven't setup the : "Use password-less key-based authentication" does it make any difference?
And can you see the Log section? seem to take forever to load

Update: I also had to set 4G and 4 Cores in order for it to run okay
I can see that it syncs all the existing files on my existing drive share again, so there must be a DB file where Seedsync safes all transactions to? How does it manage this over time?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants