enable_chunked_transfer_encoding_without_webserver #110
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dobefore
changed the title
|
Hey, thanks for your work! Traceback (most recent call last):
File "/usr/local/lib/python3.9/wsgiref/handlers.py", line 137, in run
self.result = application(self.environ, self.start_response)
File "/opt/ankisyncd/ankisyncd/sync_app.py", line 590, in __call__
p=req.params
File "/opt/ankisyncd/ankisyncd/sync_app.py", line 496, in params
r=self.parse_request
File "/opt/ankisyncd/ankisyncd/sync_app.py", line 427, in parse_request
items=re.split(repeat,bd)
File "/usr/local/lib/python3.9/re.py", line 231, in split
return _compile(pattern, flags).split(string, maxsplit)
File "/usr/local/lib/python3.9/re.py", line 294, in _compile
return _cache[type(pattern), pattern, flags]
TypeError: unhashable type: 'list'Anki 2.1.46 (2.1.44 throws exactly the same error) |
|
thanks for your detailed and useful debug info! ,so that checking window must be cancelled ,and hit sync once more. |
|
Thanks, it is working now and fixes #108. I am using anki sync server behind a reverse proxy (Traefik). |
|
I thought it is working but now I get "problem with AnkiWeb" and this log on the server (forcing a full sync works though): Traceback (most recent call last):
File "/opt/ankisyncd/ankisyncd/thread.py", line 98, in _run
ret = self.wrapper.execute(func, args, kw, return_queue)
File "/opt/ankisyncd/ankisyncd/collection.py", line 45, in execute
ret = func(*args, **kw)
File "/opt/ankisyncd/ankisyncd/sync_app.py", line 714, in run_func
res = handler_method(**keyword_args)
File "/opt/ankisyncd/ankisyncd/sync_app.py", line 152, in sanityCheck2
return dict(status, c=client, s=server)
ValueError: dictionary update sequence element #0 has length 1; 2 is required
Traceback (most recent call last):
File "/usr/local/lib/python3.9/wsgiref/handlers.py", line 137, in run
self.result = application(self.environ, self.start_response)
File "/opt/ankisyncd/ankisyncd/sync_app.py", line 653, in __call__
result = self._execute_handler_method_in_thread(url, data, session)
File "/opt/ankisyncd/ankisyncd/sync_app.py", line 723, in _execute_handler_method_in_thread
result = thread.execute(run_func, kw=keyword_args)
File "/opt/ankisyncd/ankisyncd/thread.py", line 79, in execute
raise ret
File "/opt/ankisyncd/ankisyncd/thread.py", line 98, in _run
ret = self.wrapper.execute(func, args, kw, return_queue)
File "/opt/ankisyncd/ankisyncd/collection.py", line 45, in execute
ret = func(*args, **kw)
File "/opt/ankisyncd/ankisyncd/sync_app.py", line 714, in run_func
res = handler_method(**keyword_args)
File "/opt/ankisyncd/ankisyncd/sync_app.py", line 152, in sanityCheck2
return dict(status, c=client, s=server)
ValueError: dictionary update sequence element #0 has length 1; 2 is requiredTraceback (most recent call last):
File "/usr/local/lib/python3.9/wsgiref/handlers.py", line 137, in run
self.result = application(self.environ, self.start_response)
File "/opt/ankisyncd/ankisyncd/sync_app.py", line 626, in __call__
raise HTTPNotFound()
webob.exc.HTTPNotFound: The resource could not be found. |
|
I also get this error: |
|
thanks for pointing out these constructive matters. after more debuging,server code sync normally.Tested in version 2.1.44,46,47 On Windows and 2.15.6 on Ankidroid,especially full-upload and full-download in order to enjoy this, some advice hope to be taken: if there are also issues about the server,feel free to post! |
|
there is still some problems about sync with a bulk of media , the progress is not yet finished |
|
after revising parse request code,this time server support multi-deck,importing .colpkg,.apkg file syncing |
|
It mostly works good now. The server logs this error for AnkiDroid: For Anki desktop this error appears: |
|
thanks for your testing!
issues like those are caused by anki module upgrade. would you mind updating anki module by using |
This was referenced Sep 17, 2021
Closed
Closed
|
Ok, I was on anki 2.1.40 on the server because that's defined here https://github.com/ankicommunity/anki-sync-server/blob/develop/src/requirements.txt#L4 After upgrading the other errors are gone. But now I have this error (it also came up for AnkiDroid): |
|
thanks for rasing it. problem solved |
|
What needs to be done to get this merged? |
|
thanks for your concern.The reasons may be as follows:
|
|
I tested it with latest version
|
I set it up the same way and it worked fine too. |
|
I'm going to mark this as That being said, it's okay if it's not 100% battle tested, that's what the develop branch is for. I can also cut a release before merging this PR if instability is a major concern. |
|
change log: @VikashKothary ,ready to be merged |
VikashKothary
requested changes
Feb 2, 2022
There was a problem hiding this comment.
Great job with this. It's a really big win because it makes the web server optional.
I've just left some comments to improve the clarity and maintainability of the code. But once that's done, it should be good to go. I can also see you've already got users testing it which is great.
|
@VikashKothary ,there are still some cases which we need to use reverse proxies to forward requests,for example,we want to make secure connection and set certicate.Could you have a look at readme and make a revision? |
VikashKothary
approved these changes
Feb 6, 2022
have not poetry set up
add a guard case to def finish remove redundant blank line of def sanityCheck
add type hint to def upload
|
Thank you so much for your hard work @dobefore. It's good to see this feature merged. Thank you to @marzzzello and @bitsf for testing out this new feature. It's always appreciated. |
This was referenced May 7, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
this update is based on Anki module 2.1.46
and this PR is incompatible with these PRs #104 ,#105,#106 ,because these PRs are based on webserver like nginx.
So,if this PR is merged,then other three will be discarded.
As the title shows,this make it valid HTTP transfer encoding:Chunked
How to implement?
read file stream handle provided by wsgiref from client request into byte-string,
skip each chunk size by add 2 bytes size,strip CRLF per line
finally , split it into a multi-field dictionary,fields like 'data','v'...
Other non-chunk message ,of course,is handled,mostly with regex.
Other changes are made due to missing attributes or redundant arguments caused by Anki module upgrade.
1.removed unused fn removed in sync.py
2.split remove in sync.py into add_grave apply_graves
...
Known Issue
1.importing a colpkg into an existing and already synced account on Ankidroid may cause media loss on server side
Tested in 2.1.44 ,2.1.47 On PC and 2.15.6 on Ankidroid