Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

race conditions affecting MBTilesWriter #30

Open
mash-graz opened this issue Oct 29, 2019 · 0 comments
Open

race conditions affecting MBTilesWriter #30

mash-graz opened this issue Oct 29, 2019 · 0 comments

Comments

@mash-graz
Copy link

mash-graz commented Oct 29, 2019

your plugin included in QGIS 3.10 for linux seems to be plagued by race conditions when it becomes executed in multiple threads.

the issue doesn't appear in case of the DirectoryWriter, but the MBTilesWriter will hardly work at all and produce corrupt files with lots of missing tiles and other defects...

as a symptom of this particular issue, you should often see error messages like this:

ERROR 5: /tmp/test7.mbtiles: Access window out of range in RasterIO().  Requested (-569088,-368384) of size 256x256 on raster of 512x512.

the strange negative numbers are a side effect of unexpected changed values of the self._first_tile variable resp. colliding zoom factors which are running in parallel and overwrite the expected state one another...

i did try to fix the issue by replacing self._first_tile and self._zoom_ds with dictionaries indexed by the zoom factor to reduce the observable collisions. this had some positive effect, but it doesn't completely fix the issues. :(

unfortunately i do not know, how to fix this kind of thread lock requirements in case of QGIS plugins in a more adequate manner.

the versions of my affected debian linux installation:

QGIS-Version: 3.10.0-A Coruña
QGIS-Codeversion: 6c816b4204
Qt-Version: 5.11.3
GDAL-Version: 2.4.2
GEOS-Version: 3.8.0-CAPI-1.13.1 
PROJ-Version: Rel. 5.2.0, September 15th, 2018
Verarbeite Algorithmus…
Algorithmus XYZ-Kacheln erzeugen (MBTiles) startet…
Eingabeparameter:
{ 'BACKGROUND_COLOR' : QColor(0, 0, 0, 0), 'DPI' : 96, 'EXTENT' : '1716053.858612443,1721455.9642655044,5951485.136729757,5955838.399322829 [EPSG:3857]', 'METATILESIZE' : 4, 'OUTPUT_FILE' : '/tmp/test7.mbtiles', 'QUALITY' : 75, 'TILE_FORMAT' : 0, 'ZOOM_MAX' : 16, 'ZOOM_MIN' : 12 }

Using 8 CPU Threads:
Pushing all tiles at once: 11 tiles.
Ausführung nach 2.34 Sekunden abgeschlossen
Ergebnisse:
{'OUTPUT_FILE': '/tmp/test7.mbtiles'}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant