Tests fail for Python 3.6

[stlehmann]

@kim-acmehi

@stlehmann I am using Python 3.7 so this is not an issue for me. I know how it goes being busy. I will have a look at the failure in travis-ci and see if I can get a clue why it is failing on Python 3.6. But I might be too busy to get to it this weekend. :)

Originally posted by @kim-acmehi in #79 (comment)

[stlehmann]

@kim-acmehi

@stlehmann I finally had some time to look at the failing Python 3.6 test. Running deploy.sh by hand with Python 3.6 I found that test_on_topic does pass sometimes but mostly fails. I set up an overnight run of the deploy.sh iterations. Out of about 1150 test runs test_on_topic:

FAILED: 805
PASSED: 350

I did try upping the wait() time from 5 seconds but it did not seem to make any difference. Since it fails more often than not I think the next step to to watch the log messages on the mosquitto server and see how a passing and failing test compares. If you have any other idea please let me know.

Originally posted by @kim-acmehi in #79 (comment)

[kim-acmehi]

It looks like there is a race in test_on_topic() where it does not get the on_connect() registered soon enough so it can subscribe to the message it is about to publish. Depending on the version of Python it seems to mostly work or mostly fail. For me Python 3.7 would mostly work for a single trial of deploy.sh. It would mostly fail with Python 3.6. When I run 100 trials of deploy.sh it would always show PASSED and FAILED for test_on_topic(). I think it is chance that this test shows up a PASSED in the CI run.

I changed test_on_topic() to use the same sequence as test_disconnect() which calls mqtt.init_app() early and before it waits for the on_connect(). I can run 100 trials with any version of Python and no failures.

There might be other tests that should be doing this (e.g, test_publish()). For now I think I have a good fix for test_on_topic().

[kim-acmehi]

I just noticed that Travis CI is showing everything passing as of 3 days ago. I don't see any commits that might have fixed something to get passing tests. My guess is that test_on_topic() is passing by luck.

[kim-acmehi]

Pull request #81 build failure: not sure what is going on there. I have not seen a failure like that. My test on Focal worked:

(venv-3.6) kimk@kimk-VirtualBox-UB18:~/Flask-MQTT$ ./deploy.sh 
flask_mqtt/__init__.py:389: error: On Python 3 '{}'.format(b'abc') produces "b'abc'", not 'abc'; use '{!r}'.format(b'abc') if this is desired behavior
Found 1 error in 1 file (checked 1 source file)
============================================== test session starts ===============================================
platform linux -- Python 3.6.12, pytest-6.1.1, py-1.9.0, pluggy-0.13.1 -- /home/kimk/workspace/flask-mqtt-dev/Flask-MQTT/venv-3.6/bin/python3.6
cachedir: .pytest_cache
rootdir: /home/kimk/Flask-MQTT
plugins: cov-2.10.1
collected 13 items                                                                                               

tests/test_flaskmqtt.py::FlaskMQTTTestCase::test_connect_disconnect PASSED                                 [  7%]
tests/test_flaskmqtt.py::FlaskMQTTTestCase::test_early_initialization_app_is_not_none PASSED               [ 15%]
tests/test_flaskmqtt.py::FlaskMQTTTestCase::test_late_initialization_app_is_none PASSED                    [ 23%]
tests/test_flaskmqtt.py::FlaskMQTTTestCase::test_mqtt_config_values PASSED                                 [ 30%]
tests/test_integration.py::FlaskMQTTTestCase::test_connect_with_username PASSED                            [ 38%]
tests/test_integration.py::FlaskMQTTTestCase::test_disconnect PASSED                                       [ 46%]
tests/test_integration.py::FlaskMQTTTestCase::test_logging PASSED                                          [ 53%]
tests/test_integration.py::FlaskMQTTTestCase::test_on_topic PASSED                                         [ 61%]
tests/test_integration.py::FlaskMQTTTestCase::test_publish PASSED                                          [ 69%]
tests/test_integration.py::FlaskMQTTTestCase::test_qos PASSED                                              [ 76%]
tests/test_integration.py::FlaskMQTTTestCase::test_simple_connect PASSED                                   [ 84%]
tests/test_integration.py::FlaskMQTTTestCase::test_subscribe PASSED                                        [ 92%]
tests/test_integration.py::FlaskMQTTTestCase::test_topic_count PASSED                                      [100%]
Coverage.py warning: No data was collected. (no-data-collected)


---------- coverage: platform linux, python 3.6.12-final-0 -----------
Name                     Stmts   Miss  Cover
--------------------------------------------
flask_mqtt/__init__.py     188    188     0%


============================================== 13 passed in 40.55s ===============================================
(venv-3.6) kimk@kimk-VirtualBox-UB18:~/Flask-MQTT$