Upload treatments to multiple Nightscout sites

[Navid200]

Fixes: #266

xDrip only uploads treatments to the first site if the user enters multiple URLs separated by space as the base URL.

I have confirmed this is because during the very first run of the for loop going through the multiple sites, the treatment upload queue is completed (up.completed).

I have added a counter to detect the last run and made the completed commands conditional on being in the last run of the loop.
My tests show that treatments are now uploaded to multiple sites.

I did the auto format because the indentations are very confusing.

I had unintentionally left some test logs in a previous PR. I have removed those.

[Navid200]

These are the changes:

[Navid200]

I created two new free Nightscout sites using our Google Cloud scheme.

I used a test phone with xDrip as a follower of my active xDrip on my main phone.
I set up this test phone to upload to both new test Nightscout sites.
The test phone uploaded readings to both test sites successfully.

First, before making any changes, I added lots of logs to make the behavior of xDrip on the test phone obvious.
When adding treatments on the test phone, I could see that they were only uploaded to the first site, but not the second.
The logs showed that while the first run of the loop addressing the two sites was executed, the queue would be terminated by xDrip resulting in treatments not being uploaded to the second site.

After adding the changes in this PR to xDrip, treatments were uploaded to both sites.
Yet, the logs showed that the queue was terminated after the second upload was completed and no more uploads were attempted.

[Navid200]

I am not 100% sure. But, it may be that the queue has a wider scope than it should have. Therefore, when it is completed inside a member of the loop, when the next member of the loop attempts to recreate the same queue, it fails.

So, I am sure there are other ways to fix this issue. I have fixed it by making the completion conditional. So, when the second member of the loop attempts to create the queue and it sees that it already exists, I suppose it does not recreate it.

Please tell me how I can help to make the review easier.

Would it help if I undo the auto formatting?

[Navid200]

Edit:
If you merge the following PR first, I can rebase this PR and all the code formatting will then disappear and you can easily see only the few changed lines in this PR.
#3834

@jamorham Would you please have a look and let me know how I can improve it if needed?
We need this fixed so that people can upload everything to multiple servers.
Thanks

[Navid200]

I rebased this PR. Then, repeated the test.

Installed our latest Nightly on the test phone. The following image shows the status page while set up to get readings from my main Nightscout and upload to two test Nightscout sites.

I then entered 5 units of insulin on the test phone.

The following image shows the two Nightscout sites.

You can see that the treatment has been uploaded only to the first site.

Then, I installed this PR on the same test phone (Android 11).

And again, I entered 5 units of insulin.

And the following shows the two Nightscout sites.

You can see that this time, the treatment has been uploaded to both sites.

[Navid200]

@jamorham Would you please review this when you get a chance?

If testing by others would help, I can release a test release and ask a few individuals to test and report.

[jamorham]

The current logic is intended that if any site upload succeeds then the queue will be marked as completed. This is because multiple sites was added later and the queue entry isn't able to differentiate which site uploads were successful to.

Looking through the code it looks like this fails for treatment uploads because the function being called postTreatments() attempts to read the pending queue entries which were already marked as completed if the first site upload succeeds.

I think the changes you propose will create the opposite situation, so that if site 1 is up and site 2 is down then items will never leave the queue and will be endlessly uploaded to site 1.

The proper fix for this would be queues per site but the architecture doesn't lend itself very well to that at all and if we were going to do that we should also start properly using the nightscout sdk which would also likely improve overall handling of this.

In the short term, I think the fix which preserves the original behavior of just ignoring failures to sites which fail so long as one upload succeeds can quite easily be preserved by simply moving the line of code which retrieves the queue list:

final List<UploaderQueue> tups = UploaderQueue.getPendingbyType(Treatments.class.getSimpleName(), THIS_QUEUE);

Move this to the parent caller outside of the loop per site and then pass it in as a variable. The fact that entries will be marked as completed I don't think will affect their upload then, it is likely only where they are filtered out by getPendingByType() which is the problem here.

You'll probably have to move that code two levels out to doRESTUpload() but you can do the conditional preference check there before doing the work to populate an array and then just check if it is a non-null parameter where we normally would check if the preference is enabled. I'm not sure if this class is thread-safe in terms of how it is used so you'll probably want to pass it in as a parameter between those functions rather than store it in a class member field.

I believe that should work and is a safer solution that you have proposed in this PR.

[Navid200]

Thank you very much. I had overlooked the intent of the queue. Thanks so much for teaching me that here.
I will work on this as you have instructed. Thanks

[Navid200]

@jamorham I repeated my test and disabled one of the two sites. xDrip showed an error in red on the system status page while readings were only uploaded to the active site.
After I enabled the other site, xDrip continued to upload to both. But, it never backfilled the readings into the site that was down for a while.
So, there is no queue per site even for readings.
I did not know that.

Would doing this the proper way be extremely complicated so that you feel you must do it yourself?
Or, would I have a chance of being able to help?

This is really bad that if someone has two sites entered, the queue will not be maintained if only one site is active.
I would like this to be fixed. I wish to fix it if you think I have a chance.