In JIRA: IBCSPRT: DCA disconnecting from server on testing dev instance

[MiekoHash]

https://sagebionetworks.jira.com/browse/IBCSPRT-122

I have not created a cross reference ticket in the IBCSPRT project as this issue is from mid-Dec and it may have been already addressed offline.

[milen-sage]

@MiekoHash let's bring this up to the team for discussion today. It's important and may need escalation.

Unfortunately what's reported in this issue is the current expected behavior for all test instances on shinyapps.io.

This is due to shinyapps.io not supporting python versions > 3.8 and schematic in dev requiring versions >= 3.9

@afwillia has been in communication with shinyapps.io and Sage IT team confirming that's the case.

Until shinyapps.io supports appropriate versions or Sage IT proposes a way to mitigate the problem we are blocked with updates on shinyapps.io.

The problem becomes more severe if not fixed: once we release a new version of schematic it would also be incompatible with shinyapps.io. All DCA shinyapps.io on staging and production will not be able to update; in fact they shouldn't because they'd start crashing as well because of python versions incompatibility.

At this point we have six options:

1. freeze new releases of schematic until we transition all DCA instances to Sage's AWS shiny apps deployment
2. in all DCA instances on shinyapps.io, fix the version of schematic so that they don't automatically update to the next one; this implies new features and fixers won't be available to these apps.
3. either someone on D&T or Sage IT should push shinyapps.io as customers to fix the shinyapps.io environment and support python >=3.9 (and ideally allow us to install our own versions)
4. maintain a separate schematic deployment version for shinyapps.io until needed; this version would only contain bug and security updates
5. change schematic requirements to only install schematic_db if python version is >= 3.9
6. transition all shiny apps we deploy on shinyapps.io to the Sage AWS schematic REST API deployment; this will be feasible once the REST API is stable. It will however introduce unknown latencies since we have no control over where shinyapps.io instances run; and what their connection is to Sage's AWS instances.

As you can see from the schedule for multi-tenant deployment above, we will have to support DCA on shinyapps.io at least until May. This makes 1 and 2 untenable.

Option 4 may work but introduces not insignificant overhead on our engineers and is more error prone.

6. is an option but has the unknown latencies noted above; would introduce overhead on our engineers; and is only available after the schematic REST API is fully functional on Sage's AWS infra. That's mid-march. All-in-all this is a risky option.

That leaves option 3 as most desirable. Given @afwillia already contacted shinyapps.io and given it's not part of FAIR Data engineers responsibility to be engaged with shinyapps.io customer support and/or fix environments provisioned by others, we need to figure out who's best positioned to move on this in the next one or two weeks.

Option 5 is feasible, but they'd miss schematic_db-related upsert functionality.

[MiekoHash]

Thank you for the very thorough explanation, @milen-sage .

[MiekoHash]

Moved to JIRA: https://sagebionetworks.jira.com/browse/DCA-129

[MiekoHash]

Closing per comment in the original IBCSPRT ticket. Linked JIRA ticket tracks of python incompatibility issue. cc: @AmyHeiser , @milen-sage