Bulk certificate generation not working from BO

[NamFra]

Bug Report

Problematic behavior

Bulk certificate generation does not seem to work from Joanie's back office, requiring the use of the Django admin panel instead (see #1061).
Currently, only certificate generation for a specific order works.

Expected behavior/code

Enable certificate generation directly from the back office, at the product level.

Ideally, this process could be improved (as discussed between @jonathanreveille and @cuchulainn-fun 😉 – which could be a separate feature request), for example:

• Allowing the upload of a list of eligible order/user references.
• Displaying a list of eligible orders with the ability to deselect those that should not receive a certificate (🤩).

This would help prevent accidental certificate generation for previous sessions.

For reference, for courses offering both Certificates and Open Badges, we frequently need to modify the course grading in Open edX before de generation. This prevents setting up an automatic certificate generation via a cron job :/

Steps to Reproduce

1. In the back office, go to the page of a product (or course) where certificates need to be generated.
2. Click on the "Generate certificates" button.
3. No certificates are generated.
4. Try generating a certificate for a specific order and see that it works.

Environment

• Joanie version: 2.16.0

Possible Solution

There seems to be a verification step that fails, even in the back office.
This step does not occur when generating a certificate for a specific order.

For exemple, this requests fails in 404: https://joanie.preprod-fun.apps.openfun.fr/api/v1.0/admin/course-product-relations/590316b1-958f-4601-a64b-d6c2044b3e49/check_certificates_generation_process/
And its preview shows details: "No cache data found."

Additional context/Screenshots

Bulk generation (left) vs. Order generation (right):

"No cache data found." preview:

Bulk generation attempt:

joanie-bulk-gen-fails_v2.webm

Order generation:

joanie-order-gen-OK.webm

More info: Trello card

🙏

[jonathanreveille]

Just for information, there can be 2 reasons why you have this message : No cache data found.
This endpoint check_certificates_generation_process just verifies if the (celery) task in background is done or not depending if it finds it.

It's either :

• the request to generate was never initiated,
• the request is already done.
  Both returns 404 Not Found status code.

Only when the process is ongoing that you can find the cache data, here it returns a status code 200 OK.

Although, there should something else going wrong since no certificates are generated.

[jonathanreveille]

We have decided to make this issue in priority and make sure that the generation of certificates for products of type certificates works. We want our admin users to use the BO, instead of the Django admin.

[NamFra]

Thanks for the info, @jonathanreveille, that makes sense!

We ran a new test in production (for the first time on this product) to go through all the steps, and we noticed that a 404 appears even before the generation starts:

joanie-gen-bo-fail.webm

This is great for focusing on generation from the back-office rather than from the admin panel. 🙏
We also prefer this option!

If there’s still anything to investigate or understand about this bug, @cuchulainn-fun has two certificate generations planned this week (we discussed this with @jbpenrath).
Feel free to let him know if you need to observe what happens in real conditions!

[jonathanreveille]

The 404 NOT FOUND error is not yet explicit, we apologize for that matter.

Actually, the queryset that retrieves eligible orders that can get their certificates generated does not work as intended. That queryset would only work with product type credential, and not yet product type certificate... 😓
With the upcoming fix that is not yet in pre-production (still in review), it will be possible to generate those.

Thus, the second endpoint that callscheck_certificates_generation_process verifies if the process is ongoing in the background with a celery task. Since the queryset did not find any orders to generate certificate, the task has already ended and it returns NOT_FOUND 404 status code. Only when the generation of certificates is in process, you should be able to see a 200 OK response with additional data telling you how many certificates are being generated, and how many were already generated before calling the action initially.

[NamFra]

Thanks for this explanation @jonathanreveille, it's very clear!
Great job identifying and fixing the issue 👏 We'll test it in pre-production as soon as it's available
🙏