Engagement Metadata Management - API changes

CHANGELOG.MD

@@ -1,57 +1,76 @@
+## Current: feature/DESENG-443: Engagement Metadata Management
+
+- **Task**: Remove "default_values" from metadata taxa.
+  Replace with "preset values", metadata entries that are not assigned to an engagement.
+- **Task**: Update authorization documentation in the API blueprint. Update
+  metadata management to rely on normal authorization check functions.
+- **Task**: Clean up metadata management code and tests.
+
 ## February 08, 2024
+
 - **Task**Cache CORS preflight responses with the browser for a given period of time [DESENG-484](https://apps.itsm.gov.bc.ca/jira/browse/DESENG-484)
-    - Introduces a new configuration variable to specify the maximum age for Cross-Origin Resource Sharing (CORS)
-    - Modified the CORS preflight method to utilize this newly introduced variable.
+  - Introduces a new configuration variable to specify the maximum age for Cross-Origin Resource Sharing (CORS)
+  - Modified the CORS preflight method to utilize this newly introduced variable.
 - **Task**Consolidate and re-write old migration files [DESENG-452](https://apps.itsm.gov.bc.ca/jira/browse/DESENG-452)
-    - Change some foreign key field to nullbale false in model files
-    - Change `rejected_reason_other` to nullable true in `submission` model
-    - Generated new migration file based on the pending model changes which confirmed to be valid
-    - Updated Unit test of email verfication to send type to the api
+  - Change some foreign key field to nullbale false in model files
+  - Change `rejected_reason_other` to nullable true in `submission` model
+  - Generated new migration file based on the pending model changes which confirmed to be valid
+  - Updated Unit test of email verfication to send type to the api
 
 ## February 06, 2024
+
 - **Task**Convert keycloak groups to composite roles for permission levels [DESENG-447](https://apps.itsm.gov.bc.ca/jira/browse/DESENG-447)
-    - Commented out unit test related to Keycloak groups
-    - Changed reference of Keycloak `groups` to `roles`
-    - Commented out code related to Keycloak groups
+  - Commented out unit test related to Keycloak groups
+  - Changed reference of Keycloak `groups` to `roles`
+  - Commented out code related to Keycloak groups
 
 ## February 06, 2024
+
 - **Task** Streamline CRON jobs [DESENG-493](https://apps.itsm.gov.bc.ca/jira/browse/DESENG-493)
   - Aligned the CRON configuration and sample environment files with the structure used in the Met API.
   - Eliminated the reliance on engagement metadata within CRON jobs.
   - Implemented necessary code adjustments to seamlessly integrate with the updated CRON configuration.
 
 ## February 05, 2024
+
 - **Task** Change "Superuser" to "Administrator" [DESENG-476](https://apps.itsm.gov.bc.ca/jira/browse/DESENG-476)
 
 ## February 02, 2024
+
 - **Task** Updated Timeline widget icons so that the circles are more consistent. [🎟️DESENG-488](https://apps.itsm.gov.bc.ca/jira/browse/DESENG-488)
 
 ## February 01, 2024
+
 - **Task** Change name from "Engagement Core" to "Engagement Content". [🎟️DESENG-489](https://apps.itsm.gov.bc.ca/jira/browse/DESENG-489)
 
 ## January 29, 2024
+
 - **Task** Updated Babel Traverse library. [🎟️DESENG-474](https://apps.itsm.gov.bc.ca/jira/browse/DESENG-474)
   - Run `npm audit fix` to update the vulnerable Babel traverse library.
 
 ## January 26, 2024
+
 - **Task** Poll Widget: Front-end. [🎟️DESENG-464](https://apps.itsm.gov.bc.ca/jira/browse/DESENG-464)
   - Created UI for Poll Widget.
   - Updated Poll widget API and unit tests.
 
 ## January 25, 2024
+
 - **Task** Resolve issue preventing met-web from deploying on the Dev OpenShift environment. [🎟️DESENG-469](https://apps.itsm.gov.bc.ca/jira/browse/DESENG-469)
   - Remove Epic Engage-related links and update Keycloak link.
   - Remove additional authentication method.
 
 ## January 24, 2024
+
 - **Task** Update default project type to GDX for all deployments by default. [🎟️DESENG-472](https://apps.itsm.gov.bc.ca/jira/browse/DESENG-472)
   - Set the default project type to GDX on all continuous deployment (CD) files.
   - Removed the option to deploy to EAO.
 
 ## January 22, 2024
+
 - **Task** Poll Widget: Back-end [🎟️DESENG-463](https://apps.itsm.gov.bc.ca/jira/browse/DESENG-463)
   - Created Database models for Widget Poll, Poll Answers, Poll Response.
-  - Created API to manage  Widget Poll, Poll Answers, Poll Response.
+  - Created API to manage Widget Poll, Poll Answers, Poll Response.
   - Created Unit tests to test the code.
 - **Task** Add missing unit tests for met api [🎟️DESENG-481](https://apps.itsm.gov.bc.ca/jira/browse/DESENG-481)
   - Added missing unit tests for met api
@@ -91,6 +110,7 @@
 ## January 9, 2024
 
 - **Task** Improvements from Epic [🎟️DESENG-468](https://apps.itsm.gov.bc.ca/jira/browse/DESENG-468)
+
   - Improvements to Survey Result Tracking analytics
   - New Rejection Email Template for Closed Engagements
   - Export Format for Proponent updated to be in excel format
@@ -139,9 +159,10 @@
 ## November 6, 2023
 
 - **Feature**: Switch MET to use Keycloak SSO service [🎟️DESENG-408](https://apps.itsm.gov.bc.ca/jira/browse/DESENG-408)
+
   - Switch all role-based checks on the API to use a single callback function (`current_app.config['JWT_ROLE_CALLBACK']`)
   - Added a configurable path `JWT_ROLE_CLAIM` to indicate where your SSO instance places role information in the JWT token. If your access token looks like:
-  `{ ..., "realm_access": { "roles": [ "role1", "role2"]}}` you would set `JWT_ROLE_CLAIM=realm_access.roles`
+    `{ ..., "realm_access": { "roles": [ "role1", "role2"]}}` you would set `JWT_ROLE_CLAIM=realm_access.roles`
   - Explicitly disable single tenant mode by default to ensure correct multi-tenancy behaviour
   - Remove local Keycloak instances and configuration
   - Default to the "standard" realm for Keycloak
@@ -162,7 +183,6 @@
   - Remove one old production .env file with obsolete settings
 - Changes to DEVELOPMENT.md to reflect the current state of the project
 
-
 ## v1.0.0 - 2023-10-01
 
 - App handoff from EAO to GDX

met-api/migrations/versions/dbe023373f4f_remove_taxon_default_add_constraint.py

@@ -0,0 +1,31 @@
+"""Remove default_value from engagement_metadata_taxa and add unique constraint
+
+Revision ID: dbe023373f4f
+Revises: ec0128056a33
+Create Date: 2024-01-30 17:05:25.313222
+
+"""
+from alembic import op
+import sqlalchemy as sa
+from sqlalchemy.dialects import postgresql
+
+# revision identifiers, used by Alembic.
+revision = 'dbe023373f4f'
+down_revision = 'ec0128056a33'
+branch_labels = None
+depends_on = None
+
+
+def upgrade():
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.create_unique_constraint(None, 'engagement_metadata_taxa', ['id'])
+    op.drop_column('engagement_metadata_taxa', 'default_value')
+    # ### end Alembic commands ###
+
+
+def downgrade():
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.add_column('engagement_metadata_taxa', sa.Column(
+        'default_value', sa.TEXT(), autoincrement=False, nullable=True))
+    op.drop_constraint(None, 'engagement_metadata_taxa', type_='unique')
+    # ### end Alembic commands ###

met-api/src/met_api/auth.py

@@ -18,11 +18,18 @@
 from flask_jwt_oidc import JwtManager
 from flask_jwt_oidc.exceptions import AuthError
 
+from met_api.utils.constants import TENANT_ID_HEADER
+
 auth_methods = {  # for swagger documentation
     'apikey': {
         'type': 'apiKey',
         'in': 'header',
         'name': 'Authorization'
+    },
+    'tenant': {
+        'type': 'apiKey',
+        'in': 'header',
+        'name': TENANT_ID_HEADER
     }
 }
 
@@ -53,7 +60,8 @@ def decorated(*args, **kwargs):
                 token = jwt.get_token_auth_header()
                 # pylint: disable=protected-access
                 jwt._validate_token(token)
-                g.authorization_header = request.headers.get('Authorization', None)
+                g.authorization_header = request.headers.get(
+                    'Authorization', None)
                 g.token_info = g.jwt_oidc_token_info
             except AuthError:
                 g.authorization_header = None

met-api/src/met_api/models/engagement_metadata.py

@@ -17,7 +17,8 @@ class EngagementMetadata(BaseModel):
     """A unit of metadata for an Engagement. Can be used to store arbitrary data."""
 
     __tablename__ = 'engagement_metadata'
-    id = db.Column(db.Integer, primary_key=True, nullable=False, autoincrement=True)
+    id = db.Column(db.Integer, primary_key=True,
+                   nullable=False, autoincrement=True)
     engagement_id = db.Column(db.Integer,
                               db.ForeignKey('engagement.id', ondelete='CASCADE'), nullable=True, index=True)
     engagement = db.relationship('Engagement', backref='metadata')
@@ -69,21 +70,16 @@ def __repr__(self) -> str:
 class MetadataTaxonDataType(str, enum.Enum):
     """The data types that can be stored in a metadata property."""
 
-    TEXT = 'string'
-    LONG_TEXT = 'long-text'
+    TEXT = 'text'
+    LONG_TEXT = 'long_text'
     NUMBER = 'number'
     DATE = 'date'
+    TIME = 'time'
     DATETIME = 'datetime'
     BOOLEAN = 'boolean'
-    SELECT = 'select'
-    IMAGE = 'image'
-    VIDEO = 'video'
-    AUDIO = 'audio'
-    FILE = 'other_file'
     URL = 'url'
     EMAIL = 'email'
     PHONE = 'phone'
-    ADDRESS = 'address'
     OTHER = 'other'
 
     @classmethod
@@ -102,7 +98,8 @@ class MetadataTaxon(BaseModel):
 
     __tablename__ = 'engagement_metadata_taxa'
 
-    id = db.Column(db.Integer, primary_key=True, unique=True, autoincrement=True)
+    id = db.Column(db.Integer, primary_key=True,
+                   unique=True, autoincrement=True)
     tenant_id = db.Column(db.Integer,
                           db.ForeignKey('tenant.id', ondelete='CASCADE'),
                           nullable=False, index=True)
@@ -111,7 +108,6 @@ class MetadataTaxon(BaseModel):
     description = db.Column(db.String(256), nullable=True)
     freeform = db.Column(db.Boolean, nullable=False, default=False)
     data_type = db.Column(db.String(64), nullable=True, default='text')
-    default_value = db.Column(db.Text, nullable=True)
     one_per_engagement = db.Column(db.Boolean)
     position = db.Column(db.Integer, nullable=False, index=True)
 
@@ -122,7 +118,8 @@ def __init__(self, **kwargs) -> None:
             self.data_type = 'text'
         if not self.position:
             # find other taxa in this tenant and set position to the next highest
-            max_position = MetadataTaxon.query.filter_by(tenant_id=self.tenant_id).count()
+            max_position = MetadataTaxon.query.filter_by(
+                tenant_id=self.tenant_id).count()
             self.position = max_position + 1
 
     @validates('id')
@@ -145,6 +142,23 @@ def __repr__(self) -> str:
             return '<MetadataTaxon: None>'
         return f'<MetadataTaxon #{self.id}: {self.name}>'
 
+    @property
+    def preset_values(self) -> list[str]:
+        # Get preset values - any metadata entries with no specific engagement
+        return [entry.value for entry in self.entries if entry.engagement_id is None]
+
+    @preset_values.setter
+    @transactional()
+    def preset_values(self, values: list[str]) -> None:
+        # Update preset values to match the provided list
+        for entry in self.entries:
+            if entry.engagement_id is None and entry.value not in values:
+                entry.delete()
+        for value in values:
+            if value not in self.preset_values:
+                entry = EngagementMetadata(taxon_id=self.id, value=value)
+                entry.save()
+
     @transactional()
     def move_to_position(self, new_position: int) -> None:
         """
@@ -194,6 +208,7 @@ def reorder_taxa(cls, tenant_id: int, taxon_order: list[int]) -> None:
         Setting their positions accordingly.
         """
         for index, taxon_id in enumerate(taxon_order):
-            taxon = cls.query.filter_by(tenant_id=tenant_id, taxon_id=taxon_id).first()
+            taxon = cls.query.filter_by(
+                tenant_id=tenant_id, taxon_id=taxon_id).first()
             if taxon:
                 taxon.position = index

met-api/src/met_api/resources/__init__.py

@@ -81,7 +81,7 @@
 API.add_namespace(VALUE_COMPONENTS_API)
 API.add_namespace(SHAPEFILE_API)
 API.add_namespace(TENANT_API)
-API.add_namespace(METADATA_TAXON_API, path='/tenants/<string:tenant_name>/metadata')
+API.add_namespace(METADATA_TAXON_API, path='/engagement_metadata')
 API.add_namespace(ENGAGEMENT_METADATA_API, path='/engagements/<int:engagement_id>/metadata')
 API.add_namespace(ENGAGEMENT_MEMBERS_API, path='/engagements/<string:engagement_id>/members')
 API.add_namespace(WIDGET_DOCUMENTS_API, path='/widgets/<string:widget_id>/documents')

met-api/src/met_api/resources/engagement_metadata.py

@@ -31,7 +31,8 @@
 from met_api.utils.util import allowedorigins, cors_preflight
 
 EDIT_ENGAGEMENT_ROLES = [Role.EDIT_ENGAGEMENT.value]
-VIEW_ENGAGEMENT_ROLES = [Role.VIEW_ENGAGEMENT.value, Role.EDIT_ENGAGEMENT.value]
+VIEW_ENGAGEMENT_ROLES = [
+    Role.VIEW_ENGAGEMENT.value, Role.EDIT_ENGAGEMENT.value]
 
 API = Namespace('engagement_metadata',
                 path='/engagements/<engagement_id>/metadata',
@@ -76,7 +77,8 @@ def get(engagement_id):
     @cross_origin(origins=allowedorigins())
     @API.doc(security='apikey')
     @API.expect(metadata_create_model)
-    @API.marshal_with(metadata_return_model, code=HTTPStatus.CREATED)  # type: ignore
+    # type: ignore
+    @API.marshal_with(metadata_return_model, code=HTTPStatus.CREATED)
     @auth.has_one_of_roles(EDIT_ENGAGEMENT_ROLES)
     def post(engagement_id: int):
         """Create a new metadata entry for an engagement."""
@@ -95,7 +97,7 @@ def post(engagement_id: int):
 
 
 @cors_preflight('GET,PUT,DELETE')
-@API.route('/<metadata_id>')  # /api/engagements/{engagement.id}/metadata/{metadata.id}
+@API.route('/<metadata_id>')  # /metadata/{metadata.id}
 @API.doc(params={'engagement_id': 'The numeric id of the engagement',
                  'metadata_id': 'The numeric id of the metadata entry'})
 @API.doc(security='apikey')