Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Release 1.5.6
* Update traitlets
* Black and flake8 checks
* Use pytest in circleci
* Rationalise Procfile triggered processes
* Branding changes
* Bump version
* Fix dependabot alert fixes
* Patch front end vulnerability
* bump version to 1.5.7
* feat: add management command to patch audit logs
Changes to speed up model exports include an Audit's case title
pre-compute. However existing audit logs won't have the pre-computed
values in their json field. This management command patches all
audit logs accordingly.
* feat: test patch_audit managment command
* hotfix: TRLST 242 optimise logging in API project
* hotfix: robust logging trlst-252
hotfix to make logging more robust (trlst-252)
- Django signal receivers used to log db events cannot depend on
instance attrs being present so made them more forgiving so we
can guarantee the logging to debug issues. As additional belt
and braces, updated receivers to not rely on instance presence.
While this is unlikely, maybe possible if receiver is misconfigured.
- added pytest catchlog.
- added tests.
- added pytest config.
* release 1.5.8
Merge 1.5.8 release branch into master branch
* hotfix: trlst 282 high sev vulnerabilities
hotfix: release for Django and Pillow vulnerabilities
- Bumped django to 2.2.18
- Bumped Pillow to 8.1.2
* hotfix: high severity vulnerabilities.
- Applies fixes for y18n and lxml vulnerabilities
- Bump version to 1.5.8.2
* merge: release 1.5.9 branch into master
Release TR API 1.5.9
* hotfix: bump python runtime
* hotfix: bump deps to resolve vulnerabilities.
- Bumped django - CVE-2021-31542
- Bumped boto3, requests, urllib3 - CVE-2021-33503
- Bump service version
* hotfix: bump Pillow dep to resolve vulnerabilities.
- Bumped Pillow to 8.2.0 - CVE-2021-28677, CVE-2021-25287, CVE-2021-28675,
CVE-2021-25288, CVE-2021-28676, CVE-2021-28678.
* LSGH-7 sec: bump django to resolve vulnerability
* merge: release 1.5.10 branch into master
* merge: release 1.5.11 branch into master
* Release 1.5.12
* fix: bump python runtime to 3.9.7.
* hotfix: TRLST-493-invite-fails.
This change applies the fix for TRLST-493 and elements of fix for TRLST-484 to resolve 500 error raise when a public user registers.
* add check that there is a submission on the invite.
* Add guard group name check.
* Added pre-create of user profile in invite processing.
* bumped version to 1.5.12.1
* Release 1.5.13
* Release 1.5.14
* Release 1 5 15 (#143)
* fix: tidy and refactor base api view.
- Fixed issue with pagination support
- Fixed anomaly in user/group memebrship logic
- Added documentation
- Added proper initialisation of TradeRemediesApiView members
- Removed cruft
* fix: add optional args to base constructor.
* merge: release 1.5.7 into develop
This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes:
* Update traitlets
* Rationalise Procfile triggered processes
* Branding changes
* Dependabot alert fixes
* Patch front end vulnerability
* bump version to 1.5.7
* hotfix: TRLST 242 optimise logging in API project
* merge: 1.5.7.1 hotfix into develop (trlst 252) (#48)
Merge 1.5.7.1 hotfix (to make logging more robust) into develop (trlst-252)
- Django signal receivers used to log db events cannot depend on
instance attrs being present so made them more forgiving so we
can guarantee the logging to debug issues. As additional belt
and braces, updated receivers to not rely on instance presence.
While this is unlikely, maybe possible if receiver is misconfigured.
- added pytest catchlog.
- added tests.
- added pytest config.
* fix: TRLST_255 traceback when org is null.
Recent refactoring of trade_remedies_api/core/services/base.py base view introduced a small issue where, when the view is invoked, in certain circumstances the organisation is null and referencing its id causes a traceback. This was spotted in the logs but not visible to the user on the UI (the error seems to be dutifully ignored and the API responds to the client normally, however this could cause issues when called from other pages in the Public and Caseworker portal).
* feat: trlst-40 document submission sub-categories
As a caseworker, I need the ability to ADD a subcategory when I publish a document so that it is clearer for the users of the Public site to see what the publication is for, and specifically fulfil one of the important USFTA requirements.
- Removed json fixture files in favour of adding initial data for submission type and submission status values.
- Added migrations to add data.
- Added non-automatic key fields to Submission and SubmissionType models.
* fix: test environment issue.
- Fixed an issue that caused document uploads to fail in
test environment because CACHES setting was overridden
incorrectly. The test env settings module is also referenced
when executing tests in CircleCI. We could just have a CI specific
env settings module - for now just sniffed for CircleCI execution
environment.
- Tidied all per-env settings for consistency.
* feat: trlst-262 bump python to 3.9.2
- Bumped Dockerfile python layer to 3.9.2
- Bumped PaaS runtime directive accordingly
- Due to 3.9 api changes bumped gevent to latest
- Used latest freezegun due to dep clashes
- pytest now includes catchlog
- pip-compile latest generates more readable txt output
hence significant txt diffs.
- bump circleci python to 3.9.2
- bump pyopenxl to fix test issues
- fixed invalid manifest.yml
* fix: trlst 259 issue post tasks
Investigating an issue under TRLST 259 found some non critical problems in TR API. This PR is mainly a tidy up of the base settings file and the local.env.example file. also bumped the Postgres version used for local development:
- Removed environ.Env.read_env(). We use docker-compose for local development
and feed the container settings from a local.env file. In PaaS we pick up the
app's environment and do not deploy a '.env' file.
- Simplified REDIS URL definition and removed hard coding of database number.
- Simplified CELERY_BROKER_URL definition and removed database number hardcoding.
- Removed 'AWS_DEFAULT_REGION' this is an AWS API client setting and not used
in the codebase.
- Corrected the typo in S3_DOWNLOAD_LINK_EXPIRY_SECONDS var name.
- Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base.
- Removed unused imports.
- Removed cruft comments.
- Pepified.
* fix: Django and Pillow vulnerabilities
- Bumped django to 2.2.18
- Bumped Pillow to 8.1.2
* Fix: Only include cases that are not under preparation.
Investigators using Caseworker portal get confused because "Register an Interest" submissions started by a public user (but not yet finished/submitted) appear under "Cases and Applications" on an Organisation page but don't appear on the Case Page for that organisation.
This fix applies the same filtering on the organisation page so that only completed submissions are displayed.
It uses the ROLE_PREPARING to filter out submissions not yet completed.
* fix: document upload issues
TRLST 271 fix document upload issues:
- Refactored av scan to be more readable and fix flakiness
- Added av_reason to Document model
- Removed unused 'indexed' attribute
- Refactored document tasks to be more robust
* fix: detect when ES not configured properly.
When Elastic search service is not configured in VCAP or using ELASTIC_HOST/PORT the API tries to uses the None returned from get_elastic. This fix tidies this up a bit and logs a proper error.
* fix: vulnerabilities
- Removed django-silk which was unused to resolve pygments and Jinja vulnerabilities.
- Bumped ipython and pygments to resolve vulnerabilities.
Note that iPython bump alone didn't bump pygments but
updated anyway
- Bumped DRF to 3.12.4 to resolve vulnerabilities.
* fix: y18n and lxml vulnerability
* fix: logging config.
Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn.
* feat: assist caseworker submissions page speedup.
Caseworker portal needs to decorate a submission that needs review.
The original approach made the page load excessively slow.
This change adds extra info to a submission to reduce API calls to
enable new submission display.
- Updated SubmissionDocument model 'to_dict' method to include a
'needs_review' value inferred from a submission document that is
safe, and not sufficient or deficient.
- Updated Submission model:
- Moved document inclusion to 'embedded_dict' representation of
a submission and added 'is_new_submission' dict value inferred
from all related submission document statuses.
- Added '_prepare_documents' helper to generate dict representation
of a submission's documents.
- Cleaned up some module cruft.
* Feat/trlst 267 speedup 2 (#66)
A first attempt at improving submissions page performance didn't
work as expected and more doc processing was required. Added a
helper method to Submission model to determine if a submission
requires review (used by caseworker portal), determined as follows:
- The submission's status is not an initial one (default)
- The submission is not version 1
- The document was created by a customer (not TRA)
- At least one document is flagged as needing review (i.e. safe,
not sufficient and not deficient)
Some tidy up included in this change, in particular:
- get_documents is not really used, submission_documents is
favoured. Where it was used (in ApplicationStateAPIView)
the calling Public Portal client doesn't use it. It probably
should - after getting the docs as part of the overall case
application state data the Public portal promptly calls API to
get all documents.
- Some small cosmetic changes to appease linting.
* fix: trlst-299 django vulnerability
- Bump django version to 2.2.20
* fix: TRLST-281 save and continue
This change fixes an issue where a third party invitee's organisation details
were not entirely returned in the json representation of a contact
Endpoint now returns organisation name, address and companies_house_id
in json representation.
* feat: trlst 296 merge 1.5.9 release into develop
Merge 1.5.9 release branch back into develop. Updated
mismatched deps after conflict resolution required
when merging release branch with develop. Also
includes updated version.
* feat: TRLST-306 add pip-tools to dev deps
TRLST 304 introduced an improved mechanism in TR public service
to use pip-compile on the container rather than in the developer's
local env.
However, pip-tools (required for pip-compile) is not in the
API/Public/Caseworker deps and therefore does not get included
in the container build.
This change adds pip-tools as a dev dep, and also performs
on-container build of requirements.
* feat: TRLST-301 add audit log middleware.
Added audit log middleware to service in production.
See https://github.com/uktrade/django-audit-log-middleware.
* feat: TRLST-315 bump python runtime. (#74)
PaaS buildpack python support requires bump of python runtime.
* fix: bump lodash to resolve vulnerability. (#75)
* feat: add IHTC compliance settings
* feat: trlst 327 hardening 2
- Restructured so we can apply settings in lower envs if required.
- included in staging.
- Also tidies logging def inconsistencies.
* hotfix: bump deps to resolve vulnerabilities.
- Bumped django - CVE-2021-31542
- Bumped boto3 and requests to effect urllib3 bump - CVE-2021-33503
* fix: bump Pillow dep to resolve vulnerabilities.
- Bumped Pillow to 8.2.0 - CVE-2021-28677, CVE-2021-25287, CVE-2021-28675,
CVE-2021-25288, CVE-2021-28676, CVE-2021-28678.
* LSGH-7 sec: bump django to resolve vulnerability
* sec: TRLST-342 patch XLSX formula injection
* fix: trlst-308 third party invited org
fix: refactor third party invite processing
* use manager create method
* sanitise country code names
* resolve country/code confusion
* add clearer handling of dict generation issues
* don't recreate invites
* default the tz.
* add invitations related name.
* include contact org id
* copy invites onto cloned submission.
* feat: trlst 219 extend 2fa period
The requirement for this change is to make it possible to extend the Two-Factor Authentication code validity period, but just for the email delivery type. Users in certain regions were experiencing severe lag in 2FA code deliver (via SMS and Email). In lieu of adding an app based authenticator solution (lots of work) special dispensation has been given by cyber to extend the 2FA Code validity period but just for email delivery as sms delivery is deemed less secure and more prone to interception. Summary of changes:
- Added delivery_type to the TwoFactorAuth model, limited to DELIVERY_TYPE_CHOICES. This enables validation of the code's expiry based on delivery type (currently email or sms).
- Added new setting for 2FA code validity for emails.
- General tidy up, improvement of setting and var names, doc strings.
* merge: release 1.5.10 back into develop
* fix: set org on contact.
Fixes various issues with public user onboarding.
auth.py:
- During registration, the API now returns a response to notify
the Public portal correctly so it can redirect the user to
verify email page.
- We can't assume the org role until we establish if this user
is the first org member or not.
models.py
- The organisation creation was happening after the contact was
created hence the contact's org was always none. Moved org creation
after contact creation and also determined correct user role for org
based on existing user count.
* feat: trlst-376 make contact email case specific in notifications.
This change endeavours to add a case specific contact email if a case is in the notification context. If it is not then we fall back to the default contact email system parameter value.
- Removed contact from NOTIFY_BLOCK_FOOTER system param value.
- Added TRADE_REMEDIES_EMAIL_DOMAIN system param so contact email can be optionally assembled.
- Added helpers to `core.notifier` module enable contact email construction based on if a case is in context. Where `get_context` is used to build a personalisation, set contact email and footer base on if a case is in context.
- Used notify helpers to build notification contact email and footer based on case in context in:
- `core.services.api`
- `invitations.models`
- `organisations.models`
* fix: TRLST-372 fix user create
An issue was discovered in API when fixing user create in caseworker.
- Added check ensuring org is not null before attempting purge delete.
- Cherry picked a change in the release branch that fixes an API notification test. While this is unrelated, it was causing circleci build to fail
* fix: more specific catch DoesNotExist.
Note the entire post method is crazy and should be refactored, unfortunately it's pivotal in all document operations. This is a minimal fix pending deeper analysis of code.
Fixes an issue in TR API where, when a child document does not exist, or does not exist for a submission `DoesNotExist` is raised. However the exception catch was too narrow (only caught `Document.DoesNotExist` but need to catch `SubmissionDocument.DoesNotExist` too). Note we still want to distinguish which type of document does not exist.
Existing behaviour has resulted the submission getting into an odd state. If a document to be replaced does not exist then there is no need to then further raise an API Exception.
* merge: release 1.5.11 into develop
* fix: remove redundant components.
Removed redundant UI components that are not used in API
* fix: upgrade pip packages using pip-compile
* fix: bump pillow dep.
* Change file path in fixture.sh to pass PaaS
* line break
* Remove user group case assignment deletion
* fix: resolve 500 error when a case link is clicked in new applications
* fix: TRLST-484 server error for invited case investigators
* Dave comments
* Removing pre-commit hook incorrectly committed
* dave comments
* Create less volatile change to fix new applications
* reverting experimental change
* Only impact the filter for new applications
* Variable call for conciseness
* merge: release 1.5.12 into develop
- Version bump
- Runtime bump
- pip-compile conflict resolutions
* feature: pre-commit hooks
* fix: upgrade ansi-regex package
* fix: ignore NER output file.
* fix: remove ner file.
* TRLST-489 - Allow notices to be committed
* Remove comments
* Implementing a more appropriate solution
* fix: fix/TRLST-493 check there is a submission on an invite.
The invitation model does not necessarily have a submission associated with it so we need to guard our access to a submission on the invitation.
* Change variable names
* fix: implement s3_download_link_expiry_seconds variable in env file and extend length (#114)
* fix: do not display redacted users as a contact
* Refactor: TRLST-475 Remove av scan logic
This change removes all the Anti-virus scan logic in the API as it's now performed in the Public and Caseworker portals independently using the django_chunk_upload_handlers package.
* Updated Document model.
* Removed redundant av scan module.
* Removed unused settings.
* Removed beat task that is no longer required.
This is because prepare_document now only gets the S3 object etag checksum so a beat task is overkill.
* feature: implement search bar to simplify debugging process (#115)
* Fix: TRLST-491 allow users to be unassigned from a case (#118)
* feature: add statement of intended preliminary decision as option (#121)
* merge: TRLST-499 release into develop
- version bump
* fix: backs out TRLST-393 change.
- Backs out TRLST-393 change.
- Adds missing 'user_case' flag when returning a user's cases.
* feat: TRSV2-31 add initial V2 apps (part 1)
- moved trade_remedies_api -> config
- tidied README
- updated PaaS config
- refined dev requirements
- moved static root
* feat: implement initial V2 apps
Set of changes in preparation for initial refactoring work to reimplement the API authentication and authorisation components.
- Introduced ADR mechanism. ADRs will be a light and useful way to propose and document significant decisions.
- Added initial ADRs for review:
- 001 Use ADRs
- 002 Major TR Refactor
- 003 Use Token Authentication
- 004 Use Django RESTQL
- Flake and black checks are commented out in CI. Have introduced flake checks, black ready to go there are ~30 changes required which we can do under a separate PR.
- Fixed some flake issues.
- Added new `auth` app with initial some placeholder modules.
* feature: consolidate .flake8 into pyproject.toml and run black (#127)
* feat: approve adrs (#126)
* merge: TRLST-528 release into develop
* fix: LSGH-64 lxml vulnerability bump package
* fix: bump django version to 2.2.25
* fix: TRLST-535 speed up doc processing.
* LSGH-71 - Upgrade celery
* feat: TRSV2-32 New Models
This work implements a new authentication app that should be used to replace the existing authentication logic in the API. The implementation is switched off with the API_V2_ENABLED setting (defaults to False) and has no effect on the behaviour of existing authentication code. Test coverage of this package is 100%. Tests are all in trade_remedies_api/tests. Run tests with something like:
API_V2_ENABLED=true pytest trade_remedies_api/tests/
* fix: remove redundant clam av settings. (#134)
* fix: upgrade pillow to 9.0.0
* Upgrade django to 2.2.26
* fix: upgrade redis package to be compatible with latest celery upgrade (#139)
* TRLST-555 - Upgrading node-fetch to >= 2.6.7 due to CVE-2022-0235 (#141)
* TRLST-551 - upgrading iPython to >= 7.31.1 (#140)
* Create management command for user contact organisation (#136)
* Feature/trsv2 105 (#142)
* preliminary work on the migration
* Migration to OpenSearch partially completed, will run tests in develop environment to find out any problems which could occur during the production migration.
* Removed some unused code to do with SSL authentication of the OpenSearch instance, no longer necessary as all requests are made over HTTP
* TRSV2-105 Tests and Refactoring
1. Added tests of the opensearch.py file, stored in core/tests/test_open_search.py
2. Refactored and renamed any old references to ElasticSearch
3. Updated the local.env.example to reflect the new name changes
4. Added steps in the CircleCI config.yml file to add an OpenSearch instance so these new tests will run
5. Removed elasticsearch Python library from requirements.in
* TRSV2-105 Refactoring
5. Removed elasticsearch Python library from requirements.in
* Updated opensearch image in CircleCI config.yml as 1.0.0 is causing a high GID error
* Removed overridden settings variables from test_open_search.py as CircleCI already had the correct settings variables and didn't need them to be overridden
* test_open_search.py flake8 code formatting error
* Removed redundant assert statement from test_open_search.py test
* Updated version.py to 1.5.15
* Added package.json overrides key to force node-fetch >= 2.6.7
Co-authored-by: Dave Charles <[email protected]>
Co-authored-by: Luisella Strona <[email protected]>
Co-authored-by: Ross Miller <[email protected]>
Co-authored-by: Tash Boyse <[email protected]>
Co-authored-by: nboyse <[email protected]>
* Release 1 5 16 (#149)
* fix: tidy and refactor base api view.
- Fixed issue with pagination support
- Fixed anomaly in user/group memebrship logic
- Added documentation
- Added proper initialisation of TradeRemediesApiView members
- Removed cruft
* fix: add optional args to base constructor.
* merge: release 1.5.7 into develop
This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes:
* Update traitlets
* Rationalise Procfile triggered processes
* Branding changes
* Dependabot alert fixes
* Patch front end vulnerability
* bump version to 1.5.7
* hotfix: TRLST 242 optimise logging in API project
* merge: 1.5.7.1 hotfix into develop (trlst 252) (#48)
Merge 1.5.7.1 hotfix (to make logging more robust) into develop (trlst-252)
- Django signal receivers used to log db events cannot depend on
instance attrs being present so made them more forgiving so we
can guarantee the logging to debug issues. As additional belt
and braces, updated receivers to not rely on instance presence.
While this is unlikely, maybe possible if receiver is misconfigured.
- added pytest catchlog.
- added tests.
- added pytest config.
* fix: TRLST_255 traceback when org is null.
Recent refactoring of trade_remedies_api/core/services/base.py base view introduced a small issue where, when the view is invoked, in certain circumstances the organisation is null and referencing its id causes a traceback. This was spotted in the logs but not visible to the user on the UI (the error seems to be dutifully ignored and the API responds to the client normally, however this could cause issues when called from other pages in the Public and Caseworker portal).
* feat: trlst-40 document submission sub-categories
As a caseworker, I need the ability to ADD a subcategory when I publish a document so that it is clearer for the users of the Public site to see what the publication is for, and specifically fulfil one of the important USFTA requirements.
- Removed json fixture files in favour of adding initial data for submission type and submission status values.
- Added migrations to add data.
- Added non-automatic key fields to Submission and SubmissionType models.
* fix: test environment issue.
- Fixed an issue that caused document uploads to fail in
test environment because CACHES setting was overridden
incorrectly. The test env settings module is also referenced
when executing tests in CircleCI. We could just have a CI specific
env settings module - for now just sniffed for CircleCI execution
environment.
- Tidied all per-env settings for consistency.
* feat: trlst-262 bump python to 3.9.2
- Bumped Dockerfile python layer to 3.9.2
- Bumped PaaS runtime directive accordingly
- Due to 3.9 api changes bumped gevent to latest
- Used latest freezegun due to dep clashes
- pytest now includes catchlog
- pip-compile latest generates more readable txt output
hence significant txt diffs.
- bump circleci python to 3.9.2
- bump pyopenxl to fix test issues
- fixed invalid manifest.yml
* fix: trlst 259 issue post tasks
Investigating an issue under TRLST 259 found some non critical problems in TR API. This PR is mainly a tidy up of the base settings file and the local.env.example file. also bumped the Postgres version used for local development:
- Removed environ.Env.read_env(). We use docker-compose for local development
and feed the container settings from a local.env file. In PaaS we pick up the
app's environment and do not deploy a '.env' file.
- Simplified REDIS URL definition and removed hard coding of database number.
- Simplified CELERY_BROKER_URL definition and removed database number hardcoding.
- Removed 'AWS_DEFAULT_REGION' this is an AWS API client setting and not used
in the codebase.
- Corrected the typo in S3_DOWNLOAD_LINK_EXPIRY_SECONDS var name.
- Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base.
- Removed unused imports.
- Removed cruft comments.
- Pepified.
* fix: Django and Pillow vulnerabilities
- Bumped django to 2.2.18
- Bumped Pillow to 8.1.2
* Fix: Only include cases that are not under preparation.
Investigators using Caseworker portal get confused because "Register an Interest" submissions started by a public user (but not yet finished/submitted) appear under "Cases and Applications" on an Organisation page but don't appear on the Case Page for that organisation.
This fix applies the same filtering on the organisation page so that only completed submissions are displayed.
It uses the ROLE_PREPARING to filter out submissions not yet completed.
* fix: document upload issues
TRLST 271 fix document upload issues:
- Refactored av scan to be more readable and fix flakiness
- Added av_reason to Document model
- Removed unused 'indexed' attribute
- Refactored document tasks to be more robust
* fix: detect when ES not configured properly.
When Elastic search service is not configured in VCAP or using ELASTIC_HOST/PORT the API tries to uses the None returned from get_elastic. This fix tidies this up a bit and logs a proper error.
* fix: vulnerabilities
- Removed django-silk which was unused to resolve pygments and Jinja vulnerabilities.
- Bumped ipython and pygments to resolve vulnerabilities.
Note that iPython bump alone didn't bump pygments but
updated anyway
- Bumped DRF to 3.12.4 to resolve vulnerabilities.
* fix: y18n and lxml vulnerability
* fix: logging config.
Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn.
* feat: assist caseworker submissions page speedup.
Caseworker portal needs to decorate a submission that needs review.
The original approach made the page load excessively slow.
This change adds extra info to a submission to reduce API calls to
enable new submission display.
- Updated SubmissionDocument model 'to_dict' method to include a
'needs_review' value inferred from a submission document that is
safe, and not sufficient or deficient.
- Updated Submission model:
- Moved document inclusion to 'embedded_dict' representation of
a submission and added 'is_new_submission' dict value inferred
from all related submission document statuses.
- Added '_prepare_documents' helper to generate dict representation
of a submission's documents.
- Cleaned up some module cruft.
* Feat/trlst 267 speedup 2 (#66)
A first attempt at improving submissions page performance didn't
work as expected and more doc processing was required. Added a
helper method to Submission model to determine if a submission
requires review (used by caseworker portal), determined as follows:
- The submission's status is not an initial one (default)
- The submission is not version 1
- The document was created by a customer (not TRA)
- At least one document is flagged as needing review (i.e. safe,
not sufficient and not deficient)
Some tidy up included in this change, in particular:
- get_documents is not really used, submission_documents is
favoured. Where it was used (in ApplicationStateAPIView)
the calling Public Portal client doesn't use it. It probably
should - after getting the docs as part of the overall case
application state data the Public portal promptly calls API to
get all documents.
- Some small cosmetic changes to appease linting.
* fix: trlst-299 django vulnerability
- Bump django version to 2.2.20
* fix: TRLST-281 save and continue
This change fixes an issue where a third party invitee's organisation details
were not entirely returned in the json representation of a contact
Endpoint now returns organisation name, address and companies_house_id
in json representation.
* feat: trlst 296 merge 1.5.9 release into develop
Merge 1.5.9 release branch back into develop. Updated
mismatched deps after conflict resolution required
when merging release branch with develop. Also
includes updated version.
* feat: TRLST-306 add pip-tools to dev deps
TRLST 304 introduced an improved mechanism in TR public service
to use pip-compile on the container rather than in the developer's
local env.
However, pip-tools (required for pip-compile) is not in the
API/Public/Caseworker deps and therefore does not get included
in the container build.
This change adds pip-tools as a dev dep, and also performs
on-container build of requirements.
* feat: TRLST-301 add audit log middleware.
Added audit log middleware to service in production.
See https://github.com/uktrade/django-audit-log-middleware.
* feat: TRLST-315 bump python runtime. (#74)
PaaS buildpack python support requires bump of python runtime.
* fix: bump lodash to resolve vulnerability. (#75)
* feat: add IHTC compliance settings
* feat: trlst 327 hardening 2
- Restructured so we can apply settings in lower envs if required.
- included in staging.
- Also tidies logging def inconsistencies.
* hotfix: bump deps to resolve vulnerabilities.
- Bumped django - CVE-2021-31542
- Bumped boto3 and requests to effect urllib3 bump - CVE-2021-33503
* fix: bump Pillow dep to resolve vulnerabilities.
- Bumped Pillow to 8.2.0 - CVE-2021-28677, CVE-2021-25287, CVE-2021-28675,
CVE-2021-25288, CVE-2021-28676, CVE-2021-28678.
* LSGH-7 sec: bump django to resolve vulnerability
* sec: TRLST-342 patch XLSX formula injection
* fix: trlst-308 third party invited org
fix: refactor third party invite processing
* use manager create method
* sanitise country code names
* resolve country/code confusion
* add clearer handling of dict generation issues
* don't recreate invites
* default the tz.
* add invitations related name.
* include contact org id
* copy invites onto cloned submission.
* feat: trlst 219 extend 2fa period
The requirement for this change is to make it possible to extend the Two-Factor Authentication code validity period, but just for the email delivery type. Users in certain regions were experiencing severe lag in 2FA code deliver (via SMS and Email). In lieu of adding an app based authenticator solution (lots of work) special dispensation has been given by cyber to extend the 2FA Code validity period but just for email delivery as sms delivery is deemed less secure and more prone to interception. Summary of changes:
- Added delivery_type to the TwoFactorAuth model, limited to DELIVERY_TYPE_CHOICES. This enables validation of the code's expiry based on delivery type (currently email or sms).
- Added new setting for 2FA code validity for emails.
- General tidy up, improvement of setting and var names, doc strings.
* merge: release 1.5.10 back into develop
* fix: set org on contact.
Fixes various issues with public user onboarding.
auth.py:
- During registration, the API now returns a response to notify
the Public portal correctly so it can redirect the user to
verify email page.
- We can't assume the org role until we establish if this user
is the first org member or not.
models.py
- The organisation creation was happening after the contact was
created hence the contact's org was always none. Moved org creation
after contact creation and also determined correct user role for org
based on existing user count.
* feat: trlst-376 make contact email case specific in notifications.
This change endeavours to add a case specific contact email if a case is in the notification context. If it is not then we fall back to the default contact email system parameter value.
- Removed contact from NOTIFY_BLOCK_FOOTER system param value.
- Added TRADE_REMEDIES_EMAIL_DOMAIN system param so contact email can be optionally assembled.
- Added helpers to `core.notifier` module enable contact email construction based on if a case is in context. Where `get_context` is used to build a personalisation, set contact email and footer base on if a case is in context.
- Used notify helpers to build notification contact email and footer based on case in context in:
- `core.services.api`
- `invitations.models`
- `organisations.models`
* fix: TRLST-372 fix user create
An issue was discovered in API when fixing user create in caseworker.
- Added check ensuring org is not null before attempting purge delete.
- Cherry picked a change in the release branch that fixes an API notification test. While this is unrelated, it was causing circleci build to fail
* fix: more specific catch DoesNotExist.
Note the entire post method is crazy and should be refactored, unfortunately it's pivotal in all document operations. This is a minimal fix pending deeper analysis of code.
Fixes an issue in TR API where, when a child document does not exist, or does not exist for a submission `DoesNotExist` is raised. However the exception catch was too narrow (only caught `Document.DoesNotExist` but need to catch `SubmissionDocument.DoesNotExist` too). Note we still want to distinguish which type of document does not exist.
Existing behaviour has resulted the submission getting into an odd state. If a document to be replaced does not exist then there is no need to then further raise an API Exception.
* merge: release 1.5.11 into develop
* fix: remove redundant components.
Removed redundant UI components that are not used in API
* fix: upgrade pip packages using pip-compile
* fix: bump pillow dep.
* Change file path in fixture.sh to pass PaaS
* line break
* Remove user group case assignment deletion
* fix: resolve 500 error when a case link is clicked in new applications
* fix: TRLST-484 server error for invited case investigators
* Dave comments
* Removing pre-commit hook incorrectly committed
* dave comments
* Create less volatile change to fix new applications
* reverting experimental change
* Only impact the filter for new applications
* Variable call for conciseness
* merge: release 1.5.12 into develop
- Version bump
- Runtime bump
- pip-compile conflict resolutions
* feature: pre-commit hooks
* fix: upgrade ansi-regex package
* fix: ignore NER output file.
* fix: remove ner file.
* TRLST-489 - Allow notices to be committed
* Remove comments
* Implementing a more appropriate solution
* fix: fix/TRLST-493 check there is a submission on an invite.
The invitation model does not necessarily have a submission associated with it so we need to guard our access to a submission on the invitation.
* Change variable names
* fix: implement s3_download_link_expiry_seconds variable in env file and extend length (#114)
* fix: do not display redacted users as a contact
* Refactor: TRLST-475 Remove av scan logic
This change removes all the Anti-virus scan logic in the API as it's now performed in the Public and Caseworker portals independently using the django_chunk_upload_handlers package.
* Updated Document model.
* Removed redundant av scan module.
* Removed unused settings.
* Removed beat task that is no longer required.
This is because prepare_document now only gets the S3 object etag checksum so a beat task is overkill.
* feature: implement search bar to simplify debugging process (#115)
* Fix: TRLST-491 allow users to be unassigned from a case (#118)
* feature: add statement of intended preliminary decision as option (#121)
* merge: TRLST-499 release into develop
- version bump
* fix: backs out TRLST-393 change.
- Backs out TRLST-393 change.
- Adds missing 'user_case' flag when returning a user's cases.
* feat: TRSV2-31 add initial V2 apps (part 1)
- moved trade_remedies_api -> config
- tidied README
- updated PaaS config
- refined dev requirements
- moved static root
* feat: implement initial V2 apps
Set of changes in preparation for initial refactoring work to reimplement the API authentication and authorisation components.
- Introduced ADR mechanism. ADRs will be a light and useful way to propose and document significant decisions.
- Added initial ADRs for review:
- 001 Use ADRs
- 002 Major TR Refactor
- 003 Use Token Authentication
- 004 Use Django RESTQL
- Flake and black checks are commented out in CI. Have introduced flake checks, black ready to go there are ~30 changes required which we can do under a separate PR.
- Fixed some flake issues.
- Added new `auth` app with initial some placeholder modules.
* feature: consolidate .flake8 into pyproject.toml and run black (#127)
* feat: approve adrs (#126)
* merge: TRLST-528 release into develop
* fix: LSGH-64 lxml vulnerability bump package
* fix: bump django version to 2.2.25
* fix: TRLST-535 speed up doc processing.
* LSGH-71 - Upgrade celery
* feat: TRSV2-32 New Models
This work implements a new authentication app that should be used to replace the existing authentication logic in the API. The implementation is switched off with the API_V2_ENABLED setting (defaults to False) and has no effect on the behaviour of existing authentication code. Test coverage of this package is 100%. Tests are all in trade_remedies_api/tests. Run tests with something like:
API_V2_ENABLED=true pytest trade_remedies_api/tests/
* fix: remove redundant clam av settings. (#134)
* fix: upgrade pillow to 9.0.0
* Upgrade django to 2.2.26
* fix: upgrade redis package to be compatible with latest celery upgrade (#139)
* TRLST-555 - Upgrading node-fetch to >= 2.6.7 due to CVE-2022-0235 (#141)
* TRLST-551 - upgrading iPython to >= 7.31.1 (#140)
* Create management command for user contact organisation (#136)
* Feature/trsv2 105 (#142)
* preliminary work on the migration
* Migration to OpenSearch partially completed, will run tests in develop environment to find out any problems which could occur during the production migration.
* Removed some unused code to do with SSL authentication of the OpenSearch instance, no longer necessary as all requests are made over HTTP
* TRSV2-105 Tests and Refactoring
1. Added tests of the opensearch.py file, stored in core/tests/test_open_search.py
2. Refactored and renamed any old references to ElasticSearch
3. Updated the local.env.example to reflect the new name changes
4. Added steps in the CircleCI config.yml file to add an OpenSearch instance so these new tests will run
5. Removed elasticsearch Python library from requirements.in
* TRSV2-105 Refactoring
5. Removed elasticsearch Python library from requirements.in
* Updated opensearch image in CircleCI config.yml as 1.0.0 is causing a high GID error
* Removed overridden settings variables from test_open_search.py as CircleCI already had the correct settings variables and didn't need them to be overridden
* test_open_search.py flake8 code formatting error
* Removed redundant assert statement from test_open_search.py test
* TRLST-550 - Merge release back into develop (#144)
* Release 1.5.6
* Update traitlets
* Black and flake8 checks
* Use pytest in circleci
* Rationalise Procfile triggered processes
* Branding changes
* Bump version
* Fix dependabot alert fixes
* Patch front end vulnerability
* bump version to 1.5.7
* feat: add management command to patch audit logs
Changes to speed up model exports include an Audit's case title
pre-compute. However existing audit logs won't have the pre-computed
values in their json field. This management command patches all
audit logs accordingly.
* feat: test patch_audit managment command
* hotfix: TRLST 242 optimise logging in API project
* hotfix: robust logging trlst-252
hotfix to make logging more robust (trlst-252)
- Django signal receivers used to log db events cannot depend on
instance attrs being present so made them more forgiving so we
can guarantee the logging to debug issues. As additional belt
and braces, updated receivers to not rely on instance presence.
While this is unlikely, maybe possible if receiver is misconfigured.
- added pytest catchlog.
- added tests.
- added pytest config.
* release 1.5.8
Merge 1.5.8 release branch into master branch
* hotfix: trlst 282 high sev vulnerabilities
hotfix: release for Django and Pillow vulnerabilities
- Bumped django to 2.2.18
- Bumped Pillow to 8.1.2
* hotfix: high severity vulnerabilities.
- Applies fixes for y18n and lxml vulnerabilities
- Bump version to 1.5.8.2
* merge: release 1.5.9 branch into master
Release TR API 1.5.9
* hotfix: bump python runtime
* hotfix: bump deps to resolve vulnerabilities.
- Bumped django - CVE-2021-31542
- Bumped boto3, requests, urllib3 - CVE-2021-33503
- Bump service version
* hotfix: bump Pillow dep to resolve vulnerabilities.
- Bumped Pillow to 8.2.0 - CVE-2021-28677, CVE-2021-25287, CVE-2021-28675,
CVE-2021-25288, CVE-2021-28676, CVE-2021-28678.
* LSGH-7 sec: bump django to resolve vulnerability
* merge: release 1.5.10 branch into master
* merge: release 1.5.11 branch into master
* Release 1.5.12
* fix: bump python runtime to 3.9.7.
* hotfix: TRLST-493-invite-fails.
This change applies the fix for TRLST-493 and elements of fix for TRLST-484 to resolve 500 error raise when a public user registers.
* add check that there is a submission on the invite.
* Add guard group name check.
* Added pre-create of user profile in invite processing.
* bumped version to 1.5.12.1
* Release 1.5.13
* Release 1.5.14
* Updated version.py to 1.5.15
* Added package.json overrides key to force node-fetch >= 2.6.7
Co-authored-by: Mark Higham <[email protected]>
Co-authored-by: Ross Miller <[email protected]>
Co-authored-by: davecharles <[email protected]>
* fix: adjust management command to iterate correctly through arguments (#146)
* Fix: ensure only inactive users are deleted from case (#145)
* Trlst-536 add radio buttons (#147)
* TRLST-536 - review radio buttons now appear when a notice is selected from the dropdown when a user is applying for a new investigation against an existing case/notice
* flake8
* Added breach of undertakings to show for notices selected when applying for a new investigation
* black code refactoring
* Updated tests to reflect the recent change that BREACH_OF_UNDERTAKINGS review type should show for notices
Co-authored-by: nboyse <[email protected]>
* Trlst 536 add radio buttons (#148)
* TRLST-536 - review radio buttons now appear when a notice is selected from the dropdown when a user is applying for a new investigation against an existing case/notice
* flake8
* Added breach of undertakings to show for notices selected when applying for a new investigation
* black code refactoring
* Updated tests to reflect the recent change that BREACH_OF_UNDERTAKINGS review type should show for notices
* git-hook secret and PII check
* Added pii-secret-hook/ to gitignore and deleted files
Co-authored-by: nboyse <[email protected]>
* Updated pii-exclude
Co-authored-by: Dave Charles <[email protected]>
Co-authored-by: Luisella Strona <[email protected]>
Co-authored-by: Ross Miller <[email protected]>
Co-authored-by: Tash Boyse <[email protected]>
Co-authored-by: nboyse <[email protected]>
Co-authored-by: Mark Higham <[email protected]>
* Release 1.5.17 (#156)
* fix: tidy and refactor base api view.
- Fixed issue with pagination support
- Fixed anomaly in user/group memebrship logic
- Added documentation
- Added proper initialisation of TradeRemediesApiView members
- Removed cruft
* fix: add optional args to base constructor.
* merge: release 1.5.7 into develop
This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes:
* Update traitlets
* Rationalise Procfile triggered processes
* Branding changes
* Dependabot alert fixes
* Patch front end vulnerability
* bump version to 1.5.7
* hotfix: TRLST 242 optimise logging in API project
* merge: 1.5.7.1 hotfix into develop (trlst 252) (#48)
Merge 1.5.7.1 hotfix (to make logging more robust) into develop (trlst-252)
- Django signal receivers used to log db events cannot depend on
instance attrs being present so made them more forgiving so we
can guarantee the logging to debug issues. As additional belt
and braces, updated receivers to not rely on instance presence.
While this is unlikely, maybe possible if receiver is misconfigured.
- added pytest catchlog.
- added tests.
- added pytest config.
* fix: TRLST_255 traceback when org is null.
Recent refactoring of trade_remedies_api/core/services/base.py base view introduced a small issue where, when the view is invoked, in certain circumstances the organisation is null and referencing its id causes a traceback. This was spotted in the logs but not visible to the user on the UI (the error seems to be dutifully ignored and the API responds to the client normally, however this could cause issues when called from other pages in the Public and Caseworker portal).
* feat: trlst-40 document submission sub-categories
As a caseworker, I need the ability to ADD a subcategory when I publish a document so that it is clearer for the users of the Public site to see what the publication is for, and specifically fulfil one of the important USFTA requirements.
- Removed json fixture files in favour of adding initial data for submission type and submission status values.
- Added migrations to add data.
- Added non-automatic key fields to Submission and SubmissionType models.
* fix: test environment issue.
- Fixed an issue that caused document uploads to fail in
test environment because CACHES setting was overridden
incorrectly. The test env settings module is also referenced
when executing tests in CircleCI. We could just have a CI specific
env settings module - for now just sniffed for CircleCI execution
environment.
- Tidied all per-env settings for consistency.
* feat: trlst-262 bump python to 3.9.2
- Bumped Dockerfile python layer to 3.9.2
- Bumped PaaS runtime directive accordingly
- Due to 3.9 api changes bumped gevent to latest
- Used latest freezegun due to dep clashes
- pytest now includes catchlog
- pip-compile latest generates more readable txt output
hence significant txt diffs.
- bump circleci python to 3.9.2
- bump pyopenxl to fix test issues
- fixed invalid manifest.yml
* fix: trlst 259 issue post tasks
Investigating an issue under TRLST 259 found some non critical problems in TR API. This PR is mainly a tidy up of the base settings file and the local.env.example file. also bumped the Postgres version used for local development:
- Removed environ.Env.read_env(). We use docker-compose for local development
and feed the container settings from a local.env file. In PaaS we pick up the
app's environment and do not deploy a '.env' file.
- Simplified REDIS URL definition and removed hard coding of database number.
- Simplified CELERY_BROKER_URL definition and removed database number hardcoding.
- Removed 'AWS_DEFAULT_REGION' this is an AWS API client setting and not used
in the codebase.
- Corrected the typo in S3_DOWNLOAD_LINK_EXPIRY_SECONDS var name.
- Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base.
- Removed unused imports.
- Removed cruft comments.
- Pepified.
* fix: Django and Pillow vulnerabilities
- Bumped django to 2.2.18
- Bumped Pillow to 8.1.2
* Fix: Only include cases that are not under preparation.
Investigators using Caseworker portal get confused because "Register an Interest" submissions started by a public user (but not yet finished/submitted) appear under "Cases and Applications" on an Organisation page but don't appear on the Case Page for that organisation.
This fix applies the same filtering on the organisation page so that only completed submissions are displayed.
It uses the ROLE_PREPARING to filter out submissions not yet completed.
* fix: document upload issues
TRLST 271 fix document upload issues:
- Refactored av scan to be more readable and fix flakiness
- Added av_reason to Document model
- Removed unused 'indexed' attribute
- Refactored document tasks to be more robust
* fix: detect when ES not configured properly.
When Elastic search service is not configured in VCAP or using ELASTIC_HOST/PORT the API tries to uses the None returned from get_elastic. This fix tidies this up a bit and logs a proper error.
* fix: vulnerabilities
- Removed django-silk which was unused to resolve pygments and Jinja vulnerabilities.
- Bumped ipython and pygments to resolve vulnerabilities.
Note that iPython bump alone didn't bump pygments but
updated anyway
- Bumped DRF to 3.12.4 to resolve vulnerabilities.
* fix: y18n and lxml vulnerability
* fix: logging config.
Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn.
* feat: assist caseworker submissions page speedup.
Caseworker portal needs to decorate a submission that needs review.
The original approach made the page load excessively slow.
This change adds extra info to a submission to reduce API calls to
enable new submission display.
- Updated SubmissionDocument model 'to_dict' method to include a
'needs_review' value inferred from a submission document that is
safe, and not sufficient or deficient.
- Updated Submission model:
- Moved document inclusion to 'embedded_dict' representation of
a submission and added 'is_new_submission' dict value inferred
from all related submission document statuses.
- Added '_prepare_documents' helper to generate dict representation
of a submission's documents.
- Cleaned up some module cruft.
* Feat/trlst 267 speedup 2 (#66)
A first attempt at improving submissions page performance didn't
work as expected and more doc processing was required. Added a
helper method to Submission model to determine if a submission
requires review (used by caseworker portal), determined as follows:
- The submission's status is not an initial one (default)
- The submission is not version 1
- The document was created by a customer (not TRA)
- At least one document is flagged as needing review (i.e. safe,
not sufficient and not deficient)
Some tidy up included in this change, in particular:
- get_documents is not really used, submission_documents is
favoured. Where it was used (in ApplicationStateAPIView)
the calling Public Portal client doesn't use it. It probably
should - after getting the docs as part of the overall case
application state data the Public portal promptly calls API to
get all documents.
- Some small cosmetic changes to appease linting.
* fix: trlst-299 django vulnerability
- Bump django version to 2.2.20
* fix: TRLST-281 save and continue
This change fixes an issue where a third party invitee's organisation details
were not entirely returned in the json representation of a contact
Endpoint now returns organisation name, address and companies_house_id
in json representation.
* feat: trlst 296 merge 1.5.9 release into develop
Merge 1.5.9 release branch back into develop. Updated
mismatched deps after conflict resolution required
when merging release branch with develop. Also
includes updated version.
* feat: TRLST-306 add pip-tools to dev deps
TRLST 304 introduced an improved mechanism in TR public service
to use pip-compile on the container rather than in the developer's
local env.
However, pip-tools (required for pip-compile) is not in the
API/Public/Caseworker deps and therefore does not get included
in the container build.
This change adds pip-tools as a dev dep, and also performs
on-container build of requirements.
* feat: TRLST-301 add audit log middleware.
Added audit log middleware to service in production.
See https://github.com/uktrade/django-audit-log-middleware.
* feat: TRLST-315 bump python runtime. (#74)
PaaS buildpack python support requires bump of python runtime.
* fix: bump lodash to resolve vulnerability. (#75)
* feat: add IHTC compliance settings
* feat: trlst 327 hardening 2
- Restructured so we can apply settings in lower envs if required.
- included in staging.
- Also tidies logging def inconsistencies.
* hotfix: bump deps to resolve vulnerabilities.
- Bumped django - CVE-2021-31542
- Bumped boto3 and requests to effect urllib3 bump - CVE-2021-33503
* fix: bump Pillow dep to resolve vulnerabilities.
- Bumped Pillow to 8.2.0 - CVE-2021-28677, CVE-2021-25287, CVE-2021-28675,
CVE-2021-25288, CVE-2021-28676, CVE-2021-28678.
* LSGH-7 sec: bump django to resolve vulnerability
* sec: TRLST-342 patch XLSX formula injection
* fix: trlst-308 third party invited org
fix: refactor third party invite processing
* use manager create method
* sanitise country code names
* resolve country/code confusion
* add clearer handling of dict generation issues
* don't recreate invites
* default the tz.
* add invitations related name.
* include contact org id
* copy invites onto cloned submission.
* feat: trlst 219 extend 2fa period
The requirement for this change is to make it possible to extend the Two-Factor Authentication code validity period, but just for the email delivery type. Users in certain regions were experiencing severe lag in 2FA code deliver (via SMS and Email). In lieu of adding an app based authenticator solution (lots of work) special dispensation has been given by cyber to extend the 2FA Code validity period but just for email delivery as sms delivery is deemed less secure and more prone to interception. Summary of changes:
- Added delivery_type to the TwoFactorAuth model, limited to DELIVERY_TYPE_CHOICES. This enables validation of the code's expiry based on delivery type (currently email or sms).
- Added new setting for 2FA code validity for emails.
- General tidy up, improvement of setting and var names, doc strings.
* merge: release 1.5.10 back into develop
* fix: set org on contact.
Fixes various issues with public user onboarding.
auth.py:
- During registration, the API now returns a response to notify
the Public portal correctly so it can redirect the user to
verify email page.
- We can't assume the org role until we establish if this user
is the first org member or not.
models.py
- The organisation creation was happening after the contact was
created hence the contact's org was always none. Moved org creation
after contact creation and also determined correct user role for org
based on existing user count.
* feat: trlst-376 make contact email case specific in notifications.
This change endeavours to add a case specific contact email if a case is in the notification context. If it is not then we fall back to the default contact email system parameter value.
- Removed contact from NOTIFY_BLOCK_FOOTER system param value.
- Added TRADE_REMEDIES_EMAIL_DOMAIN system param so contact email can be optionally assembled.
- Added helpers to `core.notifier` module enable contact email construction based on if a case is in context. Where `get_context` is used to build a personalisation, set contact email and footer base on if a case is in context.
- Used notify helpers to build notification contact email and footer based on case in context in:
- `core.services.api`
- `invitations.models`
- `organisations.models`
* fix: TRLST-372 fix user create
An issue was discovered in API when fixing user create in caseworker.
- Added check ensuring org is not null before attempting purge delete.
- Cherry picked a change in the release branch that fixes an API notification test. While this is unrelated, it was causing circleci build to fail
* fix: more specific catch DoesNotExist.
Note the entire post method is crazy and should be refactored, unfortunately it's pivotal in all document operations. This is a minimal fix pending deeper analysis of code.
Fixes an issue in TR API where, when a child document does not exist, or does not exist for a submission `DoesNotExist` is raised. However the exception catch was too narrow (only caught `Document.DoesNotExist` but need to catch `SubmissionDocument.DoesNotExist` too). Note we still want to distinguish which type of document does not exist.
Existing behaviour has resulted the submission getting into an odd state. If a document to be replaced does not exist then there is no need to then further raise an API Exception.
* merge: release 1.5.11 into develop
* fix: remove redundant components.
Removed redundant UI components that are not used in API
* fix: upgrade pip packages using pip-compile
* fix: bump pillow dep.
* Change file path in fixture.sh to pass PaaS
* line break
* Remove user group case assignment deletion
* fix: resolve 500 error when a case link is clicked in new applications
* fix: TRLST-484 server error for invited case investigators
* Dave comments
* Removing pre-commit hook incorrectly committed
* dave comments
* Create less volatile change to fix new applications
* reverting experimental change
* Only impact the filter for new applications
* Variable call for conciseness
* merge: release 1.5.12 into develop
- Version bump
- Runtime bump
- pip-compile conflict resolutions
* feature: pre-commit hooks
* fix: upgrade ansi-regex package
* fix: ignore NER output file.
* fix: remove ner file.
* TRLST-489 - Allow notices to be committed
* Remove comments
* Implementing a more appropriate solution
* fix: fix/TRLST-493 check there is a submission on an invite.
The invitation model does not necessarily have a submission associated with it so we need to guard our access to a submission on the invitation.
* Change variable names
* fix: implement s3_download_link_expiry_seconds variable in env file and extend length (#114)
* fix: do not display redacted users as a contact
* Refactor: TRLST-475 Remove av scan logic
This change removes all the Anti-virus scan logic in the API as it's now performed in the Public and Caseworker portals independently using the django_chunk_upload_handlers package.
* Updated Document model.
* Removed redundant av scan module.
* Removed unused settings.
* Removed beat task that is no longer required.
This is because prepare_document now only gets the S3 object etag checksum so a beat task is overkill.
* feature: implement search bar to simplify debugging process (#115)
* Fix: TRLST-491 allow users to be unassigned from a case (#118)
* feature: add statement of intended preliminary decision as option (#121)
* merge: TRLST-499 release into develop
- version bump
* fix: backs out TRLST-393 change.
- Backs out TRLST-393 change.
- Adds missing 'user_case' flag when returning a user's cases.
* feat: TRSV2-31 add initial V2 apps (part 1)
- moved trade_remedies_api -> config
- tidied README
- updated PaaS config
- refined dev requirements
- moved static root
* feat: implement initial V2 apps
Set of changes in preparation for initial refactoring work to reimplement the API authentication and authorisation components.
- Introduced ADR mechanism. ADRs will be a light and useful way to propose and document significant decisions.
- Added initial ADRs for review:
- 001 Use ADRs
- 002 Major TR Refactor
- 003 Use Token Authentication
- 004 Use Django RESTQL
- Flake and black checks are commented out in CI. Have introduced flake checks, black ready to go there are ~30 changes required which we can do under a separate PR.
- Fixed some flake issues.
- Added new `auth` app with initial some placeholder modules.
* feature: consolidate .flake8 into pyproject.toml and run black (#127)
* feat: approve adrs (#126)
* merge: TRLST-528 release into develop
* fix: LSGH-64 lxml vulnerability bump package
* fix: bump django version to 2.2.25
* fix: TRLST-535 speed up doc processing.
* LSGH-71 - Upgrade celery
* feat: TRSV2-32 New Models
This work implements a new authentication app that should be used to replace the existing authentication logic in the API. The implementation is switched off with the API_V2_ENABLED setting (defaults to False) and has no effect on the behaviour of existing authentication code. Test coverage of this package is 100%. Tests are all in trade_remedies_api/tests. Run tests with something like:
API_V2_ENABLED=true pytest trade_remedies_api/tests/
* fix: remove redundant clam av settings. (#134)
* fix: upgrade pillow to 9.0.0
* Upgrade django to 2.2.26
* fix: upgrade redis package to be compatible with latest celery upgrade (#139)
* TRLST-555 - Upgrading node-fetch to >= 2.6.7 due to CVE-2022-0235 (#141)
* TRLST-551 - upgrading iPython to >= 7.31.1 (#140)
* Create management command for user contact organisation (#136)
* Feature/trsv2 105 (#142)
* preliminary work on the migration
* Migration to OpenSearch partially completed, will run tests in develop environment to find out any problems which could occur during the production migration.
* Removed some unused code to do with SSL authentication of the OpenSearch instance, no longer necessary as all requests are made over HTTP
* TRSV2-105 Tests and Refactoring
1. Added tests of the opensearch.py file, stored in core/tests/test_open_search.py
2. Refactored and renamed any old references to ElasticSearch
3. Updated the local.env.example to reflect the new name changes
4. Added steps in the CircleCI config.yml file to add an OpenSearch instance so these new tests will run
5. Removed elasticsearch Python library from requirements.in
* TRSV2-105 Refactoring
5. Removed elasticsearch Python library from requirements.in
* Updated opensearch image in CircleCI config.yml as 1.0.0 is causing a high GID error
* Removed overridden settings variables from test_open_search.py as CircleCI already had the correct settings variables and didn't need them to be overridden
* test_open_search.py flake8 code formatting error
* Removed redundant assert statement from test_open_search.py test
* TRLST-550 - Merge release back into develop (#144)
* Release 1.5.6
* Update traitlets
* Black and flake8 checks
* Use pytest in circleci
* Rationalise Procfile triggered processes
* Branding changes
* Bump version
* Fix dependabot alert fixes
* Patch front end vulnerability
* bump version to 1.5.7
* feat: add management command to patch audit logs
Changes to speed up model exports include an Audit's case title
pre-compute. However existing audit logs won't have the pre-computed
values in their json field. This management command patches all
audit logs accordingly.
* feat: test patch_audit managment command
* hotfix: TRLST 242 optimise logging in API project
* hotfix: robust logging trlst-252
hotfix to make logging more robust (trlst-252)
- Django signal receivers used to log db events cannot depend on
…- Loading branch information
16 people
authored
Dec 20, 2023
1 parent
f94feaf
commit 71818f1