diff --git a/.markdownlint.jsonc b/.markdownlint.jsonc new file mode 100644 index 0000000000..dc8f15e712 --- /dev/null +++ b/.markdownlint.jsonc @@ -0,0 +1,10 @@ +{ + "default": true, + "MD010": { + "code_blocks": false + }, + "MD013": { + "code_blocks": false, + "tables": false + } +} diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index c482932253..a945a716bd 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -36,3 +36,22 @@ repos: additional_dependencies: - flake8-bugbear==23.9.16 - flake8-comprehensions==3.14.0 +- repo: https://github.com/igorshubovych/markdownlint-cli + rev: v0.37.0 + hooks: + - id: markdownlint + exclude: | + (?x)^( + ISSUE_TEMPLATE\.md| + src/dashboard/(frontend/app|src/media)/vendor/.* + ) +- repo: https://github.com/thlorenz/doctoc + rev: v2.2.0 + hooks: + - id: doctoc + files: | + (?x)^( + (CONTRIBUTING|SECURITY).*\.md| + hack/README\.md| + src/(MCPServer|MCPClient|dashboard)/install/README\.md + ) diff --git a/CHANGELOG.md b/CHANGELOG.md index a371c0b8cc..7d6eb7403d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,93 +1,94 @@ -1.6.0 onwards -============= +# Changelog -This CHANGELOG is not longer maintained. Please visit the Archivematica wiki for comprehensive [release notes](https://wiki.archivematica.org/Release_Notes). +## 1.6.0 onwards -1.5.0 -===== +This CHANGELOG is not longer maintained. Please visit the Archivematica wiki for +comprehensive [release notes](https://wiki.archivematica.org/Release_Notes). -* Dashboard: always show times in job detail view (https://github.com/artefactual/archivematica/pull/459) +## 1.5.0 + +* Dashboard: always show times in job detail view () * Update to PRONOM v84 () * Update to Fido 1.3.4 () -* Update to Siegfried 1.5.0 (https://github.com/artefactual/archivematica/pull/458) -* Archival storage: fix AIP pagination (https://github.com/artefactual/archivematica/pull/455) -* Migration: make File.currentLocation nullable (https://github.com/artefactual/archivematica/pull/454) -* Dashboard: strip SS API key field, refs #9838 (https://github.com/artefactual/archivematica/pull/453) -* Reingest: Do not use formatversion.description to try and match a premis:formatRegistryKey entry (https://github.com/artefactual/archivematica/pull/452) -* Migration: Fix thumbnail normalization (https://github.com/artefactual/archivematica/pull/451) -* AIC fixes (https://github.com/artefactual/archivematica/pull/450) -* Dashboard: add gunicorn support (https://github.com/artefactual/archivematica/pull/449) -* FITS: Use models not databaseInterface (https://github.com/artefactual/archivematica/pull/448) -* Add inherit_notes opt to ArchivesSpace client, refs #9872 (https://github.com/artefactual/archivematica/pull/447) -* Storage Service: Add auth to redirect URLs (https://github.com/artefactual/archivematica/pull/446) -* Use Format description in premis:formatName (https://github.com/artefactual/archivematica/pull/445) -* METS: handle missing metadata entry in structMap (https://github.com/artefactual/archivematica/pull/444) -* ArchivesSpace: optional inheritance of notes (https://github.com/artefactual/archivematica/pull/443) -* ArchivesSpace: Update ArchivesSpace help text (https://github.com/artefactual/archivematica/pull/442) -* Migrations: Update file ID text (https://github.com/artefactual/archivematica/pull/441) -* METS: clean up empty metadata dir, refs #8427 (https://github.com/artefactual/archivematica/pull/440) -* dbsettings: use database instead of db (https://github.com/artefactual/archivematica/pull/439) -* Storage Service: Add API key support (https://github.com/artefactual/archivematica/pull/438) -* Rights: fix validation issue, refs #9703 (https://github.com/artefactual/archivematica/pull/435) -* Installer: Update storage service setup text (https://github.com/artefactual/archivematica/pull/434) -* ArchivesSpace: Location of originals is DIP UUID (https://github.com/artefactual/archivematica/pull/430) -* AgentArchives: Handle notes with no content (https://github.com/artefactual-labs/agentarchives/pull/30) -* Normalize: fix preservation file UUID to match path on disk (https://github.com/artefactual/archivematica/pull/427) -* Dashboard: Fix assumed block size (https://github.com/artefactual/archivematica/pull/420) -* ArchivesSpace/ATK DIP Upload Matching GUI improvements (https://github.com/artefactual/archivematica/pull/419) -* Store DIP after upload. refs #8995 (https://github.com/artefactual/archivematica/pull/417) -* Fix display of transfer rights (https://github.com/artefactual/archivematica/pull/415) -* ArchivesSpace DIP Upload: Do not publish location of originals note (https://github.com/artefactual/archivematica/pull/412) -* Use Django migrations (https://github.com/artefactual/archivematica/pull/409) -* METS: Handle empty rows in metadata.csv (https://github.com/artefactual/archivematica/pull/404) -* Don’t autodetect METS mappings (https://github.com/artefactual/archivematica/pull/403) -* Auth: Check auth is True, not just truthy (https://github.com/artefactual/archivematica/pull/385) -* unitTransfer: tempPath is already a unicode string (https://github.com/artefactual/archivematica/pull/381) -* Prepare for move to Django migrations (https://github.com/artefactual/archivematica/pull/376) -* ArchivesSpace DIP Upload: Fix copying singlepart notes when creating DOs (https://github.com/artefactual/archivematica/pull/373) -* base64: escape invalid UTF-8 characters before encoding (https://github.com/artefactual/archivematica/pull/367) -* Make name of database configurable (https://github.com/artefactual/archivematica/pull/331) -* Archival Storage: Fix AIP data pagination (https://github.com/artefactual/archivematica/pull/322) -* ArchivesSpace DIP Upload enhancements (https://github.com/artefactual/archivematica/pull/308) -* Add MySQL Connection Pooling (improve database performance) (https://github.com/artefactual/archivematica/pull/305) -* Logging improvements (https://github.com/artefactual/archivematica/pull/303) -* change uuid used in determing file format (https://github.com/artefactual/archivematica/pull/302) -* Logging: silence verbose libraries in client script logs (https://github.com/artefactual/archivematica/pull/298) -* Django ORM fixes (https://github.com/artefactual/archivematica/pull/296) -* ArchivesSpace: find_by_field uses new endpoint (https://github.com/artefactual/archivematica/pull/295) -* Update fpr-admin commit (disable FPRules) (https://github.com/artefactual/archivematica/pull/294) -* filesystem_ajax: return proper exit codes on error (https://github.com/artefactual/archivematica/pull/292) -* Backlog search: allow custom error handling (https://github.com/artefactual/archivematica/pull/290) -* Decouple renderBacklogSearchForm from originals_browser (https://github.com/artefactual/archivematica/pull/289) -* Upload-ArchivesSpace: Only send ArchivesSpace formats (https://github.com/artefactual/archivematica/pull/286) -* ArchivesSpace.find_by_field fixes (https://github.com/artefactual/archivematica/pull/285) -* copy_to_arrange: "objects" shouldn't be dragged onto arrange root (https://github.com/artefactual/archivematica/pull/282) -* Rename endpoint, s/ransfer/transfer (https://github.com/artefactual/archivematica/pull/275) -* Make `filepath` parameter to /filesystem/delete/ APIs mandatory (https://github.com/artefactual/archivematica/pull/270) -* arrange_contents: fix calling without the `path` parameter (https://github.com/artefactual/archivematica/pull/268) -* Initialize Django in MCP client scripts ref. #8879 (https://github.com/artefactual/archivematica/pull/267) -* /ingest/backlog/: don’t redirect to self (https://github.com/artefactual/archivematica/pull/266) -* Fix FPRClient tests (https://github.com/artefactual/archivematica/pull/263) -* Removed vendored requests library (https://github.com/artefactual/archivematica/pull/262) -* Add identifier search to ArchivesSpace/ATK upload (https://github.com/artefactual/archivematica/pull/261) -* ArchivesSpaceClient: dispatch on resource type properly (https://github.com/artefactual/archivematica/pull/260) -* filesystem_ajax: don't try to JSON encode an exception (https://github.com/artefactual/archivematica/pull/259) -* ArchivesSpace: Automate matching GUI (https://github.com/artefactual/archivematica/pull/258) -* AIP re-ingest (https://github.com/artefactual/archivematica/pull/257) -* Tests: Use an in-memory database (https://github.com/artefactual/archivematica/pull/256) -* Specify python2 in shebangs. Python version choice respects path. https://github.com/artefactual/archivematica/pull/255 -* Jobs: Always show arguments (https://github.com/artefactual/archivematica/pull/254) -* Dashboard logs: use GroupWriteRotatingFileHandler (https://github.com/artefactual/archivematica/pull/253) -* Fix archival storage display of deletion requests (https://github.com/artefactual/archivematica/pull/251) -* Dependencies: Improve SSL dependencies (https://github.com/artefactual/archivematica/pull/250) -* templatetags: fix active tag when unhandled exception raised (https://github.com/artefactual/archivematica/pull/248) -* MCPServer: Log all exceptions, use ReplacementDict.replace (https://github.com/artefactual/archivematica/pull/247) -* ArchivesSpace integration (https://github.com/artefactual/archivematica/pull/185) -* Hierarchical DIP support (https://github.com/artefactual/archivematica/pull/130) -* Update to Django 1.7 (https://github.com/artefactual/archivematica/pull/124) +* Update to Siegfried 1.5.0 () +* Archival storage: fix AIP pagination () +* Migration: make File.currentLocation nullable () +* Dashboard: strip SS API key field, refs #9838 () +* Reingest: Do not use formatversion.description to try and match a + premis:formatRegistryKey entry () +* Migration: Fix thumbnail normalization () +* AIC fixes () +* Dashboard: add gunicorn support () +* FITS: Use models not databaseInterface () +* Add inherit_notes opt to ArchivesSpace client, refs #9872 () +* Storage Service: Add auth to redirect URLs () +* Use Format description in premis:formatName () +* METS: handle missing metadata entry in structMap () +* ArchivesSpace: optional inheritance of notes () +* ArchivesSpace: Update ArchivesSpace help text () +* Migrations: Update file ID text () +* METS: clean up empty metadata dir, refs #8427 () +* dbsettings: use database instead of db () +* Storage Service: Add API key support () +* Rights: fix validation issue, refs #9703 () +* Installer: Update storage service setup text () +* ArchivesSpace: Location of originals is DIP UUID () +* AgentArchives: Handle notes with no content () +* Normalize: fix preservation file UUID to match path on disk () +* Dashboard: Fix assumed block size () +* ArchivesSpace/ATK DIP Upload Matching GUI improvements () +* Store DIP after upload. refs #8995 () +* Fix display of transfer rights () +* ArchivesSpace DIP Upload: Do not publish location of originals note () +* Use Django migrations () +* METS: Handle empty rows in metadata.csv () +* Don’t autodetect METS mappings () +* Auth: Check auth is True, not just truthy () +* unitTransfer: tempPath is already a unicode string () +* Prepare for move to Django migrations () +* ArchivesSpace DIP Upload: Fix copying singlepart notes when creating DOs () +* base64: escape invalid UTF-8 characters before encoding () +* Make name of database configurable () +* Archival Storage: Fix AIP data pagination () +* ArchivesSpace DIP Upload enhancements () +* Add MySQL Connection Pooling (improve database performance) () +* Logging improvements () +* change uuid used in determing file format () +* Logging: silence verbose libraries in client script logs () +* Django ORM fixes () +* ArchivesSpace: find_by_field uses new endpoint () +* Update fpr-admin commit (disable FPRules) () +* filesystem_ajax: return proper exit codes on error () +* Backlog search: allow custom error handling () +* Decouple renderBacklogSearchForm from originals_browser () +* Upload-ArchivesSpace: Only send ArchivesSpace formats () +* ArchivesSpace.find_by_field fixes () +* copy_to_arrange: "objects" shouldn't be dragged onto arrange root () +* Rename endpoint, s/ransfer/transfer () +* Make `filepath` parameter to /filesystem/delete/ APIs mandatory () +* arrange_contents: fix calling without the `path` parameter () +* Initialize Django in MCP client scripts ref. #8879 () +* /ingest/backlog/: don’t redirect to self () +* Fix FPRClient tests () +* Removed vendored requests library () +* Add identifier search to ArchivesSpace/ATK upload () +* ArchivesSpaceClient: dispatch on resource type properly () +* filesystem_ajax: don't try to JSON encode an exception () +* ArchivesSpace: Automate matching GUI () +* AIP re-ingest () +* Tests: Use an in-memory database () +* Specify python2 in shebangs. Python version choice respects path. +* Jobs: Always show arguments () +* Dashboard logs: use GroupWriteRotatingFileHandler () +* Fix archival storage display of deletion requests () +* Dependencies: Improve SSL dependencies () +* templatetags: fix active tag when unhandled exception raised () +* MCPServer: Log all exceptions, use ReplacementDict.replace () +* ArchivesSpace integration () +* Hierarchical DIP support () +* Update to Django 1.7 () -1.4.1 -===== +## 1.4.1 * Fix "active" templatetag when Django handles an uncaught exception * Fix updating AIP records after marking files as deleted in archival storage (#8533) @@ -95,38 +96,44 @@ This CHANGELOG is not longer maintained. Please visit the Archivematica wiki for * Fix marking AIPs as stored after deletion is rejected (#8533) * MCPServer: log all exceptions (#8509) * MCPServer: remove duplicate ReplacementDict code (#8509) -* Dashboard: use GroupWriteRotatingFileHandler to ensure group-writeability of rotated logs (#8587) +* Dashboard: use GroupWriteRotatingFileHandler to ensure group-writeability of + rotated logs (#8587) -1.4.0 -===== +## 1.4.0 * Remove unused Elasticsearch backup code (#8076) * Improve performance of indexing AIP by saving uncompressed METS (#7424) * Index MODS identifiers in Elasticsearch (#7424) * Track file events in transfer METS using amdSecs (#7714) * Add a new "processed structMap" which captures the transfer's final state (#7714) -* Use the Storage Service, not Elasticsearch, to look up file metadata in SIP arrange (#7714) -* Improve Dublin Core namespacing for metadata generated from metadata.csv and from template (#8007) +* Use the Storage Service, not Elasticsearch, to look up file metadata in SIP + arrange (#7714) +* Improve Dublin Core namespacing for metadata generated from metadata.csv and + from template (#8007) * Display additional metadata (number of files, size) from the storage service (#7853) * Accession IDs associate with Transfers again (#7442) * Remove Elasticsearch tool that is now maintained in archivematica-devtools * Log exceptions in start_transfer (#8117) -* Archivist's Toolkit upload: use SQL placeholders instead of string interpolation (#7771) +* Archivist's Toolkit upload: use SQL placeholders instead of string + interpolation (#7771) * Fail SIP if checksum verification fails (#8825) -* Move logs to /var/archivematica/sharedDirectory; separate dashboard logs into its own file (#6647) +* Move logs to /var/archivematica/sharedDirectory; separate dashboard logs into + its own file (#6647) * Handle linking_agent as an integer, not a foreign key, in Django models (#8230) * Index MODS identifiers in the aips/aipfile index (#8266) * Fix bag transfer names (#8229) * DSpace transfer type accepts either files or folders * Include the contents of bag-info.txt in AIP METS as (#8177) * Enable autodetection of date fields when indexing METS contents (#8181) -* Add support for searching on the contents of transfer_metadata in archival storage (#8181) +* Add support for searching on the contents of transfer_metadata in archival + storage (#8181) * Add support for searching date ranges in archival storage (#8181) * Improve text in stderr from Transcribe microservice (#7051) * Add preliminary support for Siegfried as an identification tool (#8287) * Ignore non-Unicode data contained in metadata.csv files (#7277) * Fix blank Keyword ("term") queries in archival storage and transfer backlog (#8292) -* Fix default value of "query type" dropdown in archival storage and transfer backlog (#8292) +* Fix default value of "query type" dropdown in archival storage and transfer + backlog (#8292) * Allow DIP choices to be preconfigured (#7321) * Search: restrict nested fields searched when using string searches (#8338) * Elasticsearch: fix exception handling (#4757) diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index 5e7b4aff16..05359bccfc 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -55,7 +55,7 @@ further defined and clarified by project maintainers. ## Enforcement Instances of abusive, harassing, or otherwise unacceptable behavior may be -reported by contacting the project team at info@artefactual.com. The project team +reported by contacting the project team at . The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. @@ -67,8 +67,8 @@ members of the project's leadership. ## Attribution -This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, -available at [http://contributor-covenant.org/version/1/4][version] +This Code of Conduct is adapted from the [Contributor Covenant][homepage], +version 1.4, available at [http://contributor-covenant.org/version/1/4][version] [homepage]: http://contributor-covenant.org [version]: http://contributor-covenant.org/version/1/4/ diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0968fdcbca..c0cbafe550 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -7,17 +7,19 @@ us assess your changes faster and makes it easier for us to merge your submission! There are many ways to contribute: writing tutorials or blog posts about your -experience, improving the [documentation], submitting bug reports, answering -questions on the [mailing list], or writing code which can be incorporated into +experience, improving the [documentation], submitting bug reports, answering +questions on the [mailing list], or writing code which can be incorporated into Archivematica itself. +## Table of contents + -**Table of Contents** - [Submitting bugs](#submitting-bugs) -- [Submitting enhancement ideas](#submitting-enhancements) +- [Submitting enhancement ideas](#submitting-enhancement-ideas) - [Submitting code changes](#submitting-code-changes) + - [Permalinks](#permalinks) - [Getting started](#getting-started) - [When to submit code for review?](#when-to-submit-code-for-review) - [Opening the pull request](#opening-the-pull-request) @@ -37,6 +39,7 @@ Archivematica itself. - [Commit History](#commit-history) - [Commits should be specific and atomic](#commits-should-be-specific-and-atomic) - [Every commit should work](#every-commit-should-work) + - [Commit messages](#commit-messages) - [Commit summaries should be concise](#commit-summaries-should-be-concise) - [Commit messages should be as detailed as they need to be (and no more)](#commit-messages-should-be-as-detailed-as-they-need-to-be-and-no-more) @@ -45,37 +48,37 @@ Archivematica itself. ## Submitting bugs If you find a security vulnerability, do NOT open an issue. Email -info@artefactual.com instead. + instead. -Issues can be filed using GitHub Issues in the [Archivematica Issues repo]. -It is recommended to file issues there rather than in any of the -Archivematica-related code repositories. Artefactual staff also use GitHub +Issues can be filed using GitHub Issues in the [Archivematica Issues repo]. +It is recommended to file issues there rather than in any of the +Archivematica-related code repositories. Artefactual staff also use GitHub issues for any work they do on the Archivematica project. -You can also post in our [user] mailing list. A post to the mailing list is +You can also post in our [user] mailing list. A post to the mailing list is always welcome, especially if you're unsure if it's a bug or a local problem! Useful questions to answer if you're having problems include: -* What version of Archivematica and the Storage Service are you using? -* How was Archivematica installed? Package install, Ansible, etc? -* Was this a fresh install or an upgrade? -* What did you do to cause this bug to happen? -* What did you expect to happen? -* What did you see instead? -* Can you reproduce this reliably? -* If a specific Job is failing, what output did it produce? This is available +- What version of Archivematica and the Storage Service are you using? +- How was Archivematica installed? Package install, Ansible, etc? +- Was this a fresh install or an upgrade? +- What did you do to cause this bug to happen? +- What did you expect to happen? +- What did you see instead? +- Can you reproduce this reliably? +- If a specific Job is failing, what output did it produce? This is available by clicking on the gear icon. ## Submitting enhancement ideas Similar to submitting bugs, you are welcome to submit ideas for enhancements or -new features in the [Archivematica Issues repo]. This is also where Artefactual +new features in the [Archivematica Issues repo]. This is also where Artefactual staff record upcoming enhancements when they have been sponsored for inclusion either by Artefactual Systems or by a client. -Please feel free also to use the [Issues repo wiki] as a space for gathering and -collaborating on ideas. If you are not already a member of the +Please feel free also to use the [Issues repo wiki] as a space for gathering and +collaborating on ideas. If you are not already a member of the Archivematica repo (required for editing the wiki), file an issue there with the title "Request membership." @@ -114,7 +117,7 @@ So you have something to contribute to an Artefactual project. Great! To install Archivematica, see our [development installation] instructions. -Artefactual uses [GitHub]'s pull request feature for code review. Every change +Artefactual uses [GitHub]'s pull request feature for code review. Every change being submitted to an Artefactual project should be submitted as a pull request to the appropriate repository. A branch being submitted for code review should contain commits covering a related section of @@ -181,72 +184,74 @@ The Archivematica contributor's agreement is based almost verbatim on the [Apache Foundation]'s individual [contributor license]. If you have any questions or concerns about the Contributor's Agreement, -please email us at agreement@artefactual.com to discuss them. +please email us at to discuss them. ### Why do I have to sign a Contributor's Agreement? One of the key challenges for open source software is to support a collaborative -development environment while protecting the rights of contributors and users +development environment while protecting the rights of contributors and users over the long-term. Unifying Archivematica copyrights through contributor agreements is the best way -to protect the availability and sustainability of Archivematica over the +to protect the availability and sustainability of Archivematica over the long-term as free and open-source software. -In all cases, contributors who sign the Contributor's Agreement retain full -rights to use their original contributions for any other purpose outside of -Archivematica, while enabling Artefactual Systems, any successor organization -which may eventually take over responsibility for Archivematica, and the wider -Archivematica community to benefit from their collaboration and contributions +In all cases, contributors who sign the Contributor's Agreement retain full +rights to use their original contributions for any other purpose outside of +Archivematica, while enabling Artefactual Systems, any successor organization +which may eventually take over responsibility for Archivematica, and the wider +Archivematica community to benefit from their collaboration and contributions in this open source project. -[Artefactual Systems] has made the decision and has a proven track record of +[Artefactual Systems] has made the decision and has a proven track record of making our intellectual property available to the community at large. -By standardizing contributions on these agreements the Archivematica +By standardizing contributions on these agreements the Archivematica intellectual property position does not become too complicated. -This ensures our resources are devoted to making our project the best they can +This ensures our resources are devoted to making our project the best they can be, rather than fighting legal battles over contributions. ### How do I send in an agreement? -Please read and sign the [Contributor's Agreement] and email it to -agreement@artefactual.com. +Please read and sign the [Contributor's Agreement] and email it to +. Alternatively, you may send a printed, signed agreement to: - Artefactual Systems Inc. - 201 - 301 Sixth Street - New Westminster BC V3L 3A7 - Canada +```text +Artefactual Systems Inc. +201 - 301 Sixth Street +New Westminster BC V3L 3A7 +Canada +``` ## Contribution standards ### Style -Archivematica uses the Python [PEP8] community style guidelines. Newly-written -code should conform to PEP-8 style. PEP8 is a daunting document, but there are +Archivematica uses the Python [PEP8] community style guidelines. Newly-written +code should conform to PEP-8 style. PEP8 is a daunting document, but there are very good linters available that you can run to check style in your code. -* The [Black] tool formats the code automatically. The output is deterministic +- The [Black] tool formats the code automatically. The output is deterministic for any given input. Editor integration is possible. -* The [flake8] tool checks for style problems as well as errors and complexity. - It can be used at the command line or as a plugin in your preferred text - editor/IDE. The Archivematica [continuous integration system] will currently +- The [flake8] tool checks for style problems as well as errors and complexity. + It can be used at the command line or as a plugin in your preferred text + editor/IDE. The Archivematica [continuous integration system] will currently check code for compliance against flake8. We have integrated these tools with our CI, i.e. pull requests will fail to build when the tools above report errors. -Additionally [Pylint] is used by developers internally at Artefactual to +Additionally [Pylint] is used by developers internally at Artefactual to highlight other potential areas of improvement during code-review. #### Some extra notes A few additional stylistic preferences might not get flagged by linters: -* Don't use variable or parameter names that shadow builtin functions and +- Don't use variable or parameter names that shadow builtin functions and types. For example, don't name a variable "file". (Unfortunately, Python uses many useful names for its builtin types and functions.) -* Sort imports alphabetically within their grouping to reduce duplicate +- Sort imports alphabetically within their grouping to reduce duplicate imports. #### Exceptions @@ -267,10 +272,10 @@ however, if possible, it's great if your new code does conform. When working in sections of code that don't conform to PEP8, it's okay to relax a few PEP8 rules in order to match existing code. In particular: -* When modifying an existing function which uses camelCase variables or +- When modifying an existing function which uses camelCase variables or parameters, it's okay to make your new variables/parameters camelCase to match. -* When adding new functions to a module or class that uses camelCase naming, +- When adding new functions to a module or class that uses camelCase naming, it's okay to make your new function and its parameters camelCase to match. Try to use snake_case internally, however. @@ -282,18 +287,18 @@ behaviour should be maintained. ### Documentation New classes and functions should generally be documented using -[docstrings]; these help in providing clarity, and can also be used to generate -API documentation later. Generally any function which isn't obvious -(any function longer than a line or two) should have a docstring. +[docstrings]; these help in providing clarity, and can also be used to generate +API documentation later. Generally any function which isn't obvious +(any function longer than a line or two) should have a docstring. When in doubt: document! Python's [PEP 257] document provides a useful -guideline for docstring style. Generally, prefer using +guideline for docstring style. Generally, prefer using [Sphinx-compatible docstrings]. More [examples] and [attributes to use] can be found on the Sphinx website. ### Tests -New code should also have unit tests. Tests are written in [unittest] style -and run with [py.test]. For tests requiring the Django ORM, we use the +New code should also have unit tests. Tests are written in [unittest] style +and run with [py.test]. For tests requiring the Django ORM, we use the Django-provided[TestCase], which extends `unittest.TestCase`. Tests are found in the `tests` directory, a sibling of the directory containing @@ -301,7 +306,7 @@ the code. `test_foo.py` contains tests for `foo.py`. For clarity, group tests for the same function and similar tests into the same class within that file. This will also allow you to share setup and teardown code. -If you are testing code that makes HTTP requests, using [VCR.py] is highly +If you are testing code that makes HTTP requests, using [VCR.py] is highly recommended. It should already be a development dependency. ### Commit History @@ -359,13 +364,19 @@ and b) what the change is. For example: Clear commit summary: -> Replace 404 messages with a user-friendly one +```text +Replace 404 messages with a user-friendly one +``` Unclear commit summaries: -> Fixed some normalization bugs +```text +Fixed some normalization bugs +``` -> Bugfixes +```text +Bugfixes +``` The unclear messages make it hard to tell at a glance what changed, and that makes browsing the commit history harder. @@ -373,7 +384,9 @@ makes browsing the commit history harder. A commit message should use the [imperative mood] which should always be able to complete the following sentence: - If applied, this commit will +```text +If applied, this commit will +``` #### Commit messages should be as detailed as they need to be (and no more) @@ -392,7 +405,7 @@ Each line of a commit message should be no more than 72 characters in width. The following is an outline of a commit message combined with an ideal commit summary: -``` +```text Capitalized, short (50 chars or less) summary More detailed explanatory text, if necessary. Wrap it to about 80 @@ -404,12 +417,12 @@ two together. Further content comes after a blank line. ``` + [documentation]: https://github.com/artefactual/archivematica-docs/ [mailing list]: https://groups.google.com/forum/#!forum/archivematica [Archivematica Issues repo]: https://github.com/archivematica/Issues [user]: https://groups.google.com/forum/#!forum/archivematica -[Archivematica Issues repo]: https://github.com/archivematica/Issues -[Issues repo wiki]: https://github.com/archivematica/Issues/wiki +[Issues repo wiki]: https://github.com/archivematica/Issues/wiki [files]: https://help.github.com/articles/getting-permanent-links-to-files/ [code snippets]: https://help.github.com/articles/creating-a-permanent-link-to-a-code-snippet/ [development installation]: https://wiki.archivematica.org/Getting_started#Installation diff --git a/README.md b/README.md index f79016d37c..3c266ff999 100644 --- a/README.md +++ b/README.md @@ -1,54 +1,85 @@ -[![GitHub CI](https://github.com/artefactual/archivematica/actions/workflows/test.yml/badge.svg)](https://github.com/artefactual/archivematica/actions/workflows/test.yml) -[![codecov](https://codecov.io/gh/artefactual/archivematica/branch/qa/1.x/graph/badge.svg?token=tKlfjhmrlC)](https://codecov.io/gh/artefactual/archivematica) +# [Archivematica] -# [Archivematica](https://www.archivematica.org/) +By [Artefactual] -By [Artefactual](https://www.artefactual.com/) +[![GitHub CI]][Test workflow] +[![codecov]][Archivematica Codecov] -Archivematica is a web- and standards-based, open-source application which allows your institution to preserve long-term access to trustworthy, authentic and reliable digital content. -Our target users are archivists, librarians, and anyone working to preserve digital objects. - -You are free to copy, modify, and distribute Archivematica with attribution under the terms of the AGPLv3 license. -See the [LICENSE](LICENSE) file for details. +Archivematica is a web- and standards-based, open-source application +which allows your institution to preserve long-term access to +trustworthy, authentic and reliable digital content. Our target users +are archivists, librarians, and anyone working to preserve digital +objects. +You are free to copy, modify, and distribute Archivematica with +attribution under the terms of the AGPLv3 license. See the [LICENSE] +file for details. ## Installation -* [Production installation](https://www.archivematica.org/docs/latest/admin-manual/installation-setup/installation/installation/) -* [Development installation](https://github.com/artefactual/archivematica/tree/qa/1.x/hack) - +* [Production installation] +* [Development installation] ## Other resources -* [Website](https://www.archivematica.org/): User and administrator documentation -* [Wiki](https://www.archivematica.org/wiki/Development): Developer facing documentation, requirements analysis and community resources -* [Issues](https://github.com/archivematica/Issues): Git repository used for tracking Archivematica issues and feature/enhancement ideas -* [User Google Group](https://groups.google.com/forum/#!forum/archivematica): Forum/mailing list for user questions (both technical and end-user) -* [Paid support](https://www.artefactual.com/services/): Paid support, hosting, training, consulting and software development contracts from Artefactual - +* [Website][Archivematica]: User and administrator documentation +* [Wiki]: Developer facing documentation, requirements analysis and + community resources +* [Issues]: Git repository used for tracking Archivematica issues and + feature/enhancement ideas +* [User Google Group]: Forum/mailing list for user questions (both + technical and end-user) +* [Paid support]: Paid support, hosting, training, consulting and + software development contracts from Artefactual ## Contributing Thank you for your interest in Archivematica! -For more details, see the [contributing guidelines](CONTRIBUTING.md) - +For more details, see the [contributing guidelines] ## Reporting an issue -Issues related to Archivematica, the Storage Service, or any related repository can be filed in the [Archivematica Issues repository](https://github.com/archivematica/Issues/issues). - +Issues related to Archivematica, the Storage Service, or any related +repository can be filed in the [Archivematica Issues repository]. ### Security -If you have a security concern about Archivematica or any related repository, please see the [SECURITY file](SECURITY.md) for information about how to safely report vulnerabilities. - +If you have a security concern about Archivematica or any related +repository, please see the [SECURITY file] for information about how +to safely report vulnerabilities. ## Related projects Archivematica consists of several projects working together, including: -* [Archivematica](https://github.com/artefactual/archivematica): This repository! Main repository containing the user-facing dashboard, task manager MCPServer and clients scripts for the MCPClient -* [Storage Service](https://github.com/artefactual/archivematica-storage-service): Responsible for moving files to Archivematica for processing, and from Archivematica into long-term storage -* [Format Policy Registry](https://github.com/artefactual/archivematica/tree/qa/1.x/src/dashboard/src/fpr): Submodule shared between Archivematica and the Format Policy Registry (FPR) server that displays and updates FPR rules and commands - -For more projects in the Archivematica ecosystem, see the [getting started](https://wiki.archivematica.org/Getting_started#Projects) page. +* [Archivematica][Archivematica GitHub]: This repository! Main + repository containing the user-facing dashboard, task manager + MCPServer and clients scripts for the MCPClient +* [Storage Service]: Responsible for moving files to Archivematica for + processing, and from Archivematica into long-term storage +* [Format Policy Registry]: Submodule shared between Archivematica and + the Format Policy Registry (FPR) server that displays and updates + FPR rules and commands + +For more projects in the Archivematica ecosystem, see the [getting started] page. + +[Archivematica]: https://www.archivematica.org/ +[Artefactual]: https://www.artefactual.com/ +[GitHub CI]: https://github.com/artefactual/archivematica/actions/workflows/test.yml/badge.svg +[Test workflow]: https://github.com/artefactual/archivematica/actions/workflows/test.yml +[codecov]: https://codecov.io/gh/artefactual/archivematica/branch/qa/1.x/graph/badge.svg?token=tKlfjhmrlC +[Archivematica Codecov]: https://codecov.io/gh/artefactual/archivematica +[LICENSE]: LICENSE +[Production installation]: https://www.archivematica.org/docs/latest/admin-manual/installation-setup/installation/installation/ +[Development installation]: https://github.com/artefactual/archivematica/tree/qa/1.x/hack +[Wiki]: https://www.archivematica.org/wiki/Development +[Issues]: https://github.com/archivematica/Issues +[User Google Group]: https://groups.google.com/forum/#!forum/archivematica +[Paid support]: https://www.artefactual.com/services/ +[contributing guidelines]: CONTRIBUTING.md +[Archivematica Issues repository]: https://github.com/archivematica/Issues/issues +[SECURITY file]: SECURITY.md +[Archivematica GitHub]: https://github.com/artefactual/archivematica +[Storage Service]: https://github.com/artefactual/archivematica-storage-service +[Format Policy Registry]: https://github.com/artefactual/archivematica/tree/qa/1.x/src/dashboard/src/fpr +[getting started]: https://wiki.archivematica.org/Getting_started#Projects diff --git a/SECURITY.md b/SECURITY.md index e00664a360..546250a178 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -1,15 +1,20 @@ # Security Policy This document outlines security procedures and general policies for the -Archivematica project. See https://www.archivematica.org for more information +Archivematica project. See for more information about Archivematica. -**Contents** +## Table of contents -* [Reporting a security vulnerability](#reporting-a-security-vulnerability) -* [Disclosure policy](#disclosure-policy) -* [Supported versions](#supported-versions) -* [Reporting general bugs](#reporting-general-bugs) + + + +- [Reporting a security vulnerability](#reporting-a-security-vulnerability) +- [Disclosure policy](#disclosure-policy) +- [Supported versions](#supported-versions) +- [Reporting general bugs](#reporting-general-bugs) + + ## Reporting a security vulnerability @@ -22,18 +27,18 @@ post about the issue on the user forum.** It is critical to the safety of other users that security issues are reported in a secure manner. Instead, please email a report to: -* [security@artefactual.com](mailto:security@artefactual.com) +- [security@artefactual.com](mailto:security@artefactual.com) We will be better able to evaluate and respond to your report if it includes all the details needed for us to reproduce the issue locally. Please include the following information in your email: -* The version of Archivematica you are using. -* Basic information about your installation environment, including operating +- The version of Archivematica you are using. +- Basic information about your installation environment, including operating system and dependency versions. -* Steps to reproduce the issue. -* The resulting error or vulnerability. -* If there are any error logs related to the issue, please include the +- Steps to reproduce the issue. +- The resulting error or vulnerability. +- If there are any error logs related to the issue, please include the relevant parts as well. Your report will be acknowledged within 2 business days, and we’ll follow up @@ -43,9 +48,9 @@ within 1 week. If you haven’t received a reply to your submission after 5 business days of the original report, there are a couple steps you can take: -* Email the Archivematica Program Manager directly at +- Email the Archivematica Program Manager directly at [sromkey@artefactual.com](mailto:sromkey@artefactual.com) -* Email Artefactual's info address: [info@artefactual.com](info@artefactual.com) +- Email Artefactual's info address: [info@artefactual.com](info@artefactual.com) Any information you share with the Archivematica development team as a part of this process will be kept confidential within the team. If we determine that the @@ -63,9 +68,9 @@ When the Archivematica development team receives a security bug report, we will assign it to a primary handler. This person will coordinate the fix and release process, involving the following steps: -* Confirm the problem and determine the affected versions. -* Audit code to find any similar potential problems. -* Prepare fixes for all releases still under maintenance. These fixes will be +- Confirm the problem and determine the affected versions. +- Audit code to find any similar potential problems. +- Prepare fixes for all releases still under maintenance. These fixes will be released as fast as possible. Once new releases and/or security patches have been prepared, tested, and made diff --git a/code_review.md b/code_review.md index 851c65ce43..25dadbd378 100644 --- a/code_review.md +++ b/code_review.md @@ -1,5 +1,4 @@ -Code review -=========== +# Code review Every new feature and bugfix to a project is expected to go through code review before inclusion. This applies both to developers at Artefactual and to outside @@ -10,11 +9,11 @@ The contribution process in general is outlined in the guidelines for what to do when doing code review. It also helps set expectations for contributors about what to expect. -Checklist ---------- +## Checklist ### Meta -- [ ] Have they submitted a signed contributor's agreement? + +- [ ] Have they submitted a signed contributor's agreement? - [ ] Does this meet the requirements? Does it fix the bug? - [ ] Is this feature useful to multiple users or potential users? - [ ] Have the changes been discussed with an analyst? @@ -25,6 +24,7 @@ Checklist - [ ] Is the commit history clean? ### Architecture + - [ ] Does this duplicate functionality found elsewhere? - [ ] Does it make more sense for this to be implemented elsewhere? E.g. Should this happen in the storage service vs. a client script, or should a helper @@ -41,6 +41,7 @@ Checklist existing items? ### Style + - [ ] Does it follow [PEP8](https://www.python.org/dev/peps/pep-0008/) (code style) and [PEP257](https://www.python.org/dev/peps/pep-0257/) (docstrings)? @@ -49,12 +50,14 @@ Checklist - [ ] Are there massive functions, classes or files? Can they be split? - [ ] Are errors handled? Is the error handling consistent with similar code? -### Syntax: +### Syntax + - [ ] Is it Python 2.7 & Python 3 compatible? - [ ] Are all user-facing strings marked for translation? - [ ] Do database changes have migrations? ### Tests + - [ ] Are there tests? - [ ] Do the tests cover all the new functionality or fixes? - [ ] Do the tests handle error cases? @@ -62,11 +65,12 @@ Checklist - [ ] Does this work with non-ASCII? ### Doing + - [ ] Do I understand what this is supposed to do? - [ ] Have I checked out and run this code to verify it does what it says & is supposed to? - [ ] Is someone else an expert in this area? Should I ask them to also review this? -More advice and suggestions can be found at https://www.kevinlondon.com/2015/05/05/code-review-best-practices.html and -https://hypothes.is/blog/code-review-in-remote-teams/ +More advice and suggestions can be found at +and diff --git a/hack/README.md b/hack/README.md index 3c5ff6d13c..054de90a66 100644 --- a/hack/README.md +++ b/hack/README.md @@ -1,28 +1,37 @@ # Archivematica development on Docker Compose +## Table of contents + + + + - [Audience](#audience) - [Requirements](#requirements) - [Elasticsearch container](#elasticsearch-container) - [Installation](#installation) + - [GNU make](#gnu-make) +- [Upgrading to the latest version of Archivematica](#upgrading-to-the-latest-version-of-archivematica) - [Web UIs](#web-uis) -- [Upgrading to the latest version of Archivematica][intro-0] - [Source code auto-reloading](#source-code-auto-reloading) - [Logs](#logs) + - [Clearing the logs](#clearing-the-logs) - [Scaling](#scaling) - [Ports](#ports) - [Tests](#tests) + - [AMAUATs](#amauats) +- [Resetting the environment](#resetting-the-environment) - [Cleaning up](#cleaning-up) +- [Percona tuning](#percona-tuning) - [Instrumentation](#instrumentation) - [Running Prometheus and Grafana](#running-prometheus-and-grafana) - [Percona Monitoring and Management](#percona-monitoring-and-management) - [Troubleshooting](#troubleshooting) - [Nginx returns 502 Bad Gateway](#nginx-returns-502-bad-gateway) - - [Bootstrap seems to run but the Dashboard and Elasticsearch are still down][intro-1] + - [Bootstrap seems to run but the Dashboard and Elasticsearch are still down](#bootstrap-seems-to-run-but-the-dashboard-and-elasticsearch-are-still-down) - [PMM client service doesn't start](#pmm-client-service-doesnt-start) - [My environment is still broken](#my-environment-is-still-broken) -[intro-0]: #upgrading-to-the-latest-version-of-archivematica -[intro-1]: #Bootstrap-seems-to-run-but-the-Dashboard-and-Elasticsearch-are-still-down + ## Audience @@ -46,6 +55,7 @@ memory usage when the environment is initialized in a virtual machine with ```shell docker stats --all --format "table {{.Name}}\t{{.MemUsage}}" ``` + ```console NAME MEM USAGE / LIMIT am-archivematica-mcp-client-1 41.3MiB / 7.763GiB @@ -111,8 +121,8 @@ First, clone this repository this way: git clone https://github.com/artefactual/archivematica.git --branch qa/1.x --recurse-submodules ``` -This will set up the submodules defined in -https://github.com/artefactual/archivematica/tree/qa/1.x/hack/submodules which +This will set up the submodules defined in + which are from the `qa/1.x` branch of Archivematica and the `qa/0.x` branch of Archivematica Storage Service. These two branches are the focus of Archivematica development and pull requests are expected to target them. @@ -218,8 +228,8 @@ Working with submodules can be a little confusing. GitHub's ## Web UIs -- Archivematica Dashboard: http://127.0.0.1:62080/ -- Archivematica Storage Service: http://127.0.0.1:62081/ +- Archivematica Dashboard: +- Archivematica Storage Service: The default credentials for the Archivematica Dashboard and the Storage Service are username: `test`, password: `test`. @@ -234,13 +244,17 @@ code changes. Other components in the stack like the `MCPServer` don't offer this option and they need to be restarted manually, e.g.: - docker compose up -d --force-recreate --no-deps archivematica-mcp-server +```shell +docker compose up -d --force-recreate --no-deps archivematica-mcp-server +``` If you've added new dependencies or changes the `Dockerfile` you should also add the `--build` argument to the previous command in order to ensure that the container is using the newest image, e.g.: - docker compose up -d --force-recreate --build --no-deps archivematica-mcp-server +```shell +docker compose up -d --force-recreate --build --no-deps archivematica-mcp-server +``` ## Logs @@ -284,6 +298,7 @@ are connected to Gearman: ```shell echo workers | socat - tcp:127.0.0.1:62004,shut-none | grep "_v0.0" | awk '{print $2}' - | sort -u ``` + ```console 172.19.0.15 172.19.0.16 @@ -313,7 +328,7 @@ make help | grep test- The following targets use [`tox`](https://tox.readthedocs.io) and [`pytest`](https://docs.pytest.org) to run the tests using MySQL: -``` +```text test-all Run all tests. test-archivematica-common Run Archivematica Common tests. test-dashboard Run Dashboard tests. @@ -465,16 +480,16 @@ two Docker Compose files: docker compose -f docker-compose.yml -f docker-compose.pmm.yml up -d ``` -To access the PMM server interface, visit http://127.0.0.1:62007: +To access the PMM server interface, visit : -* Username: ``admin`` -* Password: ``admin`` +- Username: ``admin`` +- Password: ``admin`` [instrumentation-4]: https://www.percona.com/doc/percona-monitoring-and-management ## Troubleshooting -##### Nginx returns 502 Bad Gateway +### Nginx returns 502 Bad Gateway We're using Nginx as a proxy. Likely the underlying issue is that either the Dashboard or the Storage Service died. Run `docker compose @@ -483,6 +498,7 @@ ps` to confirm the state of their services like this: ```shell docker compose ps --all archivematica-dashboard archivematica-storage-service ``` + ```console NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS am-archivematica-dashboard-1 am-archivematica-dashboard "/usr/local/bin/guni…" archivematica-dashboard 11 minutes ago Up 27 seconds 8000/tcp @@ -495,6 +511,7 @@ service, e.g.: ```shell docker compose logs --no-log-prefix --tail 5 archivematica-storage-service ``` + ```console File "", line 953, in _find_and_load_unlocked ModuleNotFoundError: No module named 'storage_service.wsgi' @@ -509,7 +526,7 @@ and git is not atomically moving things around. But it's fixed now and you want to give it another shot so we run `docker compose up -d` to ensure that all the services are up again. Next run `docker compose ps` to verify that it's all up. -##### Bootstrap seems to run but the Dashboard and Elasticsearch are still down +### Bootstrap seems to run but the Dashboard and Elasticsearch are still down If after running the bootstrap processes and `docker compose ps` still shows that the dashboard and elasticsearch are still down then check the @@ -538,7 +555,7 @@ above. [es-0]: #elasticsearch-container -##### PMM client service doesn't start +### PMM client service doesn't start In some cases the `pmm_client` service fails to start reporting the following error: @@ -554,7 +571,7 @@ docker compose -f docker-compose.yml -f docker-compose.pmm.yml rm pmm_client docker compose -f docker-compose.yml -f docker-compose.pmm.yml up -d ``` -##### My environment is still broken +### My environment is still broken You've read this far but you haven't yet figured out why your development environment is not working? Here are some tips: diff --git a/src/MCPClient/README.md b/src/MCPClient/README.md index b74f32ab4d..f2de2a3a07 100644 --- a/src/MCPClient/README.md +++ b/src/MCPClient/README.md @@ -19,8 +19,8 @@ streams. You set the exit code for a job with: job.set_status(int) To record standard output and standard error, there are several -methods: write_* for storing literal strings; print_* for storing -literal strings with newlines added; pyprint for a Python +methods: `write_*` for storing literal strings; `print_*` for storing +literal strings with newlines added; `pyprint` for a Python `print`-compatible API. See the Job class for more information on these. @@ -42,20 +42,20 @@ directory. Commonly they will follow a pattern like: Some notable features: - * Where the task will insert/update/delete rows from the database, +* Where the task will insert/update/delete rows from the database, we wrap it in a transaction. Besides providing atomicity if something goes wrong, this also boosts performance significantly. - * Generally we iterate over each job one-by-one, although nothing +* Generally we iterate over each job one-by-one, although nothing stops you from working in larger batches if that suits. - * `JobContext` is a convenience class that executes its body in a +* `JobContext` is a convenience class that executes its body in a modified context: - - Any uncaught exceptions will be logged as standard error + * Any uncaught exceptions will be logged as standard error output, and the job's status will be set to 1. - - If you supply the `logger` keyword, your global logger will be + * If you supply the `logger` keyword, your global logger will be configured to send its output to the job's standard error stream. diff --git a/src/MCPClient/install/README.md b/src/MCPClient/install/README.md index 9099bf0b56..df579ad8f5 100644 --- a/src/MCPClient/install/README.md +++ b/src/MCPClient/install/README.md @@ -2,14 +2,17 @@ ## Table of contents -- [MCPClient Configuration](#mcpclient-configuration) - - [Table of contents](#table-of-contents) - - [Introduction](#introduction) - - [Environment variables](#environment-variables) - - [Configuration file](#configuration-file) - - [Parameter list](#parameter-list) - - [Metadata XML validation variables](#metadata-xml-validation-variables) - - [Logging configuration](#logging-configuration) + + + +- [Introduction](#introduction) +- [Environment variables](#environment-variables) +- [Configuration file](#configuration-file) +- [Parameter list](#parameter-list) + - [Metadata XML validation variables](#metadata-xml-validation-variables) +- [Logging configuration](#logging-configuration) + + ## Introduction @@ -72,333 +75,384 @@ non-database parameters could be set in the dbsettings file. This is the full list of variables supported by MCPClient: - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_TIME_ZONE`**: - - **Description:** application time zone. See [TIME_ZONE](https://docs.djangoproject.com/en/1.8/ref/settings/#time-zone) for more details. - - **Config file example:** `MCPClient.time_zone` - - **Type:** `string` - - **Default:** `"UTC"` + - **Description:** application time zone. See [TIME_ZONE] for more details. + - **Config file example:** `MCPClient.time_zone` + - **Type:** `string` + - **Default:** `"UTC"` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_DJANGO_SECRET_KEY`**: - - **Description:** a secret key used for cryptographic signing. See [SECRET_KEY](https://docs.djangoproject.com/en/1.8/ref/settings/#secret-key) for more details. - - **Config file example:** `MCPClient.django_secret_key` - - **Type:** `string` - - :red_circle: **Mandatory!** + - **Description:** a secret key used for cryptographic signing. See [SECRET_KEY] + for more details. + - **Config file example:** `MCPClient.django_secret_key` + - **Type:** `string` + - :red_circle: **Mandatory!** - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_SHAREDDIRECTORYMOUNTED`**: - - **Description:** location of the Archivematica Shared Directory. - - **Config file example:** `MCPClient.sharedDirectoryMounted` - - **Type:** `string` - - **Default:** `/var/archivematica/sharedDirectory/` + - **Description:** location of the Archivematica Shared Directory. + - **Config file example:** `MCPClient.sharedDirectoryMounted` + - **Type:** `string` + - **Default:** `/var/archivematica/sharedDirectory/` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_PROCESSINGDIRECTORY`**: - - **Description:** location of the Archivematica Currently Procesing Directory. - - **Config file example:** `MCPClient.processingDirectory` - - **Type:** `string` - - **Default:** `/var/archivematica/sharedDirectory/currentlyProcessing/` + - **Description:** location of the Archivematica Currently Procesing Directory. + - **Config file example:** `MCPClient.processingDirectory` + - **Type:** `string` + - **Default:** `/var/archivematica/sharedDirectory/currentlyProcessing/` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_REJECTEDDIRECTORY`**: - - **Description:** location of the Archivematica Rejected Directory. - - **Config file example:** `MCPClient.rejectedDirectory` - - **Type:** `string` - - **Default:** `/var/archivematica/sharedDirectory/rejected/` + - **Description:** location of the Archivematica Rejected Directory. + - **Config file example:** `MCPClient.rejectedDirectory` + - **Type:** `string` + - **Default:** `/var/archivematica/sharedDirectory/rejected/` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_WATCHDIRECTORYPATH`**: - - **Description:** location of the Archivematica Watched Directories. - - **Config file example:** `MCPClient.watchDirectoryPath` - - **Type:** `string` - - **Default:** `/var/archivematica/sharedDirectory/watchedDirectories/` + - **Description:** location of the Archivematica Watched Directories. + - **Config file example:** `MCPClient.watchDirectoryPath` + - **Type:** `string` + - **Default:** `/var/archivematica/sharedDirectory/watchedDirectories/` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_CLIENTSCRIPTSDIRECTORY`**: - - **Description:** location of the client scripts directory. - - **Config file example:** `MCPClient.clientScriptsDirectory` - - **Type:** `string` - - **Default:** `/usr/lib/archivematica/MCPClient/clientScripts/` + - **Description:** location of the client scripts directory. + - **Config file example:** `MCPClient.clientScriptsDirectory` + - **Type:** `string` + - **Default:** `/usr/lib/archivematica/MCPClient/clientScripts/` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_CLIENTASSETSDIRECTORY`**: - - **Description:** location of the client assets directory. - - **Config file example:** `MCPClient.clientAssetsDirectory` - - **Type:** `string` - - **Default:** `/usr/lib/archivematica/MCPClient/assets/` + - **Description:** location of the client assets directory. + - **Config file example:** `MCPClient.clientAssetsDirectory` + - **Type:** `string` + - **Default:** `/usr/lib/archivematica/MCPClient/assets/` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_LOADSUPPORTEDCOMMANDSSPECIAL`**: - - **Description:** enables loading special modules. - - **Config file example:** `MCPClient.LoadSupportedCommandsSpecial` - - **Type:** `boolean` - - **Default:** `true` + - **Description:** enables loading special modules. + - **Config file example:** `MCPClient.LoadSupportedCommandsSpecial` + - **Type:** `boolean` + - **Default:** `true` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_MCPARCHIVEMATICASERVER`**: - - **Description:** address of the Gearman server. - - **Config file example:** `MCPClient.MCPArchivematicaServer` - - **Type:** `string` - - **Default:** `localhost:4730` + - **Description:** address of the Gearman server. + - **Config file example:** `MCPClient.MCPArchivematicaServer` + - **Type:** `string` + - **Default:** `localhost:4730` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_ARCHIVEMATICACLIENTMODULES`**: - - **Description:** location of the client modules configuration file. This can be useful when the user wants to set up workers that can only work in a limited number of tasks, e.g. a worker exclusively dedicated to antivirus scanning or file identification. - - **Config file example:** `MCPClient.archivematicaClientModules` - - **Type:** `string` - - **Default:** `/usr/lib/archivematica/MCPClient/archivematicaClientModules` + - **Description:** location of the client modules configuration file. This can + be useful when the user wants to set up workers that can only work in a + limited number of tasks, e.g. a worker exclusively dedicated to antivirus + scanning or file identification. + - **Config file example:** `MCPClient.archivematicaClientModules` + - **Type:** `string` + - **Default:** `/usr/lib/archivematica/MCPClient/archivematicaClientModules` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_ELASTICSEARCHSERVER`**: - - **Description:** address of the Elasticsearch server. - - **Config file example:** `MCPClient.elasticsearchServer` - - **Type:** `string` - - **Default:** `localhost:9200` + - **Description:** address of the Elasticsearch server. + - **Config file example:** `MCPClient.elasticsearchServer` + - **Type:** `string` + - **Default:** `localhost:9200` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_ELASTICSEARCHTIMEOUT`**: - - **Description:** configures the Elasticsearch client to stop waiting for a response after a given number of seconds. - - **Config file example:** `MCPClient.elasticsearchTimeout` - - **Type:** `float` - - **Default:** `10` + - **Description:** configures the Elasticsearch client to stop waiting for a + response after a given number of seconds. + - **Config file example:** `MCPClient.elasticsearchTimeout` + - **Type:** `float` + - **Default:** `10` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_SEARCH_ENABLED`**: - - **Description:** controls what Elasticsearch indexes are enabled: - - When set to `aips` or `false`, certain client scripts will exit without interacting with the Transfers related indexes, e.g., elasticSearchIndex_v0.0 and postStoreAIPHook_v1.0. - - When set to `transfers` or `false`, certain client scripts will exit without interacting with the AIPs related indexes, e.g., indexAIP_v0.0. - - When set to `aips,transfers` (the order does not matter) or `true`, all interactions with the Elasticsearch indexes will be made. - - **Config file example:** `MCPClient.search_enabled` - - **Type:** `boolean` or `string` - - **Default:** `true` + - **Description:** controls what Elasticsearch indexes are enabled: + - When set to `aips` or `false`, certain client scripts will exit without + interacting with the Transfers related indexes, e.g., elasticSearchIndex_v0.0 + and postStoreAIPHook_v1.0. + - When set to `transfers` or `false`, certain client scripts will exit + without interacting with the AIPs related indexes, e.g., indexAIP_v0.0. + - When set to `aips,transfers` (the order does not matter) or `true`, all + interactions with the Elasticsearch indexes will be made. + - **Config file example:** `MCPClient.search_enabled` + - **Type:** `boolean` or `string` + - **Default:** `true` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_INDEX_AIP_CONTINUE_ON_ERROR`**: - - **Description:** controls whether Archivematica continues processing the package when an error occurs in `indexAIP_v0.0`. - - **Config file example:** `MCPClient.index_aip_continue_on_error` - - **Type:** `boolean` - - **Default:** `false` + - **Description:** controls whether Archivematica continues processing the + package when an error occurs in `indexAIP_v0.0`. + - **Config file example:** `MCPClient.index_aip_continue_on_error` + - **Type:** `boolean` + - **Default:** `false` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_REMOVABLEFILES`**: - - **Description:** comma-separated listed of file names that will be deleted. - - **Config file example:** `MCPClient.removableFiles` - - **Type:** `string` - - **Default:** `Thumbs.db, Icon, Icon\r, .DS_Store` + - **Description:** comma-separated listed of file names that will be deleted. + - **Config file example:** `MCPClient.removableFiles` + - **Type:** `string` + - **Default:** `Thumbs.db, Icon, Icon\r, .DS_Store` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_TEMP_DIR`**: - - **Description:** location of the temporary directory. - - **Config file example:** `MCPClient.temp_dir` - - **Type:** `string` - - **Default:** `/var/archivematica/sharedDirectory/tmp` + - **Description:** location of the temporary directory. + - **Config file example:** `MCPClient.temp_dir` + - **Type:** `string` + - **Default:** `/var/archivematica/sharedDirectory/tmp` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_STORAGE_SERVICE_CLIENT_TIMEOUT`**: - - **Description:** configures the Storage Service client to stop waiting for a response after a given number of seconds. - - **Config file example:** `MCPClient.storage_service_client_timeout` - - **Type:** `float` - - **Default:** `86400` + - **Description:** configures the Storage Service client to stop waiting for a + response after a given number of seconds. + - **Config file example:** `MCPClient.storage_service_client_timeout` + - **Type:** `float` + - **Default:** `86400` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_STORAGE_SERVICE_CLIENT_QUICK_TIMEOUT`**: - - **Description:** configures the Storage Service client to stop waiting for a response after a given number of seconds when the client uses asynchronous API endpoints. - - **Config file example:** `MCPClient.storage_service_client_quick_timeout` - - **Type:** `float` - - **Default:** `5` + - **Description:** configures the Storage Service client to stop waiting for a + response after a given number of seconds when the client uses asynchronous + API endpoints. + - **Config file example:** `MCPClient.storage_service_client_quick_timeout` + - **Type:** `float` + - **Default:** `5` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_AGENTARCHIVES_CLIENT_TIMEOUT`**: - - **Description:** configures the agentarchives client to stop waiting for a response after a given number of seconds. - - **Config file example:** `MCPClient.agentarchives_client_timeout` - - **Type:** `float` - - **Default:** `300` + - **Description:** configures the agentarchives client to stop waiting for a + response after a given number of seconds. + - **Config file example:** `MCPClient.agentarchives_client_timeout` + - **Type:** `float` + - **Default:** `300` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_PROMETHEUS_BIND_ADDRESS`**: - - **Description:** when set to a non-empty string, its value is parsed as the IP address on which to serve Prometheus metrics. If this value is not provided, but ``ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_PROMETHEUS_BIND_PORT`` is, then 127.0.0.1 will - be used. - - **Config file example:** `MCPClient.prometheus_bind_addresss` - - **Type:** `string` - - **Default:** `""` + - **Description:** when set to a non-empty string, its value is parsed as the + IP address on which to serve Prometheus metrics. If this value is not + provided, but ``ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_PROMETHEUS_BIND_PORT`` is, + then 127.0.0.1 will be used. + - **Config file example:** `MCPClient.prometheus_bind_addresss` + - **Type:** `string` + - **Default:** `""` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_PROMETHEUS_BIND_PORT`**: - - **Description:** The port on which to serve Prometheus metrics. + - **Description:** The port on which to serve Prometheus metrics. If this value is not provided, metrics will not be served. - - **Config file example:** `MCPClient.prometheus_bind_port` - - **Type:** `int` - - **Default:** `""` + - **Config file example:** `MCPClient.prometheus_bind_port` + - **Type:** `int` + - **Default:** `""` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_METADATA_XML_VALIDATION_ENABLED`**: - - **Description:** (**Experimental**) Determines if the XML files in the `metadata` directory of a SIP should be validated against a set of XML schemas, recorded in the METS file and indexed as custom metadata. See the [feature variables](#metadata-xml-validation-variables) section below. - - **Config file example:** `MCPClient.metadata_xml_validation_enabled` - - **Type:** `boolean` - - **Default:** `false` + - **Description:** (**Experimental**) Determines if the XML files in the + `metadata` directory of a SIP should be validated against a set of XML + schemas, recorded in the METS file and indexed as custom metadata. See the + [feature variables](#metadata-xml-validation-variables) section below. + - **Config file example:** `MCPClient.metadata_xml_validation_enabled` + - **Type:** `boolean` + - **Default:** `false` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_CLAMAV_SERVER`**: - - **Description:** configures the `clamdscanner` backend so it knows how to reach the clamd server via UNIX socket (if the value starts with /) or TCP socket (form `host:port`, e.g.: `myclamad:3310`). - - **Config file example:** `MCPClient.clamav_server` - - **Type:** `string` - - **Default:** `/var/run/clamav/clamd.ctl` + - **Description:** configures the `clamdscanner` backend so it knows how to + reach the clamd server via UNIX socket (if the value starts with /) or TCP + socket (form `host:port`, e.g.: `myclamad:3310`). + - **Config file example:** `MCPClient.clamav_server` + - **Type:** `string` + - **Default:** `/var/run/clamav/clamd.ctl` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_CLAMAV_PASS_BY_STREAM`**: - - **Description:** configures the `clamdscanner` backend to stream the file's contents to clamd. This is useful when clamd does not have access to the filesystem where the file is stored. When disabled, the files are read from the filesystem by reference. - - **Config file example:** `MCPClient.clamav_pass_by_stream` - - **Type:** `boolean` - - **Default:** `true` + - **Description:** configures the `clamdscanner` backend to stream the file's + contents to clamd. This is useful when clamd does not have access to the + filesystem where the file is stored. When disabled, the files are read from + the filesystem by reference. + - **Config file example:** `MCPClient.clamav_pass_by_stream` + - **Type:** `boolean` + - **Default:** `true` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_CLAMAV_CLIENT_TIMEOUT`**: - - **Description:** configures the `clamdscanner` backend to stop waiting for a response after a given number of seconds. - - **Config file example:** `MCPClient.clamav_client_timeout` - - **Type:** `float` - - **Default:** `86400` + - **Description:** configures the `clamdscanner` backend to stop waiting for a + response after a given number of seconds. + - **Config file example:** `MCPClient.clamav_client_timeout` + - **Type:** `float` + - **Default:** `86400` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_CLAMAV_CLIENT_BACKEND`**: - - **Description:** the ClamAV backend used for anti-virus scanning. The two options that are available are: `clamscanner` (via CLI) and `clamdscanner` (over TCP or UNIX socket). - - **Config file example:** `MCPClient.clamav_client_backend` - - **Type:** `string` - - **Default:** `clamdscanner` + - **Description:** the ClamAV backend used for anti-virus scanning. The two + options that are available are: `clamscanner` (via CLI) and `clamdscanner` + (over TCP or UNIX socket). + - **Config file example:** `MCPClient.clamav_client_backend` + - **Type:** `string` + - **Default:** `clamdscanner` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_CLAMAV_CLIENT_MAX_FILE_SIZE`**: - - **Description:** files larger than this limit will not be scanned. The unit used is megabyte (MB). - - **Config file example:** `MCPClient.clamav_client_max_file_size` - - **Type:** `float` - - **Default:** `2000` + - **Description:** files larger than this limit will not be scanned. The unit + used is megabyte (MB). + - **Config file example:** `MCPClient.clamav_client_max_file_size` + - **Type:** `float` + - **Default:** `2000` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_CLAMAV_CLIENT_MAX_SCAN_SIZE`**: - - **Description**: sets the maximum amount of data to be scanned for each input file. Files larger than this value will be scanned but only up to this limit. Archives and other containers are recursively extracted and scanned up to this value. The unit used is megabyte (MB). - - **Config file example:** `MCPClient.clamav_client_max_scan_size` - - **Type:** `float` - - **Default:** `2000` + - **Description**: sets the maximum amount of data to be scanned for each + input file. Files larger than this value will be scanned but only up to this + limit. Archives and other containers are recursively extracted and scanned + up to this value. The unit used is megabyte (MB). + - **Config file example:** `MCPClient.clamav_client_max_scan_size` + - **Type:** `float` + - **Default:** `2000` - **`ARCHIVEMATICA_MCPCLIENT_CLIENT_ENGINE`** - - **Description:** a database setting. See [DATABASES](https://docs.djangoproject.com/en/1.8/ref/settings/#databases) for more details. - - **Config file example:** `client.engine` - - **Type:** `string` - - **Default:** `django.db.backends.mysql` + - **Description:** a database setting. See [DATABASES] for more details. + - **Config file example:** `client.engine` + - **Type:** `string` + - **Default:** `django.db.backends.mysql` - **`ARCHIVEMATICA_MCPCLIENT_CLIENT_DATABASE`** - - **Description:** a database setting. See [DATABASES](https://docs.djangoproject.com/en/1.8/ref/settings/#databases) for more details. - - **Config file example:** `client.database` - - **Type:** `string` - - **Default:** `MCP` + - **Description:** a database setting. See [DATABASES] for more details. + - **Config file example:** `client.database` + - **Type:** `string` + - **Default:** `MCP` - **`ARCHIVEMATICA_MCPCLIENT_CLIENT_USER`** - - **Description:** a database setting. See [DATABASES](https://docs.djangoproject.com/en/1.8/ref/settings/#databases) for more details. - - **Config file example:** `client.user` - - **Type:** `string` - - **Default:** `archivematica` + - **Description:** a database setting. See [DATABASES] for more details. + - **Config file example:** `client.user` + - **Type:** `string` + - **Default:** `archivematica` - **`ARCHIVEMATICA_MCPCLIENT_CLIENT_PASSWORD`** - - **Description:** a database setting. See [DATABASES](https://docs.djangoproject.com/en/1.8/ref/settings/#databases) for more details. - - **Config file example:** `client.password` - - **Type:** `string` - - **Default:** `demo` + - **Description:** a database setting. See [DATABASES] for more details. + - **Config file example:** `client.password` + - **Type:** `string` + - **Default:** `demo` - **`ARCHIVEMATICA_MCPCLIENT_CLIENT_HOST`** - - **Description:** a database setting. See [DATABASES](https://docs.djangoproject.com/en/1.8/ref/settings/#databases) for more details. - - **Config file example:** `client.host` - - **Type:** `string` - - **Default:** `localhost` + - **Description:** a database setting. See [DATABASES] for more details. + - **Config file example:** `client.host` + - **Type:** `string` + - **Default:** `localhost` - **`ARCHIVEMATICA_MCPCLIENT_CLIENT_PORT`** - - **Description:** a database setting. See [DATABASES](https://docs.djangoproject.com/en/1.8/ref/settings/#databases) for more details. - - **Config file example:** `client.port` - - **Type:** `string` - - **Default:** `3306` + - **Description:** a database setting. See [DATABASES] for more details. + - **Config file example:** `client.port` + - **Type:** `string` + - **Default:** `3306` - **`ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_CAPTURE_CLIENT_SCRIPT_OUTPUT`**: - - **Description:** controls whether or not to capture stdout from client script subprocesses. If set to `true`, then stdout is captured; if set to `false`, then stdout is not captured. If set to `true`, then stderr is captured; if set to `false`, then stderr is captured only if the subprocess has failed, i.e., returned a non-zero exit code. - - **Config file example:** `MCPClient.capture_client_script_output` - - **Type:** `boolean` - - **Default:** `true` - -- ** `ARCHIVEMATICA_MCPCLIENT_EMAIL_BACKEND`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.backend` - - **Type:** `string` - - **Default:** `django.core.mail.backends.console.EmailBackend` - -- ** `ARCHIVEMATICA_MCPCLIENT_EMAIL_HOST`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.host` - - **Type:** `string` - - **Default:** `smtp.gmail.com` - -- ** `ARCHIVEMATICA_MCPCLIENT_EMAIL_HOST_USER`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.host_user` - - **Type:** `string` - - **Default:** `your_email@example.com` - -- ** `ARCHIVEMATICA_MCPCLIENT_EMAIL_HOST_PASSWORD`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.host_password` - - **Type:** `string` - - **Default:** `None` - -- ** `ARCHIVEMATICA_MCPCLIENT_EMAIL_PORT`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.port` - - **Type:** `integer` - - **Default:** `587` - -- ** `ARCHIVEMATICA_MCPCLIENT_EMAIL_SSL_CERTFILE`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.ssl_certfile` - - **Type:** `string` - - **Default:** `None` - -- ** `ARCHIVEMATICA_MCPCLIENT_EMAIL_SSL_KEYFILE`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.ssl_keyfile` - - **Type:** `string` - - **Default:** `None` - -- ** `ARCHIVEMATICA_MCPCLIENT_EMAIL_USE_SSL`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.use_ssl` - - **Type:** `boolean` - - **Default:** `False` - -- ** `ARCHIVEMATICA_MCPCLIENT_EMAIL_USE_TLS`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.use_tls` - - **Type:** `boolean` - - **Default:** `True` - -- ** `ARCHIVEMATICA_MCPCLIENT_EMAIL_FILE_PATH`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.file_path` - - **Type:** `string` - - **Default:** `None` - -- ** `ARCHIVEMATICA_MCPCLIENT_EMAIL_DEFAULT_FROM_EMAIL`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.default_from_email` - - **Type:** `string` - - **Default:** `webmaster@example.com` - -- ** `ARCHIVEMATICA_MCPCLIENT_EMAIL_SUBJECT_PREFIX`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.subject_prefix` - - **Type:** `string` - - **Default:** `[Archivematica]` - -- ** `ARCHIVEMATICA_MCPCLIENT_EMAIL_TIMEOUT`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.timeout` - - **Type:** `integer` - - **Default:** `300` - -- ** `ARCHIVEMATICA_MCPCLIENT_EMAIL_SERVER_EMAIL`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. When the value is `None`, Archivematica uses the value in `EMAIL_HOST_USER`. - - **Config file example:** `email.server_email` - - **Type:** `string` - - **Default:** `None` + - **Description:** controls whether or not to capture stdout from client + script subprocesses. If set to `true`, then stdout is captured; if set to + `false`, then stdout is not captured. If set to `true`, then stderr is + captured; if set to `false`, then stderr is captured only if the subprocess + has failed, i.e., returned a non-zero exit code. + - **Config file example:** `MCPClient.capture_client_script_output` + - **Type:** `boolean` + - **Default:** `true` + +- **`ARCHIVEMATICA_MCPCLIENT_EMAIL_BACKEND`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.backend` + - **Type:** `string` + - **Default:** `django.core.mail.backends.console.EmailBackend` + +- **`ARCHIVEMATICA_MCPCLIENT_EMAIL_HOST`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.host` + - **Type:** `string` + - **Default:** `smtp.gmail.com` + +- **`ARCHIVEMATICA_MCPCLIENT_EMAIL_HOST_USER`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.host_user` + - **Type:** `string` + - **Default:** `your_email@example.com` + +- **`ARCHIVEMATICA_MCPCLIENT_EMAIL_HOST_PASSWORD`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.host_password` + - **Type:** `string` + - **Default:** `None` + +- **`ARCHIVEMATICA_MCPCLIENT_EMAIL_PORT`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.port` + - **Type:** `integer` + - **Default:** `587` + +- **`ARCHIVEMATICA_MCPCLIENT_EMAIL_SSL_CERTFILE`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.ssl_certfile` + - **Type:** `string` + - **Default:** `None` + +- **`ARCHIVEMATICA_MCPCLIENT_EMAIL_SSL_KEYFILE`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.ssl_keyfile` + - **Type:** `string` + - **Default:** `None` + +- **`ARCHIVEMATICA_MCPCLIENT_EMAIL_USE_SSL`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.use_ssl` + - **Type:** `boolean` + - **Default:** `False` + +- **`ARCHIVEMATICA_MCPCLIENT_EMAIL_USE_TLS`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.use_tls` + - **Type:** `boolean` + - **Default:** `True` + +- **`ARCHIVEMATICA_MCPCLIENT_EMAIL_FILE_PATH`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.file_path` + - **Type:** `string` + - **Default:** `None` + +- **`ARCHIVEMATICA_MCPCLIENT_EMAIL_DEFAULT_FROM_EMAIL`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.default_from_email` + - **Type:** `string` + - **Default:** `webmaster@example.com` + +- **`ARCHIVEMATICA_MCPCLIENT_EMAIL_SUBJECT_PREFIX`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.subject_prefix` + - **Type:** `string` + - **Default:** `[Archivematica]` + +- **`ARCHIVEMATICA_MCPCLIENT_EMAIL_TIMEOUT`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.timeout` + - **Type:** `integer` + - **Default:** `300` + +- **`ARCHIVEMATICA_MCPCLIENT_EMAIL_SERVER_EMAIL`**: + - **Description:** an email setting. See [Sending email] for more details. + When the value is `None`, Archivematica uses the value in `EMAIL_HOST_USER`. + - **Config file example:** `email.server_email` + - **Type:** `string` + - **Default:** `None` ### Metadata XML validation variables **This feature is experimental, please share your feedback!** -These variables specify how XML files contained in the `metadata` directory of a SIP should be validated. Only applicable if `ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_METADATA_XML_VALIDATION_ENABLED` is set. +These variables specify how XML files contained in the `metadata` directory of a +SIP should be validated. Only applicable if `ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_METADATA_XML_VALIDATION_ENABLED` +is set. - **`METADATA_XML_VALIDATION_SETTINGS_FILE`**: - - **Description:** Path to a Python module containing the following Django settings: - - `XML_VALIDATION`: a dictionary which keys are strings that contain either an [XML schema location](https://www.w3.org/TR/xmlschema-1/#xsi_schemaLocation), an [XML namespace](https://www.w3.org/TR/xml-names/#sec-namespaces) or an XML element tag, and which values are either strings that contain an absolute local path or an external URL to an XML schema file, or `None` to indicate that no validation should be performed. - - `XML_VALIDATION_FAIL_ON_ERROR`: a boolean that indicates if the SIP ingest workflow should stop on validation errors. Defaults to `False`. - - An `ImproperlyConfigured` exception will be raised if the Python module cannot be imported or does not contain the required settings. - - The goal of the validation process is to determine a **validation key** from the root node of each XML file in the `metadata` directory of the SIP and then get an XML schema file to validate against using the `XML_VALIDATION` dictionary. If the value for the validation key is set to `None` the XML metadata file is added to the METS without any validation. - - The validation key of each metadata XML file is determined from its root node in the following order: - - 1. The XML schema location defined in the `xsi:noNamespaceSchemaLocation` attribute + - **Description:** Path to a Python module containing the following Django settings: + - `XML_VALIDATION`: a dictionary which keys are strings that contain either + an [XML schema location], an [XML namespace] or an XML element tag, and + which values are either strings that contain an absolute local path or an + external URL to an XML schema file, or `None` to indicate that no + validation should be performed. + - `XML_VALIDATION_FAIL_ON_ERROR`: a boolean that indicates if the SIP ingest + workflow should stop on validation errors. Defaults to `False`. + + An `ImproperlyConfigured` exception will be raised if the Python module + cannot be imported or does not contain the required settings. + + The goal of the validation process is to determine a **validation key** + from the root node of each XML file in the `metadata` directory of the SIP + and then get an XML schema file to validate against using the `XML_VALIDATION` + dictionary. If the value for the validation key is set to `None` the XML + metadata file is added to the METS without any validation. + + The validation key of each metadata XML file is determined from its root + node in the following order: + + 1. The XML schema location defined in the `xsi:noNamespaceSchemaLocation` + attribute 1. The XML schema location defined in the `xsi:schemaLocation` attribute 1. Its XML namespace 1. Its tag name - XML validation works with `DTD`, `XSD` and `RELAX NG` schema types (`.dtd`, `.xsd`, and `.rng`). + XML validation works with `DTD`, `XSD` and `RELAX NG` schema types (`.dtd`, + `.xsd`, and `.rng`). This is how an `XML_VALIDATION` dictionary would look like: @@ -416,8 +470,8 @@ These variables specify how XML files contained in the `metadata` directory of a } ``` - - **Type:** `string` - - **Default:** `` + - **Type:** `string` + - **Default:** `` ## Logging configuration @@ -440,3 +494,10 @@ behaviour described above. The [`clientConfig.logging.json`](./clientConfig.logging.json) file in this directory provides an example that implements the logging behaviour used in Archivematica 1.6.1 and earlier. + +[TIME_ZONE]: https://docs.djangoproject.com/en/1.8/ref/settings/#time-zone +[SECRET_KEY]: https://docs.djangoproject.com/en/1.8/ref/settings/#secret-key +[DATABASES]: https://docs.djangoproject.com/en/1.8/ref/settings/#databases +[Sending email]: https://docs.djangoproject.com/en/1.8/topics/email/ +[XML schema location]: https://www.w3.org/TR/xmlschema-1/#xsi_schemaLocation +[XML namespace]: https://www.w3.org/TR/xml-names/#sec-namespaces diff --git a/src/MCPServer/install/README.md b/src/MCPServer/install/README.md index 381ad95577..bb090070fa 100644 --- a/src/MCPServer/install/README.md +++ b/src/MCPServer/install/README.md @@ -2,12 +2,17 @@ ## Table of contents + + + - [Introduction](#introduction) - [Environment variables](#environment-variables) - [Configuration file](#configuration-file) - [Parameter list](#parameter-list) - [Logging configuration](#logging-configuration) + + ## Introduction Archivematica components can be configured using multiple methods. All @@ -69,259 +74,278 @@ non-database parameters could be set in the dbsettings file. This is the full list of variables supported by MCPServer: - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_TIME_ZONE`**: - - **Description:** application time zone. See [TIME_ZONE](https://docs.djangoproject.com/en/1.8/ref/settings/#time-zone) for more details. - - **Config file example:** `MCPServer.time_zone` - - **Type:** `string` - - **Default:** `"UTC"` + - **Description:** application time zone. See [TIME_ZONE] for more details. + - **Config file example:** `MCPServer.time_zone` + - **Type:** `string` + - **Default:** `"UTC"` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_DJANGO_SECRET_KEY`**: - - **Description:** a secret key used for cryptographic signing. See [SECRET_KEY](https://docs.djangoproject.com/en/1.8/ref/settings/#secret-key) for more details. - - **Config file example:** `MCPServer.django_secret_key` - - **Type:** `string` - - :red_circle: **Mandatory!** + - **Description:** a secret key used for cryptographic signing. See [SECRET_KEY] + for more details. + - **Config file example:** `MCPServer.django_secret_key` + - **Type:** `string` + - :red_circle: **Mandatory!** - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_SHAREDDIRECTORY`**: - - **Description:** location of the Archivematica Shared Directory. - - **Config file example:** `MCPServer.sharedDirectory` - - **Type:** `string` - - **Default:** `"/var/archivematica/sharedDirectory/"` + - **Description:** location of the Archivematica Shared Directory. + - **Config file example:** `MCPServer.sharedDirectory` + - **Type:** `string` + - **Default:** `"/var/archivematica/sharedDirectory/"` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_PROCESSINGXMLFILE`**: - - **Description:** name of the processing configuration file. - - **Config file example:** `MCPServer.processingXMLFile` - - **Type:** `string` - - **Default:** `"processingMCP.xml"` + - **Description:** name of the processing configuration file. + - **Config file example:** `MCPServer.processingXMLFile` + - **Type:** `string` + - **Default:** `"processingMCP.xml"` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_MCPARCHIVEMATICASERVER`**: - - **Description:** address of the Gearman server. - - **Config file example:** `MCPServer.MCPArchivematicaServer` - - **Type:** `string` - - **Default:** `"localhost:4730"` + - **Description:** address of the Gearman server. + - **Config file example:** `MCPServer.MCPArchivematicaServer` + - **Type:** `string` + - **Default:** `"localhost:4730"` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_WATCHDIRECTORYPATH`**: - - **Description:** location of the Archivematica Watched Directories. - - **Config file example:** `MCPServer.watchDirectoryPath` - - **Type:** `string` - - **Default:** `"/var/archivematica/sharedDirectory/watchedDirectories/"` + - **Description:** location of the Archivematica Watched Directories. + - **Config file example:** `MCPServer.watchDirectoryPath` + - **Type:** `string` + - **Default:** `"/var/archivematica/sharedDirectory/watchedDirectories/"` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_PROCESSINGDIRECTORY`**: - - **Description:** loation of the processing directory. - - **Config file example:** `MCPServer.processingDirectory` - - **Type:** `string` - - **Default:** `"/var/archivematica/sharedDirectory/currentlyProcessing/"` + - **Description:** loation of the processing directory. + - **Config file example:** `MCPServer.processingDirectory` + - **Type:** `string` + - **Default:** `"/var/archivematica/sharedDirectory/currentlyProcessing/"` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_REJECTEDDIRECTORY`**: - - **Description:** location of the rejected directory. - - **Config file example:** `MCPServer.rejectedDirectory` - - **Type:** `string` - - **Default:** `"/var/archivematica/sharedDirectory/rejected/"` + - **Description:** location of the rejected directory. + - **Config file example:** `MCPServer.rejectedDirectory` + - **Type:** `string` + - **Default:** `"/var/archivematica/sharedDirectory/rejected/"` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_WAITONAUTOAPPROVE`**: - - **Description:** number of seconds that a thread will wait before attempting to approve a chain when it is pre-configured. - - **Config file example:** `MCPServer.waitOnAutoApprove` - - **Type:** `int` - - **Default:** `"0"` + - **Description:** number of seconds that a thread will wait before attempting + to approve a chain when it is pre-configured. + - **Config file example:** `MCPServer.waitOnAutoApprove` + - **Type:** `int` + - **Default:** `"0"` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_SEARCH_ENABLED`**: - - **Description:** controls what Elasticsearch indexes are enabled. This can affect which options the MCP server makes available for user choices. E.g., If set to `false` or `aips`, then the MCP server will not make the "Send to backlong" option available at the "Create SIP(s)" choice point, as it will require the `transfers` indexes. Available options: - - `true`: all indexes enabled. - - `false`: no indexing enabled. - - `aips`: only AIPs related indexes. - - `transfers`: only Transfers related indexes. - - `aips,transfers` (the order does not matter): same as `true`. - - **Config file example:** `MCPServer.search_enabled` - - **Type:** `boolean` or `string` - - **Default:** `true` + - **Description:** controls what Elasticsearch indexes are enabled. This can + affect which options the MCP server makes available for user choices. E.g., + If set to `false` or `aips`, then the MCP server will not make the + "Send to backlong" option available at the "Create SIP(s)" choice point, as + it will require the `transfers` indexes. Available options: + - `true`: all indexes enabled. + - `false`: no indexing enabled. + - `aips`: only AIPs related indexes. + - `transfers`: only Transfers related indexes. + - `aips,transfers` (the order does not matter): same as `true`. + - **Config file example:** `MCPServer.search_enabled` + - **Type:** `boolean` or `string` + - **Default:** `true` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_STORAGE_SERVICE_CLIENT_TIMEOUT`**: - - **Description:** configures the Storage Service client to stop waiting for a response after a given number of seconds. - - **Config file example:** `MCPClient.storage_service_client_timeout` - - **Type:** `float` - - **Default:** `86400` + - **Description:** configures the Storage Service client to stop waiting for a + response after a given number of seconds. + - **Config file example:** `MCPClient.storage_service_client_timeout` + - **Type:** `float` + - **Default:** `86400` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_STORAGE_SERVICE_CLIENT_QUICK_TIMEOUT`**: - - **Description:** configures the Storage Service client to stop waiting for a response after a given number of seconds. This applies only to requests that are supposed to return immediately. - - **Config file example:** `MCPClient.storage_service_client_timeout` - - **Type:** `float` - - **Default:** `5` + - **Description:** configures the Storage Service client to stop waiting for a + response after a given number of seconds. This applies only to requests that + are supposed to return immediately. + - **Config file example:** `MCPClient.storage_service_client_timeout` + - **Type:** `float` + - **Default:** `5` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_PROMETHEUS_BIND_ADDRESS`**: - - **Description:** when set to a non-empty string, its value is parsed as the IP address on which to serve Prometheus metrics. If this value is not provided, but ``ARCHIVEMATICA_MCPSERVER_MCPSERVER_PROMETHEUS_BIND_PORT`` is, then 127.0.0.1 will - be used. - - **Config file example:** `MCPServer.prometheus_bind_addresss` - - **Type:** `string` - - **Default:** `""` + - **Description:** when set to a non-empty string, its value is parsed as the + IP address on which to serve Prometheus metrics. If this value is not + provided, but ``ARCHIVEMATICA_MCPSERVER_MCPSERVER_PROMETHEUS_BIND_PORT`` is, + then 127.0.0.1 will be used. + - **Config file example:** `MCPServer.prometheus_bind_addresss` + - **Type:** `string` + - **Default:** `""` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_PROMETHEUS_BIND_PORT`**: - - **Description:** The port on which to serve Prometheus metrics. + - **Description:** The port on which to serve Prometheus metrics. If this value is not provided, metrics will not be served. - - **Config file example:** `MCPServer.prometheus_bind_port` - - **Type:** `int` - - **Default:** `""` + - **Config file example:** `MCPServer.prometheus_bind_port` + - **Type:** `int` + - **Default:** `""` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_WATCH_DIRECTORY_METHOD`**: - - **Description:** how watched directory polling is done (`poll` or `inotify`). `inotify` is much more efficient, but only available - when using a local filesystem on Linux. - - **Config file example:** `MCPServer.watch_directory_method` - - **Type:** `string` - - **Default:** `"poll"` + - **Description:** how watched directory polling is done (`poll` or `inotify`). + `inotify` is much more efficient, but only available when using a local + filesystem on Linux. + - **Config file example:** `MCPServer.watch_directory_method` + - **Type:** `string` + - **Default:** `"poll"` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_WATCH_DIRECTORY_INTERVAL`**: - - **Description:** time in seconds between filesystem poll intervals. - - **Config file example:** `MCPServer.watch_directory_interval` - - **Type:** `int` - - **Default:** `"1"` + - **Description:** time in seconds between filesystem poll intervals. + - **Config file example:** `MCPServer.watch_directory_interval` + - **Type:** `int` + - **Default:** `"1"` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_BATCH_SIZE`**: - - **Description:** the amount of files that are processed by an instance of MCPClient as a group to speed up certain operations like database updates. - - **Config file example:** `MCPServer.batch_size` - - **Type:** `int` - - **Default:** `"128"` + - **Description:** the amount of files that are processed by an instance of + MCPClient as a group to speed up certain operations like database updates. + - **Config file example:** `MCPServer.batch_size` + - **Type:** `int` + - **Default:** `"128"` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_CONCURRENT_PACKAGES`**: - - **Description:** the number of packages to process concurrently. Should typically correspond - to the number of running MCPClients. - - **Config file example:** `MCPServer.concurrent_packages` - - **Type:** `int` - - **Default:** 1/2 the number of CPU cores available to MCPServer, rounded up. + - **Description:** the number of packages to process concurrently. Should + typically correspond to the number of running MCPClients. + - **Config file example:** `MCPServer.concurrent_packages` + - **Type:** `int` + - **Default:** 1/2 the number of CPU cores available to MCPServer, rounded up. - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_RPC_THREADS`**: - - **Description:** the number of threads used to process RPC requests from the dashboard. - - **Config file example:** `MCPServer.rpc_threads` - - **Type:** `int` - - **Default:** 4 + - **Description:** the number of threads used to process RPC requests from the + dashboard. + - **Config file example:** `MCPServer.rpc_threads` + - **Type:** `int` + - **Default:** 4 - **`ARCHIVEMATICA_MCPSERVER_WORKFLOW_FILE`**: - - **Description:** the path to the user-customised workflow file. If this is empty, Archivematica will load the default workflow. This configuration attribute is not recommended for general use, but only for users with development skills, an understanding for the Archivematica workflow engine, and are willing to maintain their own workflow file. - - **Config file example:** `MCPServer.workflow_file` - - **Type:** `string` - - **Default:** `""` + - **Description:** the path to the user-customised workflow file. If this is + empty, Archivematica will load the default workflow. This configuration + attribute is not recommended for general use, but only for users with + development skills, an understanding for the Archivematica workflow engine, + and are willing to maintain their own workflow file. + - **Config file example:** `MCPServer.workflow_file` + - **Type:** `string` + - **Default:** `""` - **`ARCHIVEMATICA_MCPSERVER_MCPSERVER_WORKER_THREADS`**: - - **Description:** the number of threads used to handle MCPServer jobs. - - **Config file example:** `MCPServer.worker_threads` - - **Type:** `int` - - **Default:** The number of CPU cores available to MCPServer, plus 1. + - **Description:** the number of threads used to handle MCPServer jobs. + - **Config file example:** `MCPServer.worker_threads` + - **Type:** `int` + - **Default:** The number of CPU cores available to MCPServer, plus 1. - **`ARCHIVEMATICA_MCPSERVER_CLIENT_ENGINE`**: - - **Description:** a database setting. See [DATABASES](https://docs.djangoproject.com/en/1.8/ref/settings/#databases) for more details. - - **Config file example:** `client.engine` - - **Type:** `string` - - **Default:** `django.db.backends.mysql` + - **Description:** a database setting. See [DATABASES] for more details. + - **Config file example:** `client.engine` + - **Type:** `string` + - **Default:** `django.db.backends.mysql` - **`ARCHIVEMATICA_MCPSERVER_CLIENT_DATABASE`**: - - **Description:** a database setting. See [DATABASES](https://docs.djangoproject.com/en/1.8/ref/settings/#databases) for more details. - - **Config file example:** `client.database` - - **Type:** `string` - - **Default:** `MCP` + - **Description:** a database setting. See [DATABASES] for more details. + - **Config file example:** `client.database` + - **Type:** `string` + - **Default:** `MCP` - **`ARCHIVEMATICA_MCPSERVER_CLIENT_USER`**: - - **Description:** a database setting. See [DATABASES](https://docs.djangoproject.com/en/1.8/ref/settings/#databases) for more details. - - **Config file example:** `client.user` - - **Type:** `string` - - **Default:** `archivematica` + - **Description:** a database setting. See [DATABASES] for more details. + - **Config file example:** `client.user` + - **Type:** `string` + - **Default:** `archivematica` - **`ARCHIVEMATICA_MCPSERVER_CLIENT_PASSWORD`**: - - **Description:** a database setting. See [DATABASES](https://docs.djangoproject.com/en/1.8/ref/settings/#databases) for more details. - - **Config file example:** `client.password` - - **Type:** `string` - - **Default:** `demo` + - **Description:** a database setting. See [DATABASES] for more details. + - **Config file example:** `client.password` + - **Type:** `string` + - **Default:** `demo` - **`ARCHIVEMATICA_MCPSERVER_CLIENT_HOST`**: - - **Description:** a database setting. See [DATABASES](https://docs.djangoproject.com/en/1.8/ref/settings/#databases) for more details. - - **Config file example:** `client.host` - - **Type:** `string` - - **Default:** `localhost` + - **Description:** a database setting. See [DATABASES] for more details. + - **Config file example:** `client.host` + - **Type:** `string` + - **Default:** `localhost` - **`ARCHIVEMATICA_MCPSERVER_CLIENT_PORT`**: - - **Description:** a database setting. See [DATABASES](https://docs.djangoproject.com/en/1.8/ref/settings/#databases) for more details. - - **Config file example:** `client.port` - - **Type:** `string` - - **Default:** `3306` - -- ** `ARCHIVEMATICA_MCPSERVER_EMAIL_BACKEND`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.backend` - - **Type:** `string` - - **Default:** `django.core.mail.backends.console.EmailBackend` - -- ** `ARCHIVEMATICA_MCPSERVER_EMAIL_HOST`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.host` - - **Type:** `string` - - **Default:** `smtp.gmail.com` - -- ** `ARCHIVEMATICA_MCPSERVER_EMAIL_HOST_USER`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.host_user` - - **Type:** `string` - - **Default:** `your_email@example.com` - -- ** `ARCHIVEMATICA_MCPSERVER_EMAIL_HOST_PASSWORD`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.host_password` - - **Type:** `string` - - **Default:** `None` - -- ** `ARCHIVEMATICA_MCPSERVER_EMAIL_PORT`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.port` - - **Type:** `integer` - - **Default:** `587` - -- ** `ARCHIVEMATICA_MCPSERVER_EMAIL_SSL_CERTFILE`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.ssl_certfile` - - **Type:** `string` - - **Default:** `None` - -- ** `ARCHIVEMATICA_MCPSERVER_EMAIL_SSL_KEYFILE`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.ssl_keyfile` - - **Type:** `string` - - **Default:** `None` - -- ** `ARCHIVEMATICA_MCPSERVER_EMAIL_USE_SSL`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.use_ssl` - - **Type:** `boolean` - - **Default:** `False` - -- ** `ARCHIVEMATICA_MCPSERVER_EMAIL_USE_TLS`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.use_tls` - - **Type:** `boolean` - - **Default:** `True` - -- ** `ARCHIVEMATICA_MCPSERVER_EMAIL_FILE_PATH`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.file_path` - - **Type:** `string` - - **Default:** `None` - -- ** `ARCHIVEMATICA_MCPSERVER_EMAIL_DEFAULT_FROM_EMAIL`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.default_from_email` - - **Type:** `string` - - **Default:** `webmaster@example.com` - -- ** `ARCHIVEMATICA_MCPSERVER_EMAIL_SUBJECT_PREFIX`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.subject_prefix` - - **Type:** `string` - - **Default:** `[Archivematica]` - -- ** `ARCHIVEMATICA_MCPSERVER_EMAIL_TIMEOUT`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. - - **Config file example:** `email.timeout` - - **Type:** `integer` - - **Default:** `300` - -- ** `ARCHIVEMATICA_MCPSERVER_EMAIL_SERVER_EMAIL`**: - - **Description:** an email setting. See [Sending email](https://docs.djangoproject.com/en/1.8/topics/email/) for more details. When the value is `None`, Archivematica uses the value in `EMAIL_HOST_USER`. - - **Config file example:** `email.server_email` - - **Type:** `string` - - **Default:** `None` + - **Description:** a database setting. See [DATABASES] for more details. + - **Config file example:** `client.port` + - **Type:** `string` + - **Default:** `3306` + +- **`ARCHIVEMATICA_MCPSERVER_EMAIL_BACKEND`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.backend` + - **Type:** `string` + - **Default:** `django.core.mail.backends.console.EmailBackend` + +- **`ARCHIVEMATICA_MCPSERVER_EMAIL_HOST`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.host` + - **Type:** `string` + - **Default:** `smtp.gmail.com` + +- **`ARCHIVEMATICA_MCPSERVER_EMAIL_HOST_USER`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.host_user` + - **Type:** `string` + - **Default:** `your_email@example.com` + +- **`ARCHIVEMATICA_MCPSERVER_EMAIL_HOST_PASSWORD`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.host_password` + - **Type:** `string` + - **Default:** `None` + +- **`ARCHIVEMATICA_MCPSERVER_EMAIL_PORT`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.port` + - **Type:** `integer` + - **Default:** `587` + +- **`ARCHIVEMATICA_MCPSERVER_EMAIL_SSL_CERTFILE`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.ssl_certfile` + - **Type:** `string` + - **Default:** `None` + +- **`ARCHIVEMATICA_MCPSERVER_EMAIL_SSL_KEYFILE`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.ssl_keyfile` + - **Type:** `string` + - **Default:** `None` + +- **`ARCHIVEMATICA_MCPSERVER_EMAIL_USE_SSL`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.use_ssl` + - **Type:** `boolean` + - **Default:** `False` + +- **`ARCHIVEMATICA_MCPSERVER_EMAIL_USE_TLS`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.use_tls` + - **Type:** `boolean` + - **Default:** `True` + +- **`ARCHIVEMATICA_MCPSERVER_EMAIL_FILE_PATH`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.file_path` + - **Type:** `string` + - **Default:** `None` + +- **`ARCHIVEMATICA_MCPSERVER_EMAIL_DEFAULT_FROM_EMAIL`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.default_from_email` + - **Type:** `string` + - **Default:** `webmaster@example.com` + +- **`ARCHIVEMATICA_MCPSERVER_EMAIL_SUBJECT_PREFIX`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.subject_prefix` + - **Type:** `string` + - **Default:** `[Archivematica]` + +- **`ARCHIVEMATICA_MCPSERVER_EMAIL_TIMEOUT`**: + - **Description:** an email setting. See [Sending email] for more details. + - **Config file example:** `email.timeout` + - **Type:** `integer` + - **Default:** `300` + +- **`ARCHIVEMATICA_MCPSERVER_EMAIL_SERVER_EMAIL`**: + - **Description:** an email setting. See [Sending email] for more details. + When the value is `None`, Archivematica uses the value in `EMAIL_HOST_USER`. + - **Config file example:** `email.server_email` + - **Type:** `string` + - **Default:** `None` ## Logging configuration @@ -344,3 +368,8 @@ behaviour described above. The [`serverConfig.logging.json`](./serverConfig.logging.json) file in this directory provides an example that implements the logging behaviour used in Archivematica 1.6.1 and earlier. + +[TIME_ZONE]: https://docs.djangoproject.com/en/1.8/ref/settings/#time-zone +[SECRET_KEY]: https://docs.djangoproject.com/en/1.8/ref/settings/#secret-key +[DATABASES]: https://docs.djangoproject.com/en/1.8/ref/settings/#databases +[Sending email]: https://docs.djangoproject.com/en/1.8/topics/email/ diff --git a/src/dashboard/frontend/README.md b/src/dashboard/frontend/README.md index d7ef974ced..0cfcf6f4ac 100644 --- a/src/dashboard/frontend/README.md +++ b/src/dashboard/frontend/README.md @@ -1,43 +1,55 @@ # Archivematica dashboard frontend components -Getting started ---------------- +## Getting started -This application uses npm to manage its dependencies; dependencies used in the browser are bundled together using [webpack](https://webpack.github.io). -When first installing or updating the application, run `npm install` in the terminal to install all dependencies and build a copy of the bundled app. +This application uses npm to manage its dependencies; dependencies used in the +browser are bundled together using [webpack]. When first installing or updating +the application, run `npm install` in the terminal to install all dependencies +and build a copy of the bundled app. -Running a server ----------------- +## Running a server -This application currently can't be run standalone; it can only be run as a part of an Archivematica installation, since it depends on some external JavaScript and some backend APIs. +This application currently can't be run standalone; it can only be run as a part +of an Archivematica installation, since it depends on some external JavaScript +and some backend APIs. -Application architecture ------------------------- +## Application architecture ### Structure -This application is structured according to the guidelines in the [community Angular style-guide](https://github.com/johnpapa/angular-styleguide#application-structure). +This application is structured according to the guidelines in the +[community Angular style-guide]. ### Separation of concerns -Each individual feature of the application should be developed as a separate scope with its own controller. -Scopes should never be nested in partials. -Each controller should contain as little logic as possible; controllers are primarily used to assign scope variables and call methods on other objects. -The majority of the application's logic is intended to be held in directives, filters, and services. +Each individual feature of the application should be developed as a separate +scope with its own controller. Scopes should never be nested in partials. Each +controller should contain as little logic as possible; controllers are primarily +used to assign scope variables and call methods on other objects. The majority +of the application's logic is intended to be held in directives, filters, and +services. ### Sharing data -Since each controller scope is separate, data can't be directly shared between controllers. -Instead, data sharing occurs using Angular *service* objects. -Each service object is a singleton object which can have both instance methods and properties; bindable attributes of a service should be assigned to properties, while methods should only be used for getters whose values do not need to be watched for changes. +Since each controller scope is separate, data can't be directly shared between +controllers. Instead, data sharing occurs using Angular *service* objects. Each +service object is a singleton object which can have both instance methods and +properties; bindable attributes of a service should be assigned to properties, +while methods should only be used for getters whose values do not need to be +watched for changes. This example, based on the transfer tree/report pane workflow, explains how it works. -One controller, the `TreeController`, contains a transfer tree UI that allows the user to select elements. -Selected elements are then filtered and displayed within the scope of another controller, ReportController. -Since the two controllers are completely separate scopes, we need an intermediary to share data between the two of them; to do that, we create a `SelectedFiles` service which has a method to add and remove entries, and a bindable property called `selected` which simply lists all selected files. +One controller, the `TreeController`, contains a transfer tree UI that allows +the user to select elements. Selected elements are then filtered and displayed +within the scope of another controller, ReportController. Since the two +controllers are completely separate scopes, we need an intermediary to share +data between the two of them; to do that, we create a `SelectedFiles` service +which has a method to add and remove entries, and a bindable property called +`selected` which simply lists all selected files. -First, in the TreeController, we inject the SelectedFiles service, and add or remove objects from it every time we've detected a change to the user's selection. +First, in the TreeController, we inject the SelectedFiles service, and add or +remove objects from it every time we've detected a change to the user's selection. ```js angular.module('treeController', []).controller('TreeController', function(Tree, SelectedFiles) { @@ -52,7 +64,8 @@ angular.module('treeController', []).controller('TreeController', function(Tree, ``` Then, in ReportController, we assign the service object to a scope variable. -We assign the service object itself and not its property so that the digest loop can detect changes to the property. +We assign the service object itself and not its property so that the digest loop +can detect changes to the property. ```js angular.module('reportController', []).controller('ReportController', function($scope, SelectedFiles) { @@ -60,7 +73,8 @@ angular.module('reportController', []).controller('ReportController', function($ }); ``` -Finally, in the template, we can now iterate over the contents of the `records` scope variable and have those be updated with every change made to `SelectedFiles.selected`: +Finally, in the template, we can now iterate over the contents of the `records` +scope variable and have those be updated with every change made to `SelectedFiles.selected`: ```html
@@ -68,25 +82,31 @@ Finally, in the template, we can now iterate over the contents of the `records`
``` -Testing -------- +## Testing -Unit tests are written using the [Jasmine](https://jasmine.github.io/2.3/introduction.html) test framework. -This section will give a high-level overview of this application's tests, but is not meant to replace the Jasmine documentation. +Unit tests are written using the [Jasmine] test framework. This section will +give a high-level overview of this application's tests, but is not meant to +replace the Jasmine documentation. -Unit tests can be found in the "tests/unit" directory in the root of the application. -Each distinct feature being tested should be given its own "spec" file; for example, the spec for the "facet" feature is named "facetSpec.js". +Unit tests can be found in the "tests/unit" directory in the root of the +application. Each distinct feature being tested should be given its own "spec" +file; for example, the spec for the "facet" feature is named "facetSpec.js". ### Running tests Tests can be run inside the root directory of the application by running `npm test`. By default, tests are run in Chrome; Chrome must be installed to run the tests. -### Structure +### Test structure + +Each test file is treated as a *specification* (or spec) of an aspect of the +application's functionality; each test is mean to be readable as a specification +for how the application should behave. -Each test file is treated as a *specification* (or spec) of an aspect of the application's functionality; each test is mean to be readable as a specification for how the application should behave. -Each spec is comprised of a `describe` block, which describes an individual feature; that `describe` block then contains `it` blocks, which are used to define individual aspects of that feature. -For example, a simple test file could look like this: +Each spec is comprised of a `describe` block, which describes an individual +feature; that `describe` block then contains `it` blocks, which are used to +define individual aspects of that feature. For example, a simple test file could +look like this: ```js describe('MyFeature', function() { @@ -96,8 +116,9 @@ describe('MyFeature', function() { }); ``` -At the top of each `describe` block, there should be one or more `beforeEach` statements to set up the environment for each test. -At a minimum, it's necessary to load the appropriate module being tested, for example: +At the top of each `describe` block, there should be one or more `beforeEach` +statements to set up the environment for each test. At a minimum, it's necessary +to load the appropriate module being tested, for example: ```js beforeEach(module('myFeatureModule')); @@ -105,8 +126,11 @@ beforeEach(module('myFeatureModule')); ### Mocking REST calls -Some tests call code which performs asynchronous REST calls; these require extra handling in order to ensure the data is fetched and the test conditions execute before the test function completes. -These REST calls can be mocked using Angular's `_$httpBackend_` feature, which allows specific REST requests to be mocked and responses to be returned at a predictable point of time. +Some tests call code which performs asynchronous REST calls; these require extra +handling in order to ensure the data is fetched and the test conditions execute +before the test function completes. These REST calls can be mocked using +Angular's `_$httpBackend_` feature, which allows specific REST requests to be +mocked and responses to be returned at a predictable point of time. REST calls are mocked in a `beforeEach` function within the `describe` block. Here's a simple example: @@ -118,10 +142,13 @@ beforeEach(angular.mock.inject(function(_$httpBackend_) { ``` `angular.mock.inject` is used to provide access to the `_$httpBackend_` service. -The `when` method defines a route to intercept, and the `respond` method is used to specify the object to be returned when that route is accessed using the specified HTTP verb. +The `when` method defines a route to intercept, and the `respond` method is used +to specify the object to be returned when that route is accessed using the +specified HTTP verb. -After mocking the route in a `beforeEach` function, each test that calls one of the mocked routes needs to flush the pending requests to ensure that the tests receive a response before the test method ends. -For example: +After mocking the route in a `beforeEach` function, each test that calls one of +the mocked routes needs to flush the pending requests to ensure that the tests +receive a response before the test method ends. For example: ```js it('should be able to fetch a specific file', inject(function(_$httpBackend_, File) { @@ -132,17 +159,21 @@ it('should be able to fetch a specific file', inject(function(_$httpBackend_, Fi })); ``` -Because the result from calling `File.get` is a *promise*, it's only executed when a response is received from the server - so it probably won't actually be executed before the test completes. -Calling `_$httpBackend_.flush()` ensures that the mocked response is immediately returned, and the promise returned by `File.get` immediately resolves. - +Because the result from calling `File.get` is a *promise*, it's only executed +when a response is received from the server - so it probably won't actually be +executed before the test completes. Calling `_$httpBackend_.flush()` ensures +that the mocked response is immediately returned, and the promise returned by +`File.get` immediately resolves. -Using Babel ------------ +## Using Babel -All of the source code in this project is processed through [Babel](https://babeljs.io), which allows code to be written using ES2015, the latest version of the JS standard which isn't yet fully supported by web browsers. -When a bundle is written, Babel transpiles the code into browser-compatible JavaScript. +All of the source code in this project is processed through [Babel], which +allows code to be written using ES2015, the latest version of the JS standard +which isn't yet fully supported by web browsers. When a bundle is written, Babel +transpiles the code into browser-compatible JavaScript. -Here's a quick overview of new ES2015 features and how they're used in this project; a more complete guide to new features can be found [in the Babel documentation](https://babeljs.io/docs/learn-es2015/). +Here's a quick overview of new ES2015 features and how they're used in this +project; a more complete guide to new features can be found [in the Babel documentation]. ### Arrow functions @@ -167,8 +198,8 @@ This form has two main advantages: #### Callback arguments -Arrow functions can only be used as anonymous function expressions, and look cleaner when passed as a literal argument to a function. -Compare: +Arrow functions can only be used as anonymous function expressions, and look +cleaner when passed as a literal argument to a function. Compare: ```js [1, 2].forEach(function(element) { @@ -180,7 +211,8 @@ Compare: }) ``` -With one-line arrow functions, it's also possible to omit the braces; in this case there's implicit return and the value of the statement is always returned. +With one-line arrow functions, it's also possible to omit the braces; in this +case there's implicit return and the value of the statement is always returned. This is useful to write concise `map` statements, for example: ```js @@ -191,8 +223,10 @@ This is useful to write concise `map` statements, for example: Arrow functions inherit `this` from the calling scope. -Traditional JS functions have their own scope for `this`; in strict mode `this` is undefined for unbound functions, and in non-strict mode it refers to the global object. -This makes using function expressions as callbacks in an OO context really confusing, and is why this has been a common idiom: +Traditional JS functions have their own scope for `this`; in strict mode `this` +is undefined for unbound functions, and in non-strict mode it refers to the +global object. This makes using function expressions as callbacks in an OO +context really confusing, and is why this has been a common idiom: ```js var self = this; @@ -201,7 +235,8 @@ Transfer.get().then(function(response) { }); ``` -Arrow functions don't define their own `this`; they inherit it from the parent scope. Here's an example from the new transfer browser: +Arrow functions don't define their own `this`; they inherit it from the parent +scope. Here's an example from the new transfer browser: ```js this.source_location_browser.list().then(locations => { @@ -218,13 +253,17 @@ this.source_location_browser.list().then(locations => { In order to avoid confusion, use the following rule: -* Functions which are bound to a name via the `function name()` syntax remain as traditional functions. -* Functions which are assigned to variables or which are passed as literals to function calls are written as arrow functions, even if `this` is not used. +* Functions which are bound to a name via the `function name()` syntax remain as + traditional functions. +* Functions which are assigned to variables or which are passed as literals to + function calls are written as arrow functions, even if `this` is not used. ### `let` variable definition -JavaScript's `var` statement isn't block-scoped; instead it's scoped to the enclosing function, regardless of location. -This means that it's possible (by accident) to override an existing variable in a way you might not expect to work: +JavaScript's `var` statement isn't block-scoped; instead it's scoped to the +enclosing function, regardless of location. This means that it's possible +(by accident) to override an existing variable in a way you might not expect to +work: ```js var i = 255; @@ -234,8 +273,8 @@ for (var i in [1, 2, 3]) { i // => is now 3 ``` -For backwards compatibility, `var`'s scoping remains the same, but a new variable definition statement has been added: `let`. -`let` is block-scoped: +For backwards compatibility, `var`'s scoping remains the same, but a new +variable definition statement has been added: `let`. `let` is block-scoped: ```js var i = 255; @@ -256,27 +295,35 @@ As a result, including values in strings tends to look pretty verbose: var message = 'Oh no! Something terrible happened with file "' + file.title + '"!'; ``` -ES2015 adds a new string type, the [template string](https://babeljs.io/docs/learn-es2015/#template-strings); this string type uses backticks instead of quotes, and supports string interpolation using the bash-like `${}` syntax: +ES2015 adds a new string type, the [template string]; this string type uses +backticks instead of quotes, and supports string interpolation using the +bash-like `${}` syntax: ```js let message = `Oh no! Something terrible happened with file "${file.title}"`; ``` -Prefer using template strings over catting multiple strings together, unless the latter is shorter or easier to read. +Prefer using template strings over catting multiple strings together, unless the +latter is shorter or easier to read. ### Classes -ES6 provides support for classes with constructors, alongside existing prototype-based objects. -It turns out to be quite easy to define Angular services and controllers as objects, and this makes some stuff quite a bit cleaner. -For example, instance methods on controller objects can be easily referenced from templates, which obviates the need to assign functions to properties. +ES6 provides support for classes with constructors, alongside existing +prototype-based objects. It turns out to be quite easy to define Angular +services and controllers as objects, and this makes some stuff quite a bit +cleaner. For example, instance methods on controller objects can be easily +referenced from templates, which obviates the need to assign functions to +properties. -Here's a good guide that covers over how this works: http://angular-tips.com/blog/2015/06/using-angular-1-dot-x-with-es6-and-webpack/ +Here's a good guide that covers over how this works: ### Modules ES6 provides support for real modules with their own separate namespaces. -They're somewhat similar to Python modules, except that values are exported explicitly rather than every defined name being automatically importable. -For example, instead of the `angular` name being globally available, we need to obtain a reference to it by importing it: +They're somewhat similar to Python modules, except that values are exported +explicitly rather than every defined name being automatically importable. +For example, instead of the `angular` name being globally available, we need to +obtain a reference to it by importing it: ```js import angular from 'angular'; @@ -284,35 +331,56 @@ import angular from 'angular'; This has a few implications: -* Write every file as a module, and export appropriate values if there's something to export. This behaves a bit strangely in the context of Angular modules (since Angular has its own module system which defines modules via side effects of `angular.module` calls), but works very well for things like helper functions and classes. -* Within modules, it's not necessary to worry about polluting the global object: variables are no longer being magically attached to `window`. This means IIFEs are unnecessary. -* Make sure that all dependencies get imported in the places they're used, instead of relying on global `