-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/181 support updated auth storage node #225
Feature/181 support updated auth storage node #225
Conversation
…node consent message.
Latest commit has all unit tests passing in yarn run tests Network initialization tests Basic vault tests Profile tests Schema tests Storage context tests Storage initialization tests Verida database connectivity tests Verida database tests relating to contexts Verida database tests Verida database registry tests Verida database tests Verida datastore tests Verida messaging tests Verida notification tests 55 passing (53s) |
…eparation of auth context and client operations.
…h Context Account type to help with testing.
This is fully tested and working well. Depends on the following PRs before completion:
|
* Feature/181 support updated auth storage node (#225) * Better handling opening a context without an account. Update storage node consent message. * Support CouchDB token auth for public and encrypted databases. * Include token with user permissions * Add more progress * Adding context auth interface * Remove redundant contextAuth * Core authJWT refactor complete. Still need edge cases and fixing some tests. * Fix deprecated substr() reference * Update context name * Support manual injection of contxt name * Move getAuthContext into account. Refactor. * Support caching auth contexts * Refactor auth context to be implemented in account instance. Better separation of auth context and client operations. * Support managing auth contexts and disconnecting devices. Support Auth Context Account type to help with testing. * Refactor to support authentication retry when access / refresh token expires. * Support getAuthContext in vault-account (untested) * Fix typescript issues. * Cache opened databases to avoid opening multiple and creating lots of open sockets. * Support account-web-vault handling invalid access token or refresh token. * Fix error handling with promises * Fix refresh token expiry with account-node * Support deviceId and wallet connect params when logging in. * Validate access and refresh tokens when auto-logging in. * Update HTTP status code checks to match changes in storage-node. * Add INVALID_ENDPOINTS to testing config * Replace deviceId with userAgent * Point to live docs * Feature/210 decentralize did registry (#211) * Example code showing the core changes needed to start using decentralized DID registry. * Add example unit test config change * Moved vda-did, vda-did-resolver, vda-web3-clinet * Minor updates for PR review * Minor update - gitignore for entire project * Update package descriptions * 210 - Working on Progress Current Status : Need to implement codes for updated documents. * 210 - did-client update Updated get/set document via using vda-did, vda-did-resolver, & vda-web3-client Update in this commit: Created helper functions to extract updates from documents Updated savdDocument() function * Minor updated - gitignore * Vda-web3-client Updated Updated on Review * Updates vda-did-resolver & vda on MR review Updated name of vda-did-resolver 'vda-did-resolver' to '@verida/vda-did-resolver' (package.json) Updated dependency for resolver library in vda-did * Minor update - vda-did update package.json dependency for vda-did-resolver * Minor update Need to updte all codes after review. * Test Working - Load DIDDocument success Next : Update DIDDocument & call Save() * Test working Next : Merge vda-did libraries to this branch * Test codes updated : Test done on Polygon mainnet * Minor update : Remove unnecessary files & codes deleted: src/VDADIDRegistry.abi.json * Middle : Issue on build vda-did Status : vda-did-resolver & vda-web3-client compiled successfully Issue: Issue on yarn install inside vda-did. * Update dependencies so build works * Fixed build global `verida-js` * Moved vda-did, vda-did-resolver, vda-web3-clinet * Minor updates for PR review * Minor update - gitignore for entire project * Vda-web3-client Updated Updated on Review * Updates vda-did-resolver & vda on MR review Updated name of vda-did-resolver 'vda-did-resolver' to '@verida/vda-did-resolver' (package.json) Updated dependency for resolver library in vda-did * Update package descriptions * Minor update - vda-did update package.json dependency for vda-did-resolver * Minor update Need to updte all codes after review. * Middle : Issue on build vda-did Status : vda-did-resolver & vda-web3-client compiled successfully Issue: Issue on yarn install inside vda-did. * Update dependencies so build works * Fixed build global `verida-js` * Tested on Polygon TestNet * Example code showing the core changes needed to start using decentralized DID registry. * Add example unit test config change * 210 - Working on Progress Current Status : Need to implement codes for updated documents. * 210 - did-client update Updated get/set document via using vda-did, vda-did-resolver, & vda-web3-client Update in this commit: Created helper functions to extract updates from documents Updated savdDocument() function * Minor updated - gitignore * Test Working - Load DIDDocument success Next : Update DIDDocument & call Save() * Test working Next : Merge vda-did libraries to this branch * Test codes updated : Test done on Polygon mainnet * Minor update : Remove unnecessary files & codes deleted: src/VDADIDRegistry.abi.json * Did-client tested on Mumbai net Minor updates on vda-did-resolver : controller for bulkAdd * Merged feature/218-support-get-set-did-docs DIDDocument updated: Added comparison function by Chris * DID-Document updated - Removed proof Removed proof related stuff from source & test. * DID-Client & DIDDocument Updated DID-Clinet is now using DIDDocument of verida-js library. Before, DID-Client used the DIDDocumnt from ethr-did. * Fix verida package dependencies to use latest * Add back the original unit tests that need to all pass * Feature/231 update configuration vda did libraries (#232) * Update did format for vda-did & vda-did-resolver : "did:vda:..." Work: Updated did format from "did:ethr:..." to "did:vda:..." Status : Tested vda-did & vda-did-resolver after updates Resolved document with updated format Remaining : Check for 'did-client' package Check for *.md files. There are still texts with "did:ethr:...". Need to be updated * Configuration for vda-did libraries updated Work done: Now can create vda-did & vda-did-resolver with chainname "mainnet" & "testnet" Test codes updated. (Removed unnecessary ones & updated existing ones) Test result: Tested on Polygon testnet Missing: Not tested on Mainnet * Update did format for vda-did & vda-did-resolver : "did:vda:..." Work: Updated did format from "did:ethr:..." to "did:vda:..." Status : Tested vda-did & vda-did-resolver after updates Resolved document with updated format Remaining : Check for 'did-client' package Check for *.md files. There are still texts with "did:ethr:...". Need to be updated * Configuration for vda-did libraries updated Work done: Now can create vda-did & vda-did-resolver with chainname "mainnet" & "testnet" Test codes updated. (Removed unnecessary ones & updated existing ones) Test result: Tested on Polygon testnet Missing: Not tested on Mainnet * DID-Client updated * Fix asym and sign keys not being stored separately. * Pull keys from DID document using new ID format. * Feature/231 : Added "sign" or "asym" to attribute value. (#233) * Update did format for vda-did & vda-did-resolver : "did:vda:..." Work: Updated did format from "did:ethr:..." to "did:vda:..." Status : Tested vda-did & vda-did-resolver after updates Resolved document with updated format Remaining : Check for 'did-client' package Check for *.md files. There are still texts with "did:ethr:...". Need to be updated * Configuration for vda-did libraries updated Work done: Now can create vda-did & vda-did-resolver with chainname "mainnet" & "testnet" Test codes updated. (Removed unnecessary ones & updated existing ones) Test result: Tested on Polygon testnet Missing: Not tested on Mainnet * Update did format for vda-did & vda-did-resolver : "did:vda:..." Work: Updated did format from "did:ethr:..." to "did:vda:..." Status : Tested vda-did & vda-did-resolver after updates Resolved document with updated format Remaining : Check for 'did-client' package Check for *.md files. There are still texts with "did:ethr:...". Need to be updated * Configuration for vda-did libraries updated Work done: Now can create vda-did & vda-did-resolver with chainname "mainnet" & "testnet" Test codes updated. (Removed unnecessary ones & updated existing ones) Test result: Tested on Polygon testnet Missing: Not tested on Mainnet * DID-Client updated * Add "sign" or "asym" to attribute value Tested on Mumbai. Updated: resolver.ts test code of vda-did * Update test to show save and get documents do not match * Add better description * DID-Client create same document as vda-did-resolver Updates: New DIDDocument updated to create initial document as vda-did-resolver resolved DIDDocument::addContextSignKey() updated to make same document after sign key added as vda-did-resolver resolved DIDDocument::addContextAsymKey() updated to make same document after asym key added as vda-did-resolver resolved Addded: did-document/helpers.ts for knownetworks constant and other helper function * Minor update : vda-did Test code updated to make clear that signer & vdaKey are differet. These parameters are in the vda-did configuration in direct mode. Signer is used to pay gas fees for blochchain transactions. vdaKey is used to create the signature parameter in the smart contract function call. * 235 - Add publicKeyHex fields to each item in VerificationMethod (#236) * 235 - Add publicKeyHex fields to each item in VerificationMethod Update done. Tested with test code of did-client. Missing : Last test case is being failed. * Updated for publicKey of controller Updated list: did-client did-document vda-did-resolver/src/resolver.ts Test code update: did-client/src/did-client.tests.ts * Feature/210 refactor (#234) - Refactor did-client to simplify it - Get all protocol unit tests working - Handle edge case bugs and issues * Cleanup yarn.lock * Add missing import * Only add rpcUrl to resolver if specified. This ensures default rpcUrl from resolver is used if none is specified. * Use environment variable for rpcUrl * Cleanup use of promises * Move config out of package and into testing. Update to use decentralized did client. Update create credentials tests. * Update share credential tests to work with decentralized did client. * Correctly set wallet DID based on network * Correctly set network in the DID * Add proof to signing verification methods (#243) * Remove deprecated dotenv * Update yarn.lock * Fix typescript issue with proof not part of the DID document definition * Only apply the network to the DID string, if not already set * Refactor storage link (and tests) to use DID document context creation. * Use updated storage link link() method interface (keyring) * Fix vda-did and vda-did-resolver tests * Disable linting on prepublish for forked DID libraries * Lint cleanup for vda-did-resolver * Lint cleanup for vda-did * Fixing missing include * Publish - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] * Remove deprecated `did-resolver1. See `vda-did-resolver`. * Add missing vda-did dependency in did-client * Update build process to generate es5 commonjs libs for maximum compatibility. * Add missing draft07 AJV schema * Add missing proof for unit tests * DID-Client & DID-Document updated DID-Document: Update test code for `proof` field. DID-Client: Update for `proof` filed. * Update test names to match the package name * Make network key optional. Not required when `callType` is "direct". * Throw error if `netoworkPrivateKey` not specified in web3 mode * Update DID document to generate correct context proof, signed by correct private key (#245) * Update protocol to supply private key when adding a context. * Include private key when setting storage link * Code cleanup and type fixing. * Publish - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] * Use generated wallet private key as the supplied key may actually be a mnemonic * Fix protocol so RPC default connection is managed entirely within DID resolver * Add more descriptive comments on the web3 config options. Fix incorrect setting of self transaction private key config. Add better error checking of options. * Build signer object if not specified. * Remove unecessary veridaKey config. * Remove redundant networkPrivateKey config option. Will be specified in web3config. * Remove deprecated default server URLs. Now configured via Account objects. * Use updated client config format * Only authenticate when required to avoid trying to authenticate when only needing to be read only. * Support detecting if authenticated * Use updated protocol config * Improve tests to ensure a context is correctly removed on chain * Publish - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] * Support new attributes and proof management. * Remove need to specify RPC URL. Use default. * Feature/252 support vip 4 (#262) * Delete old "vda-did" packages. Create basic structure for vda-did-resolver. * Fix removed vda-did package breaking yarn. Add xios to vda-did-resolver. * Add private key for signing * Fix broken references to deprexcated vda-did-resolver capabilities * Update dependencies * Basic create and resolve working with tests * Fix package.json dependencies * Large refactor. Move back to vda-did and vda-did-resolver as separate libraries. * Fix vda-did dependencies * Various code cleanup * Code cleanup and better variable names * vda-did-resolver tests working again * Previous basic vda-did tests now working again * Specify lookup response * Better error handling. Implement updates. * Verify updated DID Document resolves the correct version * Implement tests that check one endpoint failing and one successful. Ensure second create fails. Better version checks. * Add basic addEndpoint support * Properly support failed did resolver response * Use consistent `did` instead of `didAddress` as method params * All did-client tests passing * Update unit tests with new error messages * Alex : lookup() implemented Updated: blockchain integration implemented in src/lookup.ts Added test code for lookup.ts * Update lookup test to use assert * Add failed unit tests to show expected response from lookup() * Support specifying network * [vda-did] Support delete DIDs * Feature/253 vda web3 client vip2.0 (#254) * VDA-Web3-Client : DIDRegistry Test working Next: NameRegistry test code update * Feature/63 Update test codes Updated: Test codes for DIDRegistry & NameRegistry Whitelist of contracts in src/constants.ts * Minor update * Vda-Web3-Client Update DIDRegistry lookup function (#256) - Update contract abi - Update contract address deployed - update whitelist of VdaWeb3Client - Update test code for VDADIDRegistry * Support DID Document having multiple service endpoints per service * vda-did-resolver Complete blockchain api (#258) - Update contract address deployed - Update contract abi * VDA-DID update (#260) * VDA-DID update - Implement blockchain api - Update test codes * Minor updates on review Co-authored-by: Chris <[email protected]> * Fixing unit tests. Force lowercase with blockchain activity. * VDA-DID-Registry lookup() update for vda- libraries (#264) Updated : vda-did, vda-did-resolver, vda-web3-client - lookup() function return type updated for vda-did & vda-did-resolver - contract address of VDA-DID-Registry updated - tested on Mumbai * Update package.json for vda-did & vda-did-resolver Updated: - copy contract abi json files * Merge upstream changes * Configure vda-web3 for resolver tests * vda-did-resolver lookup.test.ts Updated * Fix vda-did-resolver tests * All current tests pass * Fix resolver error handling with invalid DID * [vda-did] Code cleanup * [did-client] Get all tests working with updated vda-did and vda-did-resolver * [storage-link] Tests all work with updated dependencies Co-authored-by: Chris <[email protected]> Co-authored-by: Alex <[email protected]> * Feature/257 storage node replication (#261) * Delete old "vda-did" packages. Create basic structure for vda-did-resolver. * Fix removed vda-did package breaking yarn. Add xios to vda-did-resolver. * Add private key for signing * Fix broken references to deprexcated vda-did-resolver capabilities * Update dependencies * Basic create and resolve working with tests * Fix package.json dependencies * Large refactor. Move back to vda-did and vda-did-resolver as separate libraries. * Fix vda-did dependencies * Various code cleanup * Code cleanup and better variable names * vda-did-resolver tests working again * Previous basic vda-did tests now working again * Specify lookup response * Better error handling. Implement updates. * Verify updated DID Document resolves the correct version * Implement tests that check one endpoint failing and one successful. Ensure second create fails. Better version checks. * Add basic addEndpoint support * Properly support failed did resolver response * Use consistent `did` instead of `didAddress` as method params * All did-client tests passing * Update unit tests with new error messages * Alex : lookup() implemented Updated: blockchain integration implemented in src/lookup.ts Added test code for lookup.ts * Update lookup test to use assert * Add failed unit tests to show expected response from lookup() * Support specifying network * [vda-did] Support delete DIDs * Feature/253 vda web3 client vip2.0 (#254) * VDA-Web3-Client : DIDRegistry Test working Next: NameRegistry test code update * Feature/63 Update test codes Updated: Test codes for DIDRegistry & NameRegistry Whitelist of contracts in src/constants.ts * Minor update * Vda-Web3-Client Update DIDRegistry lookup function (#256) - Update contract abi - Update contract address deployed - update whitelist of VdaWeb3Client - Update test code for VDADIDRegistry * Support DID Document having multiple service endpoints per service * Support multiple endpoints per context with multiple services, instead of an array of URI's in a single service. * Switch to use serviceEndpoint as source of truth for multiple endpoints instead of service. Can't use service as it will create `id` clashes. * Update to latest did-resolver * Cleanup handling of multiple service endpoints * Use DID Document ServiceEndpoint interfaces. Support array of default did endpoints in DIDClientConfig. * Large refactor (untested) to handle multiple database connection endpoints for a given database in the SDK. * Get client-ts to build. * Storage link unit tests pass * Get storage context tests working * Fix account node issues * Auth tests working * Fix access to public databases * Improve error messages when rpcUrl is missing * Increase timeout so tests pass with new blockchain code * Fixes found via unit tests * Fix incorrect test logic * Increase timeout so tests pass * Support logging of performance stats as a config option * Fix notification server to not break with multiple endpoints * Fix notification tests * Support log perofrmance in config options * Fix rpcUrl config options * Add storage endpoint test. Expose new storage node endpoints to database instances. * Output private keys for easy reference * Endpoint fixes from unit tests * Two passing unit tests * By default log performance with unit tests * Complete storage node unit tests Co-authored-by: Chris <[email protected]> Co-authored-by: Alex <[email protected]> * Add comment on how to tag releases * Fix build issues as a result of changed account interface * Fix verifiable credential build issues * Publish - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] * Feature/169 ensure deterministic sigs (#269) * Ensure deterministic signatures of JSON objects. * Add missing dependency in yarn.lock from previous commit Co-authored-by: Chris <[email protected]> * Don't use solidityPack when generating the proofString * SUPPORT locateContextProof * Vda-web3-client Gas configuration updated (#267) * Vda-web3-client Gas configuration updated Tested - for each gas configuration type * VDA-Web3-Client Added manual gas configuration Updated Use one of params, methodDefaults, or global configuration Test code updated to see the manual gas configuration * Updates on review : Moved gas configuration code Updated - Moved gas configuration code from `callMethod()` to constructor in VeridaContractBase.ts - Added gas configuration test code to `test/didregistry.test.ts` * Track endpoint errors on DID creation * Only add success endpoints to the blockchain * Feature/272 support vip6 (#273) * Initial implementation (untested, but builds). Debug info included for now. * Fix await on setting database to public * Call createDb on all endpoints. * Better handling of blockchain write errors * Migrate sleep to utils * Save nonce for easy debug output when required * Fix bug where every couch request would fail on first attempt. * Support closing databases and contexts * Fix all tests to correctly close contexts so they don'thang * Support closing external databases. Fix issues if database already closed. * Run all notification tests * Randomly select public endpoint URI. Include endpoint in database info. * Add sleep and timeouts to ensure replication has time to complete * Update registry tests databaes names * Skip notification tests. Improve profile tests timeout to ensure replication can complete. * Include sleep to ensure replication has time to complete. Skip tests for now. * Call connect account on all endpoints at once, to increase performance when opening a context. * Implement performance tests * Add helpful info on what takes the time * Run all performance tests * Add commented out performance timers * [vda-did] Make create fail or succeed, not partially save to some endpoints. Increase async for better performance. * Fix tests to support multiple endpoints * Minor config and test tweaks * Clean up debug logging * Add incomplete storage endpoint tests * Add AU based endpoints for testing * Add AU endpoints for testing * Tweaks to tests * Publish - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] * Ensure port is included in standardized URLs * Support remote HTTP request timeouts (default 10 seconds) * Add details on how to tag releases * Finish replacing dsn with endpoint. Add support for fetching status of a context. * Add name and keys to ContextDatabaseStatus * Rename ContextStatus to ContextInfo to be more consistent with DatabaseInfo * Specify the active endpoint in context info * Remove redundant endpoint replication service * Ensure context listens to and re-emits relevant endpoint error / warning events. * Publish - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] * Modify getAuthContext() so it can be used by the account-web-vault * Fix account-web-vault to correctly use endpointUri supplied in config * Publish - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] - @verida/[email protected] * Fixes for SSO to work with new replication * updateDatabases now triggers on all endpoints, not just the primary. * Fixes for tests. Remove debug stuff. * Remove debugging * Update CHANGELOG files in prep for v2.0.0 release * [client-ts] Correct CHANGELOG error * Update devnet endpoints for testing * Support per environment config * Delete owner databases implemented and working * Support deleting databases * Support clearLocal param on database and context closing that will clear local pouch data * Update unit tests to clear local context data on completion. Fix external database caching issue. * Update info on why storage endpoint tests are skipped by default * Add port to endpoints in test config so tests pass. * Update performance test comments * Destroy database is sync status is complete * Support closing datastores * Fixing tests to not hange, except for one...
Closes security issue: https://github.com/verida/security-issues/issues/1
Implements changes required to support talking to updated storage node: verida/storage-node#14