Fix/13 improve auth security (#14)

* Support running a single test or multiple. Add JWT dependency.
* Implement JWT authorization. Use a new consent message for signature verification.
* Update permission tests to support JWT auth.
* Implement JWT tests
* Add README instructions for enabling JWT auth in CouchDb
* Fix incorrect merge issues.
* Add support for generating and verifying refresh token
* More refresh token unit tests. Add access token support. Db refactor.
* Fix permission unit tests with new refactor.
* Update server endpoints to use new auth. Create server integration tests for auth.
* Hash database name server side before creating
* Verify context name when verifying refresh token. Don't generate access token if refresh token is invalid.
* Confirm expected user is admin of database before deleting it.
* Ensure database names begin with "v"
* Return an access token and host when requesting a refresh token.
* Support deleteDatabase endpoint
* Expand unit tests
* Split controller into auth and user.
* Fix broken test
* Fix broken delete database calls
* Remove JWT test no longer required
* Support invalidating device ID
* Invalidating devices now working correctly with tests
* Implement garbage collection
* Fix building DSN for public credentials
* Add CouchDB configuration note around basic auth
* Add more docs
* Fix edge case issues identified during testing
* Fix yarn dependencies
* Fix example .env files and remove duplicate.
* Describe how authentication works
* Rename to authorization
* Add missing account-node dev dependency
* Fix minor issues with tests. Better config docs in README.
* Resolve feedback for further review.
* Fix missed merge issue
* Support running single test
* Add support for saving user databases and getting the info as an authorized user. More HTTP status improvements and better checking for valid request params.
* Add missing require for PouchDB
* Cleanup didsToUsernames
* Destructure some vars
* Add cors requirement to README
* Add isTokenValid endpoint to verify a refresh token is still valid and obtain the expiry.

README.md

@@ -10,6 +10,23 @@ Key features:
 - Adding a second layer of security by managing per-database ACL validation rules
 - Providing applications with user's database connection strings
 
+## How Authorization Works
+
+This is the login flow:
+
+1. The Verida Account makes a request to the storage node API to obtain an auth JWT to be signed (`/auth/generateAuthJwt`). This prevents replay attacks.
+2. The Verida Account signs a consent message using their private key. This consent message proves the user wants to unlock a specific application context
+3. The Verida Account submits the signed authorization request (`/auth/authenticate`). Assuming the signed AuthJWT is valid, the storage node returns a refresh token and an access token
+4. The Verida Account can then use the access token to either; 1) make storage node requests (ie: create database) or 2) directly access CouchDB as an authenticated user (using `Bearer` token auth)
+5. When the access token expires, the Verida Account can use the refresh token to request a new access token (`/auth/connect`)
+6. If a refresh token is close to expiry, the Verida Account can use the active refresh token to obtain a new refresh token (`/auth/regenerateRefreshToken`)
+
+When a Verida Account authenticates, it can designate an `authenticate` requst to be linked to a particular device by specifying the `deviceId` in the request.
+
+This allows a specific device to be linked to a refresh token. A call to `/auth/invalidateDeviceId` can be used to invalidate any refresh tokens linked to the specified `deviceId`. This allows the Verida Vault to remotely log out an application that previously logged in.
+
+Note: This only invalidates the refresh token. The access token will remain valid until it expires. It's for this reason that access tokens are configured to have a short expiry (5 minutes by default). CouchDB does not support manually invalidating access tokens, so we have to take this timeout approach to invalidation.
+
 ## Usage
 
 ```bash
@@ -40,6 +57,10 @@ A `sample.env` is included. Copy this to `.env` and update the configuration:
 - `DB_REJECT_UNAUTHORIZED_SSL`: Boolean indicating if unauthorized SSL certificates should be rejected (`true` or `false`). Defaults to `false` for development testing. Must be `true` for production environments otherwise SSL certificates won't be verified.
 - `DB_PUBLIC_USER`: Alphanumeric string for a public database user. These credentials can be requested by anyone and provide access to all databases where the permissions have been set to `public`.
 - `DB_PUBLIC_PASS`: Alphanumeric string for a public database password.
+- `ACCESS_TOKEN_EXPIRY`: Number of seconds before an access token expires. The protocol will use the refresh token to obtain a new access token. CouchDB does not support a way to force the expiry of an issued token, so the access token expiry should always be set to 5 minutes (300)
+- `REFRESH_TOKEN_EXPIRY`: Number of seconds before a refresh token expires. Users will be forced to re-login once this time limit is reached. This should be set to 7 days (604800).
+- `ACCESS_JWT_SIGN_PK`: The access token private key. The base64 version of this must be specified in the CouchDB configuration under `jwt_keys/hmac:_default`
+- `REFRESH_JWT_SIGN_PK`: The refresh token private key
 
 ### Setting up environment variables on Windows
 
@@ -63,21 +84,58 @@ $env:DB_PUBLIC_PASS="784c2n780c9cn0789"
 - CORS must be enabled so that database requests can come from any domain name
 - A valid user must be enforced for security reasons
 
+[Ensure `{chttpd_auth, jwt_authentication_handler}` is added to the list of the active `chttpd/authentication_handlers`](https://docs.couchdb.org/en/stable/api/server/authn.html?highlight=jwt#jwt-authentication)
+
+DO NOT include ` {chttpd_auth, default_authentication_handler}` in the authentication handlers. This option is a default enable in CouchDB and causes web browsers to display a HTTP Basic Auth popup if authentication fails. This creates an awful UX and is unecessary as the protocol handles authentication issues automatically.
+
+Learn more here: https://stackoverflow.com/questions/32670580/prevent-authentication-popup-401-with-couchdb-pouchdb
+
 ```
-[httpd]
-WWW-Authenticate = Basic realm="administrator"
-enable_cors = true
+[chttpd]
+authentication_handlers = {chttpd_auth, jwt_authentication_handler}, {chttpd_auth, cookie_authentication_handler}
+enable_cors = false
 
 [chttpd_auth]
 require_valid_user = true
 
+[jwt_auth]
+required_claims = exp
+
+[jwt_keys]
+hmac:_default = <base64 secret key>
+
 [cors]
 origins = *
 credentials = true
 methods = GET, PUT, POST, HEAD, DELETE
 headers = accept, authorization, content-type, origin, referer, x-csrf-token
 ```
 
+The `hmac:_default` key is a base64 encoded representation of the 
+
+## Generating JWT key
+
+Note: A secret key (string) suitable for `jwt_keys` can be base64 encoded with the following:
+
+```
+const secretKey = 'secretKey'
+const encodedKey = Buffer.from(secretKey).toString('base64')
+```
+
+This can be tested via curl:
+
+```
+curl -H "Host: localhost:5984" \
+ -H "accept: application/json, text/plain, */*" \
+ -H "authorization: Bearer <bearer_token>" \
+  "http://localhost:5984/_session"
+```
+
+Where:
+
+- `bearer_token` - A bearer token generated via the `test/jwt` unit test
+- `localhost` - Replace this with the hostname of the server being tested
+
 ## Lambda deployment
 
 We use [Claudia.js](https://claudiajs.com/) to turn our Express app into an Express-on-Lambda app.
@@ -87,3 +145,13 @@ See the [Claudia Docs for information](https://claudiajs.com/news/2016/11/24/cla
 
 Verida staff can see the [internal Verida repo]( https://github.com/verida/infrastructure/blob/develop/storage_node.md) for docs on this. 
 
+## Tests
+
+Run tests with `yarn run tests`
+
+Note: The tests in `server.js` require the server to be running locally. The other tests operate fine without the server running.
+
+Common issues when running tests:
+
+1. `Bad key`: The key in CouchDB configuration for `jwt_keys/hmac:_default` is not a valid Base64 encoded key
+2. `HMAC error`: The key in CouchDB configuration for `jwt_keys/hmac:_default` does not match `ACCESS_JWT_SIGN_PK` in `.env`

package.json

@@ -5,7 +5,8 @@
   "main": "dist/server.js",
   "scripts": {
     "clean": "rm -rf dist",
-    "test": "mocha -mocha --require @babel/polyfill --require @babel/register './test/**/*.js' --timeout 30000",
+    "tests": "mocha -mocha --require @babel/polyfill --require @babel/register './test/**/*.js' --timeout 30000",
+    "test": "mocha -mocha --require @babel/polyfill --require @babel/register --timeout 30000",
     "dev": "nodemon --exec babel-node src/server.js",
     "build": "yarn run clean && babel src -d dist --extensions .js",
     "serve": "node dist/server.js",
@@ -52,17 +53,20 @@
   "homepage": "https://github.com/verida/storage-node/README.md",
   "dependencies": {
     "@babel/runtime": "^7.16.7",
-    "@verida/did-client": "^0.1.2",
+    "@verida/did-client": "^0.1.8",
+    "@verida/encryption-utils": "^1.1.3",
     "aws-serverless-express": "^3.4.0",
     "cors": "^2.8.5",
     "did-resolver": "^3.1.0",
     "dotenv": "^8.2.0",
     "ethers": "^4.0.42",
     "express": "^4.17.1",
     "express-basic-auth": "git+https://github.com/Mozzler/express-basic-auth.git",
+    "jsonwebtoken": "^8.5.1",
     "lodash": "^4.17.21",
     "memory-cache": "^0.2.0",
-    "nano": "^8.1.0"
+    "nano": "^8.1.0",
+    "rand-token": "^1.0.1"
   },
   "devDependencies": {
     "@babel/cli": "^7.15.7",
@@ -71,6 +75,8 @@
     "@babel/plugin-transform-runtime": "^7.16.7",
     "@babel/polyfill": "^7.8.0",
     "@babel/preset-env": "^7.7.7",
+    "@verida/account-node": "^1.1.9",
+    "axios": "^0.27.2",
     "babel-core": "^6.26.3",
     "babel-loader": "^8.2.3",
     "babel-polyfill": "^6.26.0",

sample.env

@@ -6,8 +6,20 @@ DB_USER="admin"
 DB_PASS="admin"
 DB_HOST="localhost"
 DB_PORT=5984
-DB_REJECT_UNAUTHORIZED_SSL=false
+DB_REJECT_UNAUTHORIZED_SSL=true
+ACCESS_JWT_SIGN_PK="insert-random-access-symmetric-key"
+# 5 Minutes
+ACCESS_TOKEN_EXPIRY=300
+REFRESH_JWT_SIGN_PK="insert-random-refresh-symmetric-key"
+# 30 Days
+REFRESH_TOKEN_EXPIRY=2592000
+DB_REFRESH_TOKENS="verida_refresh_tokens"
+DB_DB_INFO="verida_db_info"
+# How often garbage collection runs (1=100%, 0.5 = 50%)
+GC_PERCENT=0.1
 
 // alpha numeric only
 DB_PUBLIC_USER="784c2n780c9cn0789"
 DB_PUBLIC_PASS="784c2n780c9cn0789"
+
+PORT=5151