Support user's console_preferences storage

[nicholaspcr]

Summary

References #6483

Changes

• Add console_preferences migration
• Update user_registry to support new user field console_preferences
• Add tests to user_registry checking the updates to the new field

Testing

Unit tests and manual testing, more details on test steps below.

Test Steps

Create an user and perform the requests listed below. The expected response is attached to each request example.

# Update sort_by field
 curl -X PUT "http://localhost:1885/api/v3/users/<user_id>" \
        -H "Accept: application/json" \
        -H "Authorization: Bearer <token>" \
  -d '{ "user": { "console_preferences": {"console_theme": "DASHBOARD_LAYOUT_GRID"}}, "field_mask": { "paths": [ "console_preferences.console_theme" ] } }'
## Response
{
  "ids": {
    "user_id": "admin"
  },
  "created_at": "2024-02-26T16:07:11.726461Z",
  "updated_at": "2024-02-26T16:12:59.050315Z",
  "console_preferences": {
    "console_theme": "CONSOLE_THEME_LIGHT"
  }
}

# Update dashboard_layouts field
 curl -X PUT "http://localhost:1885/api/v3/users/<user_id>" \
        -H "Accept: application/json" \
        -H "Authorization: Bearer <token>" \
  -d '{ "user": { "console_preferences": {"dashboard_layouts": {"application": "DASHBOARD_LAYOUT_LIST"}}}, "field_mask": { "paths": [ "console_preferences.dashboard_layouts" ] } }'
## Response
{
  "ids": {
    "user_id": "admin"
  },
  "created_at": "2024-02-26T16:07:11.726461Z",
  "updated_at": "2024-02-26T16:10:13.265911Z",
  "console_preferences": {
    "dashboard_layouts": {
      "application": "DASHBOARD_LAYOUT_LIST"
    }
  }
}

# Update sort_by field
 curl -X PUT "http://localhost:1885/api/v3/users/<user_id>" \
        -H "Accept: application/json" \
        -H "Authorization: Bearer <token>" \
  -d '{ "user": { "console_preferences": {"sort_by": {"application": "name"}}}, "field_mask": { "paths": [ "console_preferences.sort_by" ] } }'
## Response
{
  "ids": {
    "user_id": "admin"
  },
  "created_at": "2024-02-26T16:07:11.726461Z",
  "updated_at": "2024-02-26T16:14:34.919360Z",
  "console_preferences": {
    "sort_by": {
      "application": "name"
    }
  }
}

## GET All console_preferences
curl -i -X GET "http://localhost:1885/api/v3/users/<user_id>?field_mask=name,console_preferences" \
        -H "Accept: application/json" \
        -H "Authorization: Bearer <token>"

# Response
{
  "ids": {
    "user_id": "admin"
  },
  "created_at": "2024-02-26T16:07:11.726461Z",
  "updated_at": "2024-02-26T16:14:34.919360Z",
  "console_preferences": {
    "console_theme": "CONSOLE_THEME_LIGHT",
    "dashboard_layouts": {
      "application": "DASHBOARD_LAYOUT_LIST"
    },
    "sort_by": {
      "application": "name"
    }
  }
}

Regressions

This is the addition of a new field on users table, it should have no impact on the existing features.

Notes for Reviewers

Changed this PR just to avoid having both features (console_preferences and bookmarks in the same PR), the reason for this is just to make it easier if the front-end team wants to check how the specific feature was introduced by inspecting this PR.

Checklist

• Scope: The referenced issue is addressed, there are no unrelated changes.
• Compatibility: The changes are backwards compatible with existing API, storage, configuration and CLI, according to the compatibility commitments in README.md for the chosen target branch.
• Documentation: Relevant documentation is added or updated.
• The steps/process to test this feature are clearly explained including testing for regressions.
• Changelog: Significant features, behavior changes, deprecations and fixes are added to CHANGELOG.md.
• Commits: Commit messages follow guidelines in CONTRIBUTING.md, there are no fixup commits left.

[KrishnaIyer]

@nicholaspcr: Please be aware that this PR should target the v3.30 branch. We'll create one soon.

[nicholaspcr]

https://github.com/TheThingsIndustries/lorawan-stack/issues/4009 relates to the comment that I left explaining why I left the end devices entity out. Meaning that after the mentioned issue is finished we could also include just the device_id in the user's bookmarks.

Previous comment disregarded as is not true due to the problem of app1.dev1 and app2.dev2 being possible, so its not possible to have the bookmark of a device without having the parent entity stored as well.

[nicholaspcr]

@nicholaspcr: Please be aware that this PR should target the v3.30 branch. We'll create one soon.

@KrishnaIyer

Notes for Reviewers
Important notes:
This will be merged on v3.30, ignore the target branch while its a draft PR.

[nicholaspcr]

Instead of even storing the dashboard_layouts and sort_order as a map<string,string>, it would be more efficient space wise to have each necessary field defined in the UserConsolePreferences. Something like this:

// UserConsolePreferences is the message that defines the user preferences for the Console.
message UserConsolePreferences {
  option (thethings.flags.message) = {
    select: true,
    set: true
  };
  bool dark_mode = 1;
  message DashboardLayouts {
    string ApplicationLayout = 1;
    string GatewayLayout = 2;
    string OrganizationLayout = 3;
  }
  DashboardLayouts dashboard_layouts = 2;
  message SortOrders {
    string ApplicationFieldOrder = 1;
    string GatewayFieldOrder = 2;
    string OrganizationFieldOrder = 3;
  }
  SortOrders sort_orders = 3;
}

{

  "dark_mode": true,
  "dashboard_layouts": {
    "application_layout": "name",
    "gateway_layout": "name",
    "organization_layout": "name"
  },
  "sort_orders": {
    "application_field_order": "name",
    "gateway_field_order": "name",
    "organization_field_order": "name"
  }
}

I believe this doesn't change much of the implementation on the front-end side of things but it makes the process of adding a new field a bit more extensive as you need to manually add each new key to the struct definition. The benefit is that on the DB side of things we wouldn't be storing the keys to the map for each user.

Additionally we can apply validation of an specific entity on the specific field, ensuring the something like application_field_order only has valid application field values before even applying the change.

@kschiffer with this in mind, the only thing missing is what are the initial fields that you would like to have for the ConsolePreferences.

[kschiffer]

This looks mainly good to me. Some annotations:

• We can skip organization_layout as the new design of this will likely not have a dashboard/overview
• I realized that dark_mode needs three possible values actually: on, off and auto (for using system preferences)
• What's important for me is that we can make extensions to the preferences relatively easy but from what I see this is currently the case

Regarding the default values: these should be mostly empty where possible. The dark_mode would be off by default.

[kschiffer]

What's left here to merge this?

[adriansmares]

I've left my comments in Enterprise.