Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix account device to MQTT Team device naming, clean up terminology, improve logging and docs, etc #7

Merged
merged 9 commits into from
Jan 30, 2025
39 changes: 24 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# mqtt-bridge-mosquitto
Bridges messages sent within an nrfcloud account to an AWS IoT broker.
Uses an account device to subscribe to messages on the nrfcloud side.
Bridges messages sent within an nRF Cloud account to an AWS IoT broker.
Uses an [MQTT Team Device](https://docs.nordicsemi.com/bundle/nrf-cloud/page/Devices/Properties/Types.html#mqtt-team-devices) to subscribe to messages on the nRF Cloud side.
Republishes messages from `{stage}/{tenantId}/#` => `data/#` in the
local AWS IoT message broker.

Expand All @@ -14,27 +14,36 @@ local AWS IoT message broker.

**Setup steps**
1. Install dependencies
* `yarn install` or `npm install`
* `yarn install` or `npm install`
2. Compile package
* `yarn compile`
3. Initialize context. This creates account device credentials, pulls
nrfcloud account info, creates certificates for the local aws iot broker,
and saves the resulting keys to ssm parameters.
* `yarn bridge-init <nrfcloud api key> -e <nrfcloud endpoint>`
* `<nrfcloud endpoint>` defaults to https://api.nrfcloud.com
4. Deploy the application
* You may first need to run cdk bootstrap with your aws account info
* `yarn cdk bootstrap`
3. Initialize context. This pulls nRF Cloud account info, creates an MQTT Team Device with credentials,
creates a certificate for the local AWS IoT broker, and saves the resulting keys to AWS SSM parameters.
If SSM parameters already exist for the Team device and local certificate, they are not re-created
unless `--reset` is supplied.
* `yarn bridge-init <nRF Cloud API key> [-e <nRF Cloud endpoint>] [--reset] `
* `<nRF Cloud endpoint>` defaults to https://api.nrfcloud.com
* `--reset` will create a new Team Device and local certificate even if they are
already recorded in AWS SSM. Any pre-existing Team Devices are not deleted.
4. You may need to bootstrap the CDK if you have never deployed CDK resources before.
Ensure your AWS profile info is defined in your environment first.
* `yarn cdk bootstrap`

## Bridge Stack
Deploy the CDK application, which creates an AWS CloudFormation stack named `nrfcloud-mqtt-bridge`
* `yarn cdk deploy`

## Demo Stack
In addition the bridge stack, this repo also includes a demo stack that gives a good example
of what you could use a bridge for. The demo stores data using Iot rules that persist
In addition to the bridge stack, this repo also includes a demo stack that gives a
good example of a use case for the bridge. The demo stores data using IoT rules that persist
data to Timestream, and starts up a grafana instance for visualizing the data.

**Setup steps**
1. Setup bridge stack using the steps above
2. Deploy the demo stack
2. Deploy the demo CloudFormation stack named `nrfcloud-mqtt-bridge-demo`
* `yarn deploy-demo`
3. Connect some devices to nrfcloud or use the device simulator to start
3. Connect some devices to nRF Cloud or use the device simulator to start
sending data into your account
4. Go to the URL of the bridge demo dashboard, which can be found in the `grafanaendpoint` key of the
`Outputs` section of the `nrfcloud-mqtt-bridge-demo` CloudFormation stack. Use the initial user/password of
`admin/admin` to log in to Grafana. It will prompt you to change your pasword.
12 changes: 11 additions & 1 deletion container/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,3 +1,13 @@
FROM eclipse-mosquitto:2.0.11

COPY scripts/docker-entrypoint.sh /
# Copy the custom entrypoint script into the container
COPY scripts/docker-entrypoint.sh /docker-entrypoint.sh

# Ensure the entrypoint script has executable permissions
RUN chmod +x /docker-entrypoint.sh

# Set the custom entrypoint
ENTRYPOINT ["/docker-entrypoint.sh"]

# Default command to run Mosquitto in verbose mode
CMD ["mosquitto", "-v"]
3 changes: 3 additions & 0 deletions container/scripts/docker-entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ if [ "$user" = '0' ]; then
[ -d "/mosquitto" ] && chown -R mosquitto:mosquitto /mosquitto || true
fi

echo "Mosquitto configuration file:"
cat /mosquitto/config/mosquitto.conf

echo "$@"

exec "$@"
6 changes: 3 additions & 3 deletions src/DemoStack.ts
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ export class DemoStack extends Stack {
from 'data/m/d/+/d2c'
where appId='TEMP'`,
ruleDisabled: false,
description: 'Saves asset tracker temp data into timestream',
description: 'Saves asset tracker temp data into Timestream.',
actions: [
{
timestream: {
Expand All @@ -120,7 +120,7 @@ export class DemoStack extends Stack {
from 'data/m/d/+/d2c'
where (appId='GPS' or appId='GNSS') and exists (data.lng)`,
ruleDisabled: false,
description: 'Parses asset tracker gps strings into lat,lon pairs then stores them in timestream',
description: 'Parses asset tracker GPS strings into lat,lon pairs then stores them in Timestream.',
actions: [
{
timestream: {
Expand Down Expand Up @@ -163,7 +163,7 @@ export class DemoStack extends Stack {

`,
ruleDisabled: false,
description: 'Parses asset tracker gps strings into lat,lon pairs then stores them in timestream',
description: 'Parses asset tracker GPS strings into lat,lon pairs then stores them in Timestream.',
actions: [
{
timestream: {
Expand Down
87 changes: 49 additions & 38 deletions src/MqttBridgeStack.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import {App, Duration, Stack} from 'aws-cdk-lib';
import {App, Stack} from 'aws-cdk-lib';
import {Cluster, ContainerImage, FargateService, FargateTaskDefinition, LogDriver, Secret} from 'aws-cdk-lib/aws-ecs'
import {InstanceClass, InstanceSize, InstanceType, NatProvider, Vpc} from "aws-cdk-lib/aws-ec2"
import {StringParameter} from "aws-cdk-lib/aws-ssm"
Expand All @@ -7,14 +7,15 @@ import ajv = require("ajv");
const Ajv = new ajv.default();

interface Config {
mqttEndpoint: string;
nrfCloudMqttEndpoint: string;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-ordering of parameters throughout to make them consistent, can cause some readability issues. E.g. I didn't change mqttEndpoint to nrfCloudMqttEndpoint, I moved nrfCloudMqttEndpoint up here in the list with the other nrfcloud params, and moved mqttEndpoint down with the other local params and renamed it localMqttEndpoint for clarity.

mqttTopicPrefix: string;
accountDeviceClientId: string;
accountDeviceCertSSMParam: string;
accountDeviceKeySSMParam: string;
iotCertSSMParam: string;
iotKeySSMParam: string;
nrfcloudMqttEndpoint: string;
mqttTeamDeviceClientId: string;
mqttTeamDeviceClientIdSSMParam: string;
mqttTeamDeviceCertSSMParam: string;
mqttTeamDeviceKeySSMParam: string;
localMqttEndpoint: string;
localIotClientCertSSMParam: string;
localIotClientKeySSMParam: string;
}

export class MqttBridgeStack extends Stack {
Expand All @@ -23,11 +24,12 @@ export class MqttBridgeStack extends Stack {

const config = this.getConfig();

const accountDeviceClientCertSSMParam = StringParameter.fromStringParameterName(this, "AccountDeviceClientCertSSMParamValue", config.accountDeviceCertSSMParam)
const accountDeviceClientKeySSMParam = StringParameter.fromStringParameterName(this, "AccountDeviceClientKeySSMParamValue", config.accountDeviceKeySSMParam)
const mqttTeamDeviceClientIdSSMParam = StringParameter.fromStringParameterName(this, "MqttTeamDeviceClientIdSSMParamValue", config.mqttTeamDeviceClientIdSSMParam)
const mqttTeamDeviceClientCertSSMParam = StringParameter.fromStringParameterName(this, "MqttTeamDeviceClientCertSSMParamValue", config.mqttTeamDeviceCertSSMParam)
const mqttTeamDeviceClientKeySSMParam = StringParameter.fromStringParameterName(this, "MqttTeamDeviceClientKeySSMParamValue", config.mqttTeamDeviceKeySSMParam)

const iotKeySSMParam = StringParameter.fromStringParameterName(this, "IotKeySSMParamValue", config.iotKeySSMParam);
const iotCertSSMParam = StringParameter.fromStringParameterName(this, "IotCertSSMParamValue", config.iotCertSSMParam)
const localIotClientKeySSMParam = StringParameter.fromStringParameterName(this, "LocalIotClientKeySSMParamValue", config.localIotClientKeySSMParam);
const localIotClientCertSSMParam = StringParameter.fromStringParameterName(this, "LocalIotClientCertSSMParamValue", config.localIotClientCertSSMParam)

const cluster = new Cluster(this, 'MqttBridgeCluster', {
enableFargateCapacityProviders: true,
Expand All @@ -50,10 +52,11 @@ export class MqttBridgeStack extends Stack {
streamPrefix: "nrfcloud-bridge"
}),
secrets: {
NRFCLOUD_CLIENT_CERT: Secret.fromSsmParameter(accountDeviceClientCertSSMParam),
NRFCLOUD_CLIENT_KEY: Secret.fromSsmParameter(accountDeviceClientKeySSMParam),
IOT_CERT: Secret.fromSsmParameter(iotCertSSMParam),
IOT_KEY: Secret.fromSsmParameter(iotKeySSMParam)
NRFCLOUD_CLIENT_ID: Secret.fromSsmParameter(mqttTeamDeviceClientIdSSMParam),
NRFCLOUD_CLIENT_CERT: Secret.fromSsmParameter(mqttTeamDeviceClientCertSSMParam),
NRFCLOUD_CLIENT_KEY: Secret.fromSsmParameter(mqttTeamDeviceClientKeySSMParam),
IOT_CERT: Secret.fromSsmParameter(localIotClientCertSSMParam),
IOT_KEY: Secret.fromSsmParameter(localIotClientKeySSMParam)
},
environment: {
NRFCLOUD_CA: "-----BEGIN CERTIFICATE-----\n" +
Expand All @@ -76,12 +79,11 @@ export class MqttBridgeStack extends Stack {
"5MsI+yMRQ+hDKXJioaldXgjUkK642M4UwtBV8ob2xJNDd2ZhwLnoQdeXeGADbkpy\n" +
"rqXRfboQnoZsG4q5WTP468SQvvG5\n" +
"-----END CERTIFICATE-----",
// TODO: protect these better somehow
MOSQUITTO_CONFIG: `
connection nrfcloud-bridge
address ${config.nrfcloudMqttEndpoint}:8883
address ${config.nrfCloudMqttEndpoint}:8883
local_clientid nrfcloud-bridge-local
remote_clientid ${config.accountDeviceClientId}
remote_clientid ${config.mqttTeamDeviceClientId}
bridge_protocol_version mqttv311
bridge_cafile /mosquitto/config/nrfcloud_ca.crt
bridge_certfile /mosquitto/config/nrfcloud_client_cert.crt
Expand All @@ -90,18 +92,22 @@ bridge_insecure false
cleansession true
start_type automatic
notifications false
log_type all
log_timestamp true

topic m/# in 1 data/ ${config.mqttTopicPrefix}

connection iot-bridge
address ${config.mqttEndpoint}:8883
address ${config.localMqttEndpoint}:8883
bridge_cafile /mosquitto/config/nrfcloud_ca.crt
bridge_certfile /mosquitto/config/iot_cert.crt
bridge_keyfile /mosquitto/config/iot_key.key
bridge_insecure false
cleansession true
start_type automatic
notifications false
log_type all
log_timestamp true

topic # out 1
`
Expand All @@ -118,46 +124,51 @@ topic # out 1

private getConfig(): Config {
const config: Config = {
accountDeviceCertSSMParam: this.node.tryGetContext("accountDeviceCertSSMParam"),
accountDeviceClientId: this.node.tryGetContext("accountDeviceClientId"),
mqttEndpoint: this.node.tryGetContext("mqttEndpoint"),
accountDeviceKeySSMParam: this.node.tryGetContext("accountDeviceKeySSMParam"),
nrfCloudMqttEndpoint: this.node.tryGetContext("nrfCloudMqttEndpoint"),
mqttTopicPrefix: this.node.tryGetContext("mqttTopicPrefix"),
nrfcloudMqttEndpoint: this.node.tryGetContext("nrfcloudMqttEndpoint"),
iotCertSSMParam: this.node.tryGetContext("iotCertSSMParam"),
iotKeySSMParam: this.node.tryGetContext("iotKeySSMParam")
mqttTeamDeviceClientId: this.node.tryGetContext("mqttTeamDeviceClientId"),
mqttTeamDeviceClientIdSSMParam: this.node.tryGetContext("mqttTeamDeviceClientIdSSMParam"),
mqttTeamDeviceCertSSMParam: this.node.tryGetContext("mqttTeamDeviceCertSSMParam"),
mqttTeamDeviceKeySSMParam: this.node.tryGetContext("mqttTeamDeviceKeySSMParam"),
localMqttEndpoint: this.node.tryGetContext("localMqttEndpoint"),
localIotClientCertSSMParam: this.node.tryGetContext("localIotClientCertSSMParam"),
localIotClientKeySSMParam: this.node.tryGetContext("localIotClientKeySSMParam")
}

const valid = Ajv.validate({
type: "object",
properties: {
accountDeviceCertSSMParam: {
nrfCloudMqttEndpoint: {
type: "string"
},
accountDeviceClientId: {
mqttTopicPrefix: {
type: "string"
},
mqttEndpoint: {
mqttTeamDeviceClientId: {
type: "string"
},
accountDeviceKeySSMParam: {
mqttTeamDeviceClientIdSSMParam: {
type: "string"
},
mqttTopicPrefix: {
mqttTeamDeviceCertSSMParam: {
type: "string"
},
nrfcloudMqttEndpoint: {
mqttTeamDeviceKeySSMParam: {
type: "string"
},
iotKeySSMParam: {
localMqttEndpoint: {
type: "string"
},
iotCertSSMParam: {
localIotClientCertSSMParam: {
type: "string"
}
},
localIotClientKeySSMParam: {
type: "string"
},
},
required: ["mqttTopicPrefix", "accountDeviceKeySSMParam", "mqttEndpoint", "accountDeviceClientId",
"accountDeviceCertSSMParam", "nrfcloudMqttEndpoint", "iotCertSSMParam", "iotKeySSMParam"]
required: ["nrfCloudMqttEndpoint", "mqttTopicPrefix", "mqttTeamDeviceClientId",
"mqttTeamDeviceClientIdSSMParam", "mqttTeamDeviceCertSSMParam", "mqttTeamDeviceKeySSMParam",
"localMqttEndpoint", "localIotClientCertSSMParam", "localIotClientKeySSMParam"]
}, config)

if (!valid) {
Expand Down
Loading