Skip to content

Commit

Permalink
Broke out migration tips
Browse files Browse the repository at this point in the history
  • Loading branch information
ljstella committed Jan 24, 2025
1 parent 4bcd4b9 commit a85aa00
Showing 1 changed file with 17 additions and 2 deletions.
19 changes: 17 additions & 2 deletions docs/contentctl_v5_migration_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,14 +72,29 @@ tags:
...
```

We have shifted the RBA config out of the `tags` object, and re-oriented it around the actual in-product usage. The `rba.message` field is a direct replacement for the old `tags.message` field. Risk object config is now much more straightforward. `rba.risk_objects` takes a list of risk objects that consist of a `field` (the field from your search), `type` (user, system, or other, and we strongly recommend not using Other), and a `score` - an integer between 0 and 100. The risk scores are no longer derived from an opaque formula that involves computing from other fields. Each risk object can now have their own independent risk score, as detailed in the above snippet. Finally, the threat object config is similarly less complicated. `rba.threat_objects` takes a list of objects consisting of a `field` (the field in your search) and `type` (a selection of fields that can be seen [here](https://github.com/splunk/contentctl/blob/0ce5a79d49606609cce55e66708e015abc1257d0/contentctl/objects/rba.py#L17)). These fields are now also only required AND only allowed when the detection receives a risk-related configuration via its deployment. So, your Hunting searches that power dashboards or your Correlation searches that create risk notables, but don't create more risk on their own- none of those require RBA configs, and `contentctl` will give you an error if they are present.
To begin, we have shifted the RBA config out of the `tags` object, and re-oriented it around the actual in-product usage.
- The `rba.message` field is a direct replacement for the old `tags.message` field.
- Risk object config is now much more straightforward. `rba.risk_objects` takes a list of risk objects that consist of a `field` (the field from your search), `type` (user, system, or other, and we strongly recommend not using Other), and a `score` - an integer between 0 and 100. The risk scores are no longer derived from an opaque formula that involves computing from other fields. Each risk object can now have their own independent risk score, as detailed in the above snippet.
- Finally, the threat object config is similarly less complicated. `rba.threat_objects` takes a list of objects consisting of a `field` (the field in your search) and `type` (a selection of fields that can be seen [here](https://github.com/splunk/contentctl/blob/0ce5a79d49606609cce55e66708e015abc1257d0/contentctl/objects/rba.py#L17)).

These fields are now also only required AND only allowed when the detection receives a risk-related configuration via its deployment. So, your Hunting searches that power dashboards or your Correlation searches that create risk notables, but don't create more risk on their own- none of those require RBA configs, and `contentctl` will give you an error if they are present.

If you've configured risk actions in Splunk Enterprise Security before, these changes should seem very familiar- because they are. We've largely aligned fields and enum options with the field names and options used in the UI of Enterprise Security.

Depending on how much content you have to migrate, you may consider writing some quick scripts or a notebook that will create the new RBA objects based on your existing configuration. If you go this route, we found great success with [ruamel.yaml](https://pypi.org/project/ruamel.yaml/) as opposed to PyYAML for the sake of preserving the existing order of fields in your YAMLs. _Technically_, the field order doesn't matter when the objects are parsed and read in, but we have found that the consistency of order of fields helps reviewers identify changes, and keeps diffs easy to understand. If you don't have hundreds or thousands of detections in your repo, it may be more valuable to spend time migrating these configurations by hand to ensure that your previous detections were all configured the way you thought they were, and giving you an opportunity to review risk messages, risk scores, and threat objects for improvements.

### Managed Lookup Filenames

Depending on where your Splunk deployment lives (SplunkCloud's Victoria Experience, Classic Experience, or in your own datacenter, or your own public cloud tenant), you may have experienced some significant pains with deploying lookups as part of your app. If you're not already familiar with the peculiar behaviors of lookup file updating, you can read more about it [here](https://docs.splunk.com/Documentation/SplunkCloud/9.3.2408/Admin/PrivateApps#Manage_lookups_in_Splunk_Cloud_Platform). Essentially, depending on what your Splunk deployment looks like, updating an app that has changes to lookup files may cause those new files to be entirely ignored, with the previously used versions staying in use.

We've come up with a solution for this that should generally work for folks, no matter their deployment, that will allow you to use new lookups when you update an app without manually editing them. CSV Lookups in `contentctl` built apps now have a datestamp added to the end of the filename automatically, as derived from the date in the lookup file. This means updating an app adds a new file instead of one that would be ignored during the update process. The lookup definition that gets written to `transforms.conf` also gets this new filename. Our searches leverage the lookup definitions instead of the raw filenames, so when a new version of a lookup appears, it will seamlessly be used in favor of the old one. There are some changes to the lookup YAML files to support this, which will also throw errors (likely, the first errors you'll see as part of migration). You can review the configs used in ESCU for these files [here](https://github.com/splunk/security_content/tree/develop/lookups) if you'd like to see how we're using some of the fields.

### Migration Tips

The recommended order of operations for migrating is as follows:
1. Run `contentctl validate` with the new version. It should fail early on during Lookup validation.
2. Proceed with cleaning up (removing extra fields, reconfiguring) lookups, stories, and macros.
3. Once you get to the point that you're getting validation errors for detections, you should migrate your RBA config first. Before you delete fields like `tags.impact` and `tags.confidence` you will want to create the new `rba` object so you can use the old config to build a comparable new one.
4. Once all of your detections that require risk config are ported over to the `rba` object, then you should proceed with removing all of the extra fields. Running `contentctl validate`, removing a few, saving the file, and running validate again as you go should help.


Depending on how much content you have to migrate, you may consider writing some quick scripts or a notebook that will create the new RBA objects based on your existing configuration before you delete anything. If you go this route, we found great success with [ruamel.yaml](https://pypi.org/project/ruamel.yaml/) as opposed to PyYAML for the sake of preserving the existing order of fields in your YAMLs. _Technically_, the field order doesn't matter when the objects are parsed and read in, but we have found that the consistency of order of fields helps reviewers identify changes, and keeps diffs easy to understand. If you don't have hundreds or thousands of detections in your repo, it may be more valuable to spend time migrating these configurations by hand to ensure that your previous detections were all configured the way you thought they were, and giving you an opportunity to review risk messages, risk scores, and threat objects for improvements.

0 comments on commit a85aa00

Please sign in to comment.