update compliance reporting in Automate to match CVSS 3.0 standards

[vjeffrey]

User Story

Chef InSpec itself does not assume status of a report or an individual profile results status in a report. It only reports the status of each control, leaving the consumer to interpret the impacts, failures, and passes the way they would like.

In Automate, we've been evaluating report (node and profile) status in Automate following the CVSS 2.0 standards:

Low | 0.0-0.3 (minor)
Medium | 0.4-0.7 (major)
High | 0.7-1.0 (critical)

Now that CVSS 3.0 standards are around, we should upgrade our logic to those numbers:

None | 0.0
Low | 0.1-3.9
Medium | 4.0-6.9
High | 7.0-8.9
Critical | 9.0-10.0

Added benefit: this introduces a "none" or "information" impact value. A 0 impact control (or information control) should not count against the overall profile or node status. These are for evaluation only. This can be used by those interested in exceptions, as the status of the control results will still be reported, it just won't count against the overall status of the profile/node.

UI WORK: The node results details page has button filters for the control impacts. This will need to be updated for the new statuses (we need some ux for this). The profile results details page also has some light logic for displaying the impact of the control which will need to be updated.

"Informational" will also need to be a new status in Automate in general. With the introduction of these new statuses, it could be possible to have a profile with only informational controls, and it is possible to only run an "informational" profile on a node. This means the possible statuses for nodes, profiles, and controls will be informational, skipped, passed, failed.

Definition of Done

Create a profile with at least one control of each new impact range
Scan a node with Automate using the profile
Expect the correct results. Informational control results should not contribute to overall status.
Filter by control, see status updates
Create a profile with only informational controls.
Scan a node with the profile
Expect "informational" status for both the node and profile.

Related Resources

https://nvd.nist.gov/vuln-metrics/cvss
https://chef.aha.io/ideas/ideas/A2-I-32

Aha! Link: https://chef.aha.io/epics/A2-E-161

[jonong1972]

For the UI bits, on the Compliance Nodes Details, first step what I would like to do is combine all the failed filters into just one filter button (critical/major/minor) so the filter bar would only show:

Total Controls | Failed Controls | Skipped Controls | Passed Controls | Informational(?) <-- not sure on this still.

UX team still has to design a pattern to allow a subset of filters for the Failed Controls.

Probably be some type of check-boxes with the values of:
Critical | High | Medium | Low

[vjeffrey]

note:
this work has not yet been done for a few reasons

• we've only heard one or two customer ask
• transitioning an impact value that is currently of range 0-1.0 to a range that is 0-10.0 is a challenge. Under the existing pattern, an impact value of 0.8 would be considered "critical". Under the new range, that would be minor. We certainly don't want to swap things out from under our customers like that, meaning this is a "breaking change".
• this change also requires us to update all chef produced content to match the new range