Plausibility test bom refs #14

CBeck-96 · 2023-06-13T16:47:06Z

closes #5

mmarseu

Shouldn't this check be part of the validate command? It is certainly a type of validation.
If it remains a separate command, available_commands.md should be updated.

IMO, the check that all bom-refs are unique is missing. Your code depends on this assumption but doesn't make sure it is correct. Again, this would be better suited for the validate command, because it is a requirement in the spec. It just can't be expressed in JSON schema.

mmarseu · 2023-06-14T09:58:19Z

cdxev/__main__.py


 logger: logging.Logger
 _STATUS_OK = 0
 _STATUS_APP_ERROR = 2
 _STATUS_USAGE_ERROR = 3
 _STATUS_VALIDATION_ERROR = 4
+_STATUS_plausibility_ERROR = 5


Wrong case.

mmarseu · 2023-06-14T11:06:52Z

cdxev/__main__.py

+        choices=["stdout", "warnings-ng"],
+        default="stdout",
+    )
+    add_output_argument(parser)


Do not use this argument for the report. For one, it is confusing because other commands use the same argument for outputting an SBOM, not a log file. Secondly, the help message is incorrect for your use case.

Also, the interaction between --report-format and --output is not clear to me. What if the user specifies --report-format stdout but still passes an output file? What if it is the other way around: --report-format warnings-ng but no output?

mmarseu · 2023-06-14T11:13:42Z

cdxev/__main__.py

+    if args.output is None:
+        output = Path("./issues.json")
+    else:
+        output = args.output


If the user passes a path to a file in a folder that doesn't exist or a path to a folder, writing to this path will crash.

mmarseu · 2023-06-14T11:17:54Z

cdxev/plausibility_check.py

+        if report_format == "warnings-ng":
+            warnings_ng_handler = WarningsNgReporter(file, output)
+            logger.addHandler(warnings_ng_handler)


From a design-perspective I don't find it intuitive that the command logic should interpret the command-line arguments to decide where to log it's errors. This seems better suited for __main__.py, if you can manage to make that work.

mmarseu · 2023-06-14T11:30:16Z

cdxev/plausibility_check.py

+
+    # check compositions
+    for composition in sbom.get("compositions", []):
+        for reference in composition.get("assemblies", []):


You're missing the compositions[].dependencies array which also contains refs.

mmarseu · 2023-06-14T12:14:36Z

cdxev/plausibility_check.py

+        # check if the dependency tree is connected, i.e. that the product
+        # decribed by the sbom depends directly or indirectly on every component.
+        # also checks that every component is depended on


This is a misinterpretation of CycloneDX. Not all components in an SBOM are required to be part of the same dependency tree.

That is why dependencies and assemblies are both a thing in CycloneDX. There are products assembled from multiple components, where there is no depends-on relationship but rather an is-part-of relationship.

italvi · 2023-06-14T18:47:32Z

Shouldn't this check be part of the validate command? It is certainly a type of validation. If it remains a separate command, available_commands.md should be updated.

IMO, the check that all bom-refs are unique is missing. Your code depends on this assumption but doesn't make sure it is correct. Again, this would be better suited for the validate command, because it is a requirement in the spec. It just can't be expressed in JSON schema.

Agree, though this should not be a default check when executing validate and be controlled through a flag.

Regarding the uniqueness of bom-ref within whole (for dependsOn the uniqueItems property is already used) SBOM you are correct, seems like jsonschema does not support unique key-value pairs, yet. But easy to loop through the SBOM and do this check ourselves.

italvi · 2023-07-04T05:54:39Z

cdxev/__main__.py

+        metavar="<plausability-check>",
+        choices=["yes", "y"],
+        help=(
+            "y/yes if the plausibility of the bom-refs in the"
+            "sbom should also be checked"
+        ),


Why this additional choice? Why not providing the flag plausibility-check means true?

italvi · 2023-07-04T06:11:58Z

cdxev/__main__.py

@@ -580,27 +591,40 @@ def has_target() -> bool:


 def invoke_validate(args: argparse.Namespace) -> int:
+    logger_validate = logging.getLogger(__name__)


why introduce a new logger and not use the existing?

italvi · 2023-07-04T06:16:45Z

tests/test_validate.py

+path_to_second_sbom = (
+    path_to_folder_with_test_sboms + "sub_programm_T5.0.3.96_20220217T101458_cdx.json"
+)


why introduce a second SBOM, if you still make changes to it? In this case, just adjust your first sbom.

adapting the tests for plausibility to this one is, and in doing so changing it to fit the requirements, would be work, that seemed unnecessary

italvi · 2023-07-04T06:17:35Z

tests/test_main.py

@@ -401,20 +401,22 @@ class TestValidateCommand(unittest.TestCase):
    def test_get_validate(
        self, mock_validate: unittest.mock.Mock, mock_read: unittest.mock.Mock
    ) -> None:
+        error_return = {"error"}


introduced for using it once?

italvi · 2023-07-04T06:19:19Z

cdxev/validator/helper.py

+                errors.append(
+                    create_error_orphaned_bom_ref(
+                        affected.get("ref", ""),
+                        "vulnerabilitie " + vulnerability.get("id", ""),


vulnerability

mmarseu · 2024-07-02T13:28:54Z

Has this PR been abandoned? It's been in my todo list forever but now I'm not even sure there is a point in reviewing this anymore.

CBeck-96 · 2024-07-03T08:04:12Z

Considering there are other priorities and demand seems low, i will close it for the time beeing and tackle the subject at a more oppurtune time (propably version 1.0 or higher).

CBeck-96 added 2 commits June 13, 2023 15:36

check for plausibility of bom-refs

1433042

main restored

32d83d0

github-actions bot added enhancement New feature or request unittests labels Jun 13, 2023

mmarseu requested changes Jun 14, 2023

View reviewed changes

CBeck-96 and others added 8 commits June 26, 2023 11:07

Merge branch 'Festo-se:main' into main

46d60ab

Merge branch 'Festo-se:main' into main

4f2d9af

plausibility test

9df8f18

added test for uniquness of bomrefs, test for tree connectivity removed

84a9c65

check for non unique bom-refs

ace7944

migrated non unique bom-ref to validate

b9501e1

plausibility check integrated into validate

3191238

plausability check deleted

9f40972

CBeck-96 force-pushed the plausibility_test_bom_refs branch from abe4be1 to 9f40972 Compare July 1, 2023 23:45

CBeck-96 added 6 commits July 2, 2023 02:01

tried solving isort errors

c252f4f

isorted helper

5c5aa9c

moved logging of validate to main

934e344

run isort then black

ac27a1e

debugged test validate

28dc0fe

another bug in test validate removed

73ded04

italvi reviewed Jul 4, 2023

View reviewed changes

plausibility is a flag

7f7a502

italvi requested changes Jul 4, 2023

View reviewed changes

CBeck-96 and others added 2 commits July 4, 2023 14:10

used global logger

63f1378

Merge branch 'Festo-se:main' into plausibility_test_bom_refs

203a9bc

CBeck-96 requested review from mmarseu and italvi July 4, 2023 12:23

CBeck-96 added 2 commits July 4, 2023 15:06

grammar and error

2b660da

removed test sbom

1dce2a5

CBeck-96 and others added 4 commits August 3, 2023 11:26

Merge branch 'Festo-se:main' into plausibility_test_bom_refs

3b53d0d

Merge branch 'main' into plausibility_test_bom_refs

265575a

Merge branch 'main' into plausibility_test_bom_refs

45ebdf5

Merge branch 'main' into plausibility_test_bom_refs

db01b63

CBeck-96 closed this Jul 3, 2024

italvi mentioned this pull request Jul 4, 2024

fix: remove bom-refs only for deleted components in build public #196

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Plausibility test bom refs #14

Plausibility test bom refs #14

CBeck-96 commented Jun 13, 2023

mmarseu left a comment

mmarseu Jun 14, 2023

mmarseu Jun 14, 2023

mmarseu Jun 14, 2023

mmarseu Jun 14, 2023

mmarseu Jun 14, 2023

mmarseu Jun 14, 2023

italvi commented Jun 14, 2023

italvi Jul 4, 2023

italvi Jul 4, 2023

italvi Jul 4, 2023

CBeck-96 Jul 4, 2023

CBeck-96 Jul 4, 2023

italvi Jul 4, 2023

italvi Jul 4, 2023

mmarseu commented Jul 2, 2024

CBeck-96 commented Jul 3, 2024

		@@ -580,27 +591,40 @@ def has_target() -> bool:


		def invoke_validate(args: argparse.Namespace) -> int:
		logger_validate = logging.getLogger(__name__)

Plausibility test bom refs #14

Plausibility test bom refs #14

Conversation

CBeck-96 commented Jun 13, 2023

mmarseu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

italvi commented Jun 14, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mmarseu commented Jul 2, 2024

CBeck-96 commented Jul 3, 2024