Update results corresponding to https://arxiv.org/abs/2303.11102v2 (v…

…ersion 2) (#7)

Co-authored-by: Martin <[email protected]>

.gitignore

@@ -622,5 +622,6 @@ pyrightconfig.json
 # Ignore all local history of files
 .history
 .ionide
-
+!*jbom-1.2.1.jar
 # End of https://www.toptal.com/developers/gitignore/api/java,visualstudiocode,macos,python,latex,eclipse
+.vscode/settings.json

README.md

@@ -11,16 +11,18 @@ This repository contains the code and data produced for the paper [_Challenges o
 
 
 ## SBOM Producers
-The performance of the following 6 CycloneDX SBOM producers were studied: 
+The performance of the following 6 CycloneDX SBOM producers were studied:
+
+> These are the latest versions as of `Fri 5 May 2023 13:02:33 CEST`.
 
 | Producer | Version |
 | -------- | ------- | 
-| [Build Info Go](https://github.com/jfrog/build-info-go) | 1.8.7 |
-| [CycloneDX Generator](https://github.com/CycloneDX/cdxgen) | 8.0.5 |
-| [CycloneDX Maven Plugin](https://github.com/CycloneDX/cyclonedx-maven-plugin) | 2.7.5 |
+| [Build Info Go](https://github.com/jfrog/build-info-go) | 1.9.3 |
+| [CycloneDX Generator](https://github.com/CycloneDX/cdxgen) | 8.4.3 |
+| [CycloneDX Maven Plugin](https://github.com/CycloneDX/cyclonedx-maven-plugin) | 2.7.8 |
 | [jbom](https://github.com/Contrast-Security-OSS/jbom) | 1.2.1 |
-| [OpenRewrite](https://docs.openrewrite.org/reference/rewrite-maven-plugin) | 4.40.0 |
-| [Depscan](https://github.com/AppThreat/dep-scan) | 3.6.0 |
+| [OpenRewrite](https://docs.openrewrite.org/reference/rewrite-maven-plugin) | 4.45.0 |
+| [Depscan](https://github.com/AppThreat/dep-scan) | 4.1.2 |
 
 
 ## Study Subjects

all-producers.md

@@ -0,0 +1,49 @@
+# Selected for study
+
+| Producer | Version |
+| -------- | ------- | 
+| [Build Info Go](https://github.com/jfrog/build-info-go) | 1.9.3 |
+| [CycloneDX Generator](https://github.com/CycloneDX/cdxgen) | 8.4.3 |
+| [CycloneDX Maven Plugin](https://github.com/CycloneDX/cyclonedx-maven-plugin) | 2.7.8 |
+| [jbom](https://github.com/Contrast-Security-OSS/jbom) | 1.2.1 |
+| [OpenRewrite](https://docs.openrewrite.org/reference/rewrite-maven-plugin) | 4.45.0 |
+| [Depscan](https://github.com/AppThreat/dep-scan) | 4.1.2 |
+
+# Not selected for study
+
+| URL  | Reason  |
+|-|-|
+| https://learn.castsoftware.com/highlight |  proprietary |
+| https://github.com/CycloneDX/cyclonedx-cli  | SBOM transformation tool  |
+| https://www.eclipse.org/antenna  | archived  |
+| https://fossa.com/  | proprietary  |
+| https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium  | proprietary  |
+| https://meterian.io/products/boss  | proprietary  |
+| https://github.com/SAP/jenkins-library  | designed for SAP projects  |
+| https://spack.io/  |  package manager |
+| https://github.com/veracode/srcclr_sbom_gen  | proprietary  |
+| https://github.com/coinbase/salus  | only reports vulnerabilities  |
+| https://securestack.com/  | proprietary  |
+| https://aquasecurity.github.io/trivy/v0.36/  | cannot scan Java projects  |
+| https://github.com/javixeneize/zasca  |  threw exceptions on many of our projects |
+| https://github.com/whitesource-ps/ws-sbom-generator  |  proprietary |
+| https://scribesecurity.com/scribe-platform-lp/  | proprietary  |
+| https://jfrog.com/xray/  |  proprietary |
+| https://github.com/org-metaeffekt/metaeffekt-documentation-template  | not documented how to use it  |
+| https://github.com/Labs64/swid-maven-plugin  | swid  |
+| https://github.com/usnistgov/swid-tools  | swid  |
+| https://www.npmjs.com/package/renovate  | custom format  |
+| https://qmstr.org/documentation/introduction/installation/  | too many components. client and server communication  |
+| https://slsa.dev/verification_summary/v0.1  | only files, and no dependencies  |
+| https://github.com/oss-review-toolkit/ort  | custom format  |
+| https://github.com/spdx/tools-java/blob/master/README.md  | generated a verification code, but no SBOM  |
+| https://github.com/anchore/grype | Java projects are not supported |
+| https://www.scanoss.co.uk/| custom format |
+| https://github.com/opensbom-generator/spdx-sbom-generator | SPDX (future work) |
+| https://github.com/microsoft/sbom-tool | SPDX (future work) |
+| https://github.com/spdx/spdx-maven-plugin | SPDX (future work) | 
+| https://github.blog/2023-03-28-introducing-self-service-sboms/ | SPDX (future work) |
+| https://lift.sonatype.com/ | online tool |
+| https://github.com/anchore/syft | online tool |
+| https://github.com/snyk/snyk-maven-plugin | online tool |
+| https://github.com/nexB/scancode-toolkit | does not detect transitive dependencies [nexB/scancode-toolkit@3383](https://github.com/nexB/scancode-toolkit/issues/3383) |

ground-truth-production/generateGroundTruth.sh

@@ -4,6 +4,7 @@ set -e
 docker build -t maven-dependency-tree ./maven-dependency-tree
 
 for f in ../sbom-production/study-subjects-env/*; do 
+(
   # Get the filename without the extension
   filename=$(basename "$f" .env)
   # Print the env file name
@@ -18,4 +19,6 @@ for f in ../sbom-production/study-subjects-env/*; do
   docker cp $container_ID:/$filename/tree.txt ./results/$filename/maven-dependency-tree/;
   ./maven-dependency-tree/dot-to-json.py ./results/$filename/maven-dependency-tree/tree.txt ./results/$filename/maven-dependency-tree/tree.json
   rm ./results/$filename/maven-dependency-tree/tree.txt
+)&
 done
+wait