San2Vuln Benchmark

A benchmark based on the Vulnloc benchmark, with the addition of recent vulnerabilities and vulnerabilities without fixes.

Refer to san2vuln_bench_list.md for the list of newly added benchmarks.

Additionally, functionality tests have been incorporated for each project using unit tests.

• setup_func.sh: Sets up the repository for functionality tests. If this script is unavailable, simply copy the project from /experiment to /experiment_func.
• config_func.sh: Configures the project with appropriate flags without sanitizer option.
• build_func.sh: Builds the binary containing the bug with without sanitizer option.
• test_func.sh: Performs functionality tests based on unit tests. It compares the list of unit tests passed before applying the patch with those passed after the patch is applied. For projects without unit tests, patches may need to be verified manually.

Note that the metadata, such as crash traces and vulnerability types for san2vuln datasets, has not yet been fully updated. Additionally, the instrumentation for other tools is not yet complete.

San2Vuln Benchmark List

libming

• [CVE-2024-24148] libming/libming#308 (NOFIX)
• [CVE-2024-24146] libming/libming#307 (NOFIX)

Bento4

• [NOFIX-2024-002] axiomatic-systems/Bento4#990 (comment)
• [NOFIX-2024-003] axiomatic-systems/Bento4#990 (comment)
• [NOFIX-2024-004] axiomatic-systems/Bento4#980 (comment)
• [NOFIX-2024-005] axiomatic-systems/Bento4#980 (comment)
• [NOFIX-2024-006] axiomatic-systems/Bento4#989 (comment)
• [NOFIX-2024-007] axiomatic-systems/Bento4#946
• [NOFIX-2024-008] axiomatic-systems/Bento4#964 (comment)
• [NOFIX-2024-009] axiomatic-systems/Bento4#964 (comment)
• [NOFIX-2024-010] axiomatic-systems/Bento4#991 (comment)

Kamilio

• [OSV-2024-1206] https://osv.dev/vulnerability/OSV-2024-1206
• [OSV-2024-1210] https://osv.dev/vulnerability/OSV-2024-1210
• [OSV-2024-1230] https://osv.dev/vulnerability/OSV-2024-1230
• [OSV-2024-1244] https://osv.dev/vulnerability/OSV-2024-1244

Liblouis

• [NOFIX-2024-001] liblouis/liblouis#1632
• [CVE-2022-26981] liblouis/liblouis#1171
• [CVE-2022-31783] liblouis/liblouis#1214
• [GIT-2024-1530] liblouis/liblouis#1530
• [GIT-2024-1531] liblouis/liblouis#1531
• [GIT-2024-1532] liblouis/liblouis#1532
• [GIT-2024-1533] liblouis/liblouis#1533
• [GIT-2024-1534] liblouis/liblouis#1534
• [GIT-2024-1535] liblouis/liblouis#1535
• [GIT-2024-1536] liblouis/liblouis#1536
• [GIT-2024-1537] liblouis/liblouis#1537
• [GIT-2024-1539] liblouis/liblouis#1539

Original README.md of VulnLoc Benchmark

Security vulnerability benchmark with instrumentation support for repair tools.

Usage

NOTE: Please ignore the two bugs in ffmpeg (bug id 9 and 10), since they could not be reproduced easily.

To setup and test each of the bugs, first install the dependencies to projects in the benchmark. Dockerfile contains the list of libraries to be installed. One can also use it to build a docker image, and set up bugs in its container.

For each bug (e.g. CVE-2016-9264), the scripts for setting it up and building it are under their corresponding directory:

• setup.sh: Set up source code version for that bug.
• config.sh: Configure the project with appropriate flags.
• build.sh: Build the binary which contains the bug, with the required sanitizer instrumentation.

Note that the scripts are assumed to run in some docker environment, where the project source code is in some pre-defined directories (e.g. /experiment). Users can adjust the scripts to suit their directory structure.

After building the binary, to reproduce each bug, run the binary against the provided exploit input. Inputs can be found in tests/ directory under each bug directory. The exact command and exploit input to be used for each bug can be found in meta-data.json file. In this file, for each bug, the command for bug reproduction is a combination of the binary_path, crash_input, and exploit_file_list. crash_input specifies the command line argument to be suppied after the binary path, in which the special $POC string is to be replaced with path to the actual exploit input file. For exploit input file, use any of the ones in exploit_file_list.

Note

1. In meta-data.json file, the build_command entry is not intended to be used for reproducing the bug in a dynamic analysis setting. Instead, build_script entry is for reproducing bugs with exploit input. The build_command entry is just provided as a command to build the project, which is more commonly used by static analysis tools.