releases BAP 2.4.0

This release brings This release brings Ghidra as the new disassembler
and lifting backend, significantly improves our Thumb
lifter (especially with respect to interworking), adds
forward-chainging rules and context variables to the knowledge base,
support for LLVM 12, a pass that flattens IR, and a new framework for
pattern matching on bytes that leverages the available patterns and
actions from the Ghidra project.

It also contains many bug fixes and improvements, most notable
performance improvements that make bap from 30 to 50 per cent
faster. See below for the full list of changes.

Package-wise, we split bap into three parts: `bap-core`, `bap`, and
`bap-extra`. The `bap-core` metapackage contains the minimal set of
core packages that is necessary to disassemble the binary, the `bap`
package extends this set with various analysis, finally, `bap-extra`
includes rarely used or hard to install packages, such as the symbolic
executor, which is very heavy on installation, and `bap-ghidra`, which
is right now in a very experimental stage and is only installable on
Ubuntu 18.04, since it requires the libghidra-dev package available
from ppa,

```
sudo add-apt-repository ppa:ivg/ghidra -y
sudo apt-get install libghidra-dev -y
sudo apt-get install libghidra-data -y
```

Changelog
=========

Features
--------

- BinaryAnalysisPlatform/bap#1325 adds armeb abi
- BinaryAnalysisPlatform/bap#1326 adds experimental Ghidra disassembler and lifting backend
- BinaryAnalysisPlatform/bap#1332 adds the flatten pass
- BinaryAnalysisPlatform/bap#1341 adds context variables to the knowledge base
- BinaryAnalysisPlatform/bap#1343 adds register aliases to the Core Theory
- BinaryAnalysisPlatform/bap#1358 adds LLVM 12 support
- BinaryAnalysisPlatform/bap#1360 extends the knowledge monad interface
- BinaryAnalysisPlatform/bap#1363 adds forward-chaining rules and Primus Lisp methods
- BinaryAnalysisPlatform/bap#1364 adds a generic byte pattern matcher based on Ghidra
- BinaryAnalysisPlatform/bap#1365 adds support for the Thumb IT blocks
- BinaryAnalysisPlatform/bap#1369 adds some missing `t2LDR.-i12` instructions to the Thumb lifter

Improvements
------------

- BinaryAnalysisPlatform/bap#1336 improves the `main` function discovery heuristics
- BinaryAnalysisPlatform/bap#1337 adds more Primus Lisp stubs and fixes some existing
- BinaryAnalysisPlatform/bap#1342 uses context variables to store the current theory
- BinaryAnalysisPlatform/bap#1344 uses the context variables to store the Primus Lisp state
- BinaryAnalysisPlatform/bap#1355 tweaks symbolization and function start identification facilities
- BinaryAnalysisPlatform/bap#1353 improves arm-family support
- BinaryAnalysisPlatform/bap#1356 stops proposing aliases as potential subroutine names
- BinaryAnalysisPlatform/bap#1361 rewrites knowledge and primus monads
- BinaryAnalysisPlatform/bap#1370 tweaks Primus Lisp' method resolution to keep super methods
- BinaryAnalysisPlatform/bap#1375 error handling and performance tweaks
- BinaryAnalysisPlatform/bap#1378 improves reification of calls in the IR theory (part I)
- BinaryAnalysisPlatform/bap#1379 improves semantics of some ITT instructions
- BinaryAnalysisPlatform/bap#1380 Fixes handling of fallthroughs in IR theory

Bug Fixes
---------

- BinaryAnalysisPlatform/bap#1328 fixes C.ABI.Args `popn` and `align_even` operators
- BinaryAnalysisPlatform/bap#1329 fixes frame layout calculation in the Primus loader
- BinaryAnalysisPlatform/bap#1330 fixes the address size computation in the llvm backend
- BinaryAnalysisPlatform/bap#1333 fixes and improves label handling in the IR theor
- BinaryAnalysisPlatform/bap#1338 fixes core:eff theory
- BinaryAnalysisPlatform/bap#1340 fixes the Node.update for graphs with unlabeled nodes
- BinaryAnalysisPlatform/bap#1347 fixes a knowledge base race condition in the run plugin
- BinaryAnalysisPlatform/bap#1348 fixes endianness in the raw loader
- BinaryAnalysisPlatform/bap#1349 short-circuits evaluation of terms in Bap_main.init
- BinaryAnalysisPlatform/bap#1350 fixes variable rewriter and some Primus Lisp symbolic functions
- BinaryAnalysisPlatform/bap#1351 fixes and improves aarch64 lifter
- BinaryAnalysisPlatform/bap#1352 fixes several Primus Lisp stubs
- BinaryAnalysisPlatform/bap#1357 fixes some T32 instructions that are accessing to PC
- BinaryAnalysisPlatform/bap#1359 fixes handling of let-bound variables in flatten pass
- BinaryAnalysisPlatform/bap#1366 fixes a bug in the `cmp` semantics
- BinaryAnalysisPlatform/bap#1374 fixes handling modified immediate constants in ARM T32 encoding
- BinaryAnalysisPlatform/bap#1376 fixes fresh variable generation
- BinaryAnalysisPlatform/bap#1377 fixes the IR theory implementation

Tooling
-------

- BinaryAnalysisPlatform/bap#1319 fixes the shared folder in deb packages
- BinaryAnalysisPlatform/bap#1320 removes sudo from postinst and postrm actions in the deb packages
- BinaryAnalysisPlatform/bap#1321 enables push flag in the publish-docker-image action
- BinaryAnalysisPlatform/bap#1323 fixes the ppx_bap version in the dev-repo opam file
- BinaryAnalysisPlatform/bap#1331 fixes the docker publisher, also enables manual triggering
- BinaryAnalysisPlatform/bap#1327 fixes a typo in the ubuntu dockerfiles
- BinaryAnalysisPlatform/bap#1345 fixes bapdoc
- BinaryAnalysisPlatform/bap#1346 nightly tests are failing due to a bug upstream

packages/bap-abi/bap-abi.2.4.0/opam

@@ -0,0 +1,31 @@
+opam-version: "2.0"
+name: "bap-abi"
+version: "2.4.0"
+maintainer: "Ivan Gotovchits <[email protected]>"
+authors: "BAP Team"
+homepage: "https://github.com/BinaryAnalysisPlatform/bap/"
+bug-reports: "https://github.com/BinaryAnalysisPlatform/bap/issues"
+dev-repo: "git://github.com/BinaryAnalysisPlatform/bap/"
+license: "MIT"
+build: [
+  ["./configure" "--prefix=%{prefix}%" "--enable-abi"]
+  [make]
+]
+install: [[make "install"]]
+remove: [["ocamlfind" "remove" "bap-plugin-abi"]
+         ["ocamlfind" "remove" "bap-abi"]
+         ["bapbundle" "remove" "abi.plugin"]
+         ]
+depends: [
+  "ocaml" {>= "4.08.0" }
+  "core_kernel" {>= "v0.14" & < "v0.15"}
+  "ppx_bap" {>= "v0.14" & < "v0.15"}
+  "bap-std" {= "2.4.0"}
+]
+synopsis: "BAP ABI integration subsystem"
+
+url {
+  src: "https://github.com/BinaryAnalysisPlatform/bap/archive/v2.4.0.tar.gz"
+  checksum: "md5=b8b1aff8c6846f2213eafc54de07b304"
+  mirrors: "https://mirrors.aegis.cylab.cmu.edu/bap/2.4.0/v2.4.0.tar.gz"
+}

packages/bap-analyze/bap-analyze.2.4.0/opam

@@ -0,0 +1,41 @@
+opam-version: "2.0"
+name: "bap-analyze"
+version: "2.4.0"
+maintainer: "Ivan Gotovchits <[email protected]>"
+authors: "BAP Team"
+homepage: "https://github.com/BinaryAnalysisPlatform/bap/"
+bug-reports: "https://github.com/BinaryAnalysisPlatform/bap/issues"
+dev-repo: "git://github.com/BinaryAnalysisPlatform/bap/"
+license: "MIT"
+build: [
+  ["./configure" "--prefix=%{prefix}%" "--enable-analyze"]
+  [make]
+]
+install: [[make "install"]]
+remove: [["ocamlfind" "remove" "bap-plugin-analyze"]
+         ["bapbundle" "remove" "analyze.plugin"]
+         ]
+depends: [
+  "ocaml" {>= "4.08.0" }
+  "core_kernel" {>= "v0.14" & < "v0.15"}
+  "ppx_bap" {>= "v0.14" & < "v0.15"}
+  "bap-std" {= "2.4.0"}
+  "monads"  {= "2.4.0"}
+  "bap-knowledge" {= "2.4.0"}
+  "bap-core-theory" {= "2.4.0"}
+  "bap-main" {= "2.4.0"}
+  "bitvec" {= "2.4.0"}
+  "linenoise" {>= "1.1.0" & < "2.0.0"}
+]
+synopsis: "Implements the analyze command"
+description: """
+Analyses the knowledge base. Loads the knowledge base and executes the
+specified commands or run the REPL if no commands or script files were
+specified.
+"""
+
+url {
+  src: "https://github.com/BinaryAnalysisPlatform/bap/archive/v2.4.0.tar.gz"
+  checksum: "md5=b8b1aff8c6846f2213eafc54de07b304"
+  mirrors: "https://mirrors.aegis.cylab.cmu.edu/bap/2.4.0/v2.4.0.tar.gz"
+}

packages/bap-api/bap-api.2.4.0/opam

@@ -0,0 +1,35 @@
+opam-version: "2.0"
+name: "bap-api"
+version: "2.4.0"
+maintainer: "Ivan Gotovchits <[email protected]>"
+authors: "BAP Team"
+homepage: "https://github.com/BinaryAnalysisPlatform/bap/"
+bug-reports: "https://github.com/BinaryAnalysisPlatform/bap/issues"
+dev-repo: "git://github.com/BinaryAnalysisPlatform/bap/"
+license: "MIT"
+build: [
+  ["./configure" "--prefix=%{prefix}%" "--enable-api"]
+  [make]
+]
+install: [[make "install"]]
+remove: [["ocamlfind" "remove" "bap-plugin-api"]
+         ["ocamlfind" "remove" "bap-api"]
+         ["bapbundle" "remove" "api.plugin"]
+         ["rm" "-rf" "%{prefix}%/share/bap/api/c"]
+         ]
+depends: [
+  "ocaml" {>= "4.08.0" }
+  "core_kernel" {>= "v0.14" & < "v0.15"}
+  "ppx_bap" {>= "v0.14" & < "v0.15"}
+  "bap-main" {= "2.4.0"}
+  "bap-std" {= "2.4.0"}
+  "regular" {= "2.4.0"}
+  "fileutils"
+]
+synopsis: "A pass that adds parameters to subroutines based on known API"
+
+url {
+  src: "https://github.com/BinaryAnalysisPlatform/bap/archive/v2.4.0.tar.gz"
+  checksum: "md5=b8b1aff8c6846f2213eafc54de07b304"
+  mirrors: "https://mirrors.aegis.cylab.cmu.edu/bap/2.4.0/v2.4.0.tar.gz"
+}

packages/bap-arm/bap-arm.2.4.0/opam

@@ -0,0 +1,51 @@
+opam-version: "2.0"
+name: "bap-arm"
+version: "2.4.0"
+maintainer: "Ivan Gotovchits <[email protected]>"
+authors: "BAP Team"
+homepage: "https://github.com/BinaryAnalysisPlatform/bap/"
+bug-reports: "https://github.com/BinaryAnalysisPlatform/bap/issues"
+dev-repo: "git://github.com/BinaryAnalysisPlatform/bap/"
+license: "MIT"
+build: [
+  ["./configure" "--prefix=%{prefix}%" "--enable-arm"]
+  [make]
+]
+
+install: [[make "install"]]
+
+remove: [
+        ["ocamlfind" "remove" "bap-arm"]
+        ["ocamlfind" "remove" "bap-plugin-arm"]
+        ["bapbundle"   "remove" "arm.plugin"]
+
+]
+
+depends: [
+  "ocaml" {>= "4.08.0" }
+  "core_kernel" {>= "v0.14" & < "v0.15"}
+  "ppx_bap" {>= "v0.14" & < "v0.15"}
+  "monads" {= "2.4.0"}
+  "ogre" {= "2.4.0"}
+  "regular" {= "2.4.0"}
+  "bitvec" {= "2.4.0"}
+  "bitvec-order" {= "2.4.0"}
+  "bap-knowledge" {= "2.4.0"}
+  "bap-core-theory" {= "2.4.0"}
+  "bap-main" {= "2.4.0"}
+  "bap-std" {= "2.4.0"}
+  "bap-abi" {= "2.4.0"}
+  "bap-api" {= "2.4.0"}
+  "bap-c" {= "2.4.0"}
+  "bap-primus" {= "2.4.0"}
+]
+synopsis: "BAP ARM lifter and disassembler"
+description: """
+Provides semantics for ARM instructions, disassembler, and partial
+support for the gnueabi ABI"""
+
+url {
+  src: "https://github.com/BinaryAnalysisPlatform/bap/archive/v2.4.0.tar.gz"
+  checksum: "md5=b8b1aff8c6846f2213eafc54de07b304"
+  mirrors: "https://mirrors.aegis.cylab.cmu.edu/bap/2.4.0/v2.4.0.tar.gz"
+}

packages/bap-beagle-strings/bap-beagle-strings.2.4.0/opam

@@ -0,0 +1,32 @@
+opam-version: "2.0"
+name: "bap-beagle-strings"
+version: "2.4.0"
+maintainer: "Ivan Gotovchits <[email protected]>"
+authors: "BAP Team"
+homepage: "https://github.com/BinaryAnalysisPlatform/bap/"
+bug-reports: "https://github.com/BinaryAnalysisPlatform/bap/issues"
+dev-repo: "git://github.com/BinaryAnalysisPlatform/bap/"
+license: "MIT"
+build: [
+  ["./configure" "--prefix=%{prefix}%" "--enable-strings"]
+  [make]
+]
+install: [[make "install"]]
+remove: [["ocamlfind" "remove" "bap-plugin_strings"]
+         ["bapbundle" "remove" "strings.plugin"]]
+depends: [
+  "ocaml" {>= "4.08.0" }
+  "core_kernel" {>= "v0.14" & < "v0.15"}
+  "ppx_bap" {>= "v0.14" & < "v0.15"}
+  "bap-std" {= "2.4.0"}
+  "regular" {= "2.4.0"}
+  "bap-strings" {= "2.4.0"}
+  "bap-beagle" {= "2.4.0"}
+]
+synopsis: "Finds strings of characters using microexecution"
+
+url {
+  src: "https://github.com/BinaryAnalysisPlatform/bap/archive/v2.4.0.tar.gz"
+  checksum: "md5=b8b1aff8c6846f2213eafc54de07b304"
+  mirrors: "https://mirrors.aegis.cylab.cmu.edu/bap/2.4.0/v2.4.0.tar.gz"
+}

packages/bap-beagle/bap-beagle.2.4.0/opam

@@ -0,0 +1,40 @@
+opam-version: "2.0"
+name: "bap-beagle"
+version: "2.4.0"
+maintainer: "Ivan Gotovchits <[email protected]>"
+authors: "BAP Team"
+homepage: "https://github.com/BinaryAnalysisPlatform/bap/"
+bug-reports: "https://github.com/BinaryAnalysisPlatform/bap/issues"
+dev-repo: "git://github.com/BinaryAnalysisPlatform/bap/"
+license: "MIT"
+build: [
+  ["./configure" "--prefix=%{prefix}%" "--enable-beagle"]
+  [make]
+]
+install: [[make "install"]]
+remove: [["ocamlfind" "remove" "bap-plugin-beagle"]
+         ["ocamlfind" "remove" "bap-beagle-prey"]
+         ["bapbundle" "remove" "beagle.plugin"]
+         ]
+depends: [
+  "ocaml" {>= "4.08.0" }
+  "core_kernel" {>= "v0.14" & < "v0.15"}
+  "ppx_bap" {>= "v0.14" & < "v0.15"}
+  "bap-std" {= "2.4.0"}
+  "regular" {= "2.4.0"}
+  "monads"  {= "2.4.0"}
+  "bap-microx" {= "2.4.0"}
+  "bap-primus" {= "2.4.0"}
+  "bap-strings" {= "2.4.0"}
+  "bap-future" {= "2.4.0"}
+]
+synopsis: "BAP obfuscated string solver"
+description: """
+Like strings on steroids - finds strings encoded in binaries,
+even if they are not truly static."""
+
+url {
+  src: "https://github.com/BinaryAnalysisPlatform/bap/archive/v2.4.0.tar.gz"
+  checksum: "md5=b8b1aff8c6846f2213eafc54de07b304"
+  mirrors: "https://mirrors.aegis.cylab.cmu.edu/bap/2.4.0/v2.4.0.tar.gz"
+}

packages/bap-bil/bap-bil.2.4.0/opam

@@ -0,0 +1,37 @@
+opam-version: "2.0"
+name: "bap-bil"
+version: "2.4.0"
+maintainer: "Ivan Gotovchits <[email protected]>"
+authors: "BAP Team"
+homepage: "https://github.com/BinaryAnalysisPlatform/bap/"
+bug-reports: "https://github.com/BinaryAnalysisPlatform/bap/issues"
+dev-repo: "git://github.com/BinaryAnalysisPlatform/bap/"
+license: "MIT"
+build: [
+  ["./configure" "--prefix=%{prefix}%" "--enable-bil"]
+  [make]
+]
+install: [[make "install"]]
+remove: [["ocamlfind" "remove" "bap-plugin-bil"]
+         ["bapbundle" "remove" "bil.plugin"]]
+depends: [
+  "ocaml" {>= "4.08.0" }
+  "core_kernel" {>= "v0.14" & < "v0.15"}
+  "ppx_bap" {>= "v0.14" & < "v0.15"}
+  "bap-std" {= "2.4.0"}
+  "bap-core-theory" {= "2.4.0"}
+  "bap-knowledge" {= "2.4.0"}
+  "bap-future" {= "2.4.0"}
+  "monads" {= "2.4.0"}
+  "bitvec" {= "2.4.0"}
+  "bitvec-order" {= "2.4.0"}
+  "ogre" {= "2.4.0"}
+  "bap-main" {= "2.4.0"}
+]
+synopsis: "Controls the BIL transformation pipeline"
+
+url {
+  src: "https://github.com/BinaryAnalysisPlatform/bap/archive/v2.4.0.tar.gz"
+  checksum: "md5=b8b1aff8c6846f2213eafc54de07b304"
+  mirrors: "https://mirrors.aegis.cylab.cmu.edu/bap/2.4.0/v2.4.0.tar.gz"
+}

packages/bap-build/bap-build.2.4.0/opam

@@ -0,0 +1,40 @@
+opam-version: "2.0"
+name: "bap-build"
+version: "2.4.0"
+maintainer: "Ivan Gotovchits <[email protected]>"
+authors: "BAP Team"
+homepage: "https://github.com/BinaryAnalysisPlatform/bap/"
+bug-reports: "https://github.com/BinaryAnalysisPlatform/bap/issues"
+dev-repo: "git://github.com/BinaryAnalysisPlatform/bap/"
+license: "MIT"
+build: [
+  ["./configure" "--prefix=%{prefix}%" "--enable-build"]
+  [make]
+]
+
+install: [
+  [make "install"]
+]
+
+remove: [
+  ["ocamlfind" "remove" "bap-build"]
+  ["rm" "-f" "%{bin}%/bapbuild"]
+]
+
+depends: [
+  "ocaml" {>= "4.08.0" }
+  "oasis" {build & >= "0.4.7"}
+  "ocamlbuild"
+  "ocamlfind"
+  "core_kernel" {>= "v0.14" & < "v0.15"}
+  "ppx_bap" {>= "v0.14" & < "v0.15"}
+]
+synopsis: "BAP build automation tools"
+
+flags: light-uninstall
+
+url {
+  src: "https://github.com/BinaryAnalysisPlatform/bap/archive/v2.4.0.tar.gz"
+  checksum: "md5=b8b1aff8c6846f2213eafc54de07b304"
+  mirrors: "https://mirrors.aegis.cylab.cmu.edu/bap/2.4.0/v2.4.0.tar.gz"
+}

packages/bap-bundle/bap-bundle.2.4.0/opam

@@ -0,0 +1,33 @@
+opam-version: "2.0"
+name: "bap-bundle"
+version: "2.4.0"
+maintainer: "Ivan Gotovchits <[email protected]>"
+authors: "BAP Team"
+homepage: "https://github.com/BinaryAnalysisPlatform/bap/"
+bug-reports: "https://github.com/BinaryAnalysisPlatform/bap/issues"
+dev-repo: "git://github.com/BinaryAnalysisPlatform/bap/"
+license: "MIT"
+build: [
+  ["./configure" "--prefix=%{prefix}%" "--enable-bundle"]
+  [make]
+]
+install: [[make "install"]]
+remove: [["ocamlfind" "remove" "bap-bundle"]
+         ["rm" "-f" "%{bin}%/bapbundle"]
+         ]
+depends: [
+  "ocaml" {>= "4.08.0" }
+  "oasis" {build & >= "0.4.7"}
+  "uri"
+  "camlzip"
+  "core_kernel" {>= "v0.14" & < "v0.15"}
+  "ppx_bap" {>= "v0.14" & < "v0.15"}
+  "fileutils"
+]
+synopsis: "BAP bundler"
+
+url {
+  src: "https://github.com/BinaryAnalysisPlatform/bap/archive/v2.4.0.tar.gz"
+  checksum: "md5=b8b1aff8c6846f2213eafc54de07b304"
+  mirrors: "https://mirrors.aegis.cylab.cmu.edu/bap/2.4.0/v2.4.0.tar.gz"
+}

packages/bap-byteweight-frontend/bap-byteweight-frontend.2.4.0/opam

@@ -0,0 +1,41 @@
+opam-version: "2.0"
+name: "bap-byteweight-frontend"
+version: "2.4.0"
+maintainer: "Ivan Gotovchits <[email protected]>"
+authors: "BAP Team"
+homepage: "https://github.com/BinaryAnalysisPlatform/bap/"
+bug-reports: "https://github.com/BinaryAnalysisPlatform/bap/issues"
+dev-repo: "git://github.com/BinaryAnalysisPlatform/bap/"
+license: "MIT"
+build: [
+  ["./configure" "--prefix=%{prefix}%" "--enable-byteweight-frontend"]
+  [make]
+]
+
+install: [[make "install"]]
+
+remove: [["rm" "-f" "%{bin}%/bap-byteweight"]]
+
+depends: [
+  "ocaml" {>= "4.08.0" }
+  "core_kernel" {>= "v0.14" & < "v0.15"}
+  "ppx_bap" {>= "v0.14" & < "v0.15"}
+  "bap-std" {= "2.4.0"}
+  "bap-byteweight" {= "2.4.0"}
+  "cmdliner"
+  "ocurl"
+  "fileutils"
+  "ocamlfind"
+  "re"
+]
+synopsis: "BAP Toolkit for training and controlling Byteweight algorithm"
+description: """
+A command line interface to the byteweight system that can train,
+test, download, and upload binary signatures."""
+flags: light-uninstall
+
+url {
+  src: "https://github.com/BinaryAnalysisPlatform/bap/archive/v2.4.0.tar.gz"
+  checksum: "md5=b8b1aff8c6846f2213eafc54de07b304"
+  mirrors: "https://mirrors.aegis.cylab.cmu.edu/bap/2.4.0/v2.4.0.tar.gz"
+}