Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: mirage/digestif
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.7.2
Choose a base ref
...
head repository: mirage/digestif
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
Loading
Showing with 8,912 additions and 2,549 deletions.
  1. +147 −0 .github/workflows/test.yml
  2. +10 −6 .ocamlformat
  3. +10 −0 .test-mirage.sh
  4. +3 −2 .travis.yml
  5. +91 −0 CHANGES.md
  6. +48 −12 README.md
  7. +20 −14 digestif.opam
  8. +1 −2 dune-project
  9. +6 −0 fuzz/c/dune
  10. +25 −0 fuzz/dune
  11. +32 −0 fuzz/fuzz.ml
  12. +6 −0 fuzz/ocaml/dune
  13. +1 −0 mirage/_tags
  14. +5 −0 mirage/config.ml
  15. +7 −0 mirage/unikernel.ml
  16. +355 −287 src-c/digestif.ml
  17. +222 −113 src-c/digestif_native.ml
  18. +16 −15 src-c/dune
  19. +3 −0 src-c/native/bitfn.h
  20. +2 −1 src-c/native/blake2b.c
  21. +2 −1 src-c/native/blake2s.c
  22. +8 −1 src-c/native/digestif.h
  23. +0 −5 src-c/native/dune
  24. +0 −4 src-c/native/freestanding/cflags.sh
  25. +0 −20 src-c/native/freestanding/dune
  26. +156 −0 src-c/native/sha3.c
  27. +24 −0 src-c/native/sha3.h
  28. +152 −15 src-c/native/stubs.c
  29. +0 −4 src-c/native/xen/cflags.sh
  30. +0 −20 src-c/native/xen/dune
  31. +177 −171 src-ocaml/baijiu_blake2b.ml
  32. +154 −134 src-ocaml/baijiu_blake2s.ml
  33. +31 −0 src-ocaml/baijiu_keccak_256.ml
  34. +30 −26 src-ocaml/baijiu_md5.ml
  35. +44 −39 src-ocaml/baijiu_rmd160.ml
  36. +31 −30 src-ocaml/baijiu_sha1.ml
  37. +11 −7 src-ocaml/baijiu_sha224.ml
  38. +61 −56 src-ocaml/baijiu_sha256.ml
  39. +187 −0 src-ocaml/baijiu_sha3.ml
  40. +12 −8 src-ocaml/baijiu_sha384.ml
  41. +31 −0 src-ocaml/baijiu_sha3_224.ml
  42. +31 −0 src-ocaml/baijiu_sha3_256.ml
  43. +31 −0 src-ocaml/baijiu_sha3_384.ml
  44. +31 −0 src-ocaml/baijiu_sha3_512.ml
  45. +79 −75 src-ocaml/baijiu_sha512.ml
  46. +742 −722 src-ocaml/baijiu_whirlpool.ml
  47. +339 −283 src-ocaml/digestif.ml
  48. +10 −25 src-ocaml/dune
  49. +8 −5 src-ocaml/xor.ml
  50. +205 −98 src/digestif.mli
  51. +9 −4 src/digestif_bi.ml
  52. +11 −8 src/digestif_by.ml
  53. +19 −14 src/digestif_conv.ml
  54. +0 −34 src/digestif_hash.ml
  55. +4 −4 src/dune
  56. +42 −4 test/c/dune
  57. +8 −5 test/conv/dune
  58. +32 −34 test/conv/test_conv.ml
  59. +950 −0 test/keccak_256.txt
  60. +42 −4 test/ocaml/dune
  61. +983 −0 test/sha3_224_fips_202.txt
  62. +951 −0 test/sha3_256_fips_202.txt
  63. +823 −0 test/sha3_384_fips_202.txt
  64. +695 −0 test/sha3_512_fips_202.txt
  65. +525 −237 test/test.ml
  66. +66 −0 test/test_cve.ml
  67. +155 −0 test/test_runes.ml
147 changes: 147 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
name: Cross-platform tests

on:
pull_request:
push:
branches:
- 'master'

jobs:
test-with-setup-ocaml:
strategy:
fail-fast: false
matrix:
os:
- windows-latest
- ubuntu-latest
- macos-latest
ocaml-compiler:
- '4.13.x'
runs-on: ${{ matrix.os }}
name: test-ocaml / ${{ matrix.os }}-${{ matrix.ocaml-compiler }}
steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Hack Git CRLF for ocaml/setup-ocaml issue #529
if: ${{ startsWith(matrix.os, 'windows-') }}
run: |
& "C:\Program Files\Git\bin\git.exe" config --system core.autocrlf input
- name: OCaml ${{ matrix.ocaml-compiler }} with Dune cache
uses: ocaml/setup-ocaml@v2
if: ${{ !startsWith(matrix.os, 'windows-') }}
with:
ocaml-compiler: ${{ matrix.ocaml-compiler }}
dune-cache: true
- name: OCaml ${{ matrix.ocaml-compiler }} without Dune cache
uses: ocaml/setup-ocaml@v2
if: ${{ startsWith(matrix.os, 'windows-') }}
with:
ocaml-compiler: ${{ matrix.ocaml-compiler }}
dune-cache: false
- name: Install/build/test
run: |
opam install . --deps-only --with-test
opam exec -- dune build --display=short
opam exec -- dune runtest --display=short
setup-dkml:
uses: 'diskuv/dkml-workflows/.github/workflows/setup-dkml.yml@v0'
permissions: {} # remove all rights of GITHUB_TOKEN when it is passed to setup-dkml.yml
with:
ocaml-compiler: 4.12.1

test-with-setup-dkml:
needs: setup-dkml
strategy:
fail-fast: false
matrix:
include:
- os: windows-2019
abi-pattern: win32-windows_x86
dkml-host-abi: windows_x86
opam-root: D:/.opam
default_shell: msys2 {0}
msys2_system: MINGW32
msys2_packages: mingw-w64-i686-pkg-config
bits: "32"
- os: windows-2019
abi-pattern: win32-windows_x86_64
dkml-host-abi: windows_x86_64
opam-root: D:/.opam
default_shell: msys2 {0}
msys2_system: CLANG64
msys2_packages: mingw-w64-clang-x86_64-pkg-config
bits: "64"
- os: macos-latest
abi-pattern: macos-darwin_all
dkml-host-abi: darwin_x86_64
default_shell: sh
opam-root: /Users/runner/.opam
bits: "64"
- os: ubuntu-latest
abi-pattern: manylinux2014-linux_x86
bits: "32"
default_shell: sh
dkml-host-abi: linux_x86
opam-root: .ci/opamroot # local directory of $GITHUB_WORKSPACE so available to dockcross
- os: ubuntu-latest
abi-pattern: manylinux2014-linux_x86_64
bits: "64"
default_shell: sh
dkml-host-abi: linux_x86_64
opam-root: .ci/opamroot # local directory of $GITHUB_WORKSPACE so available to dockcross
runs-on: ${{ matrix.os }}
name: test-dkml / ${{ matrix.abi-pattern }}
defaults:
run:
shell: ${{ matrix.default_shell }}
env:
OPAMROOT: ${{ matrix.opam-root }}
COMPONENT: dkml-component-staging-opam${{ matrix.bits }}
steps:
- name: Checkout
uses: actions/checkout@v3

- uses: actions/download-artifact@v3
with:
path: .ci/dist

- name: Install MSYS2 (Windows)
if: startsWith(matrix.dkml-host-abi, 'windows_')
uses: msys2/setup-msys2@v2
with:
msystem: ${{ matrix.msys2_system }}
update: true
install: >-
${{ matrix.msys2_packages }}
wget
make
rsync
diffutils
patch
unzip
git
tar
- name: Import build environments from setup-dkml
run: |
${{ needs.setup-dkml.outputs.import_func }}
import ${{ matrix.abi-pattern }}
- name: Cache Opam downloads by host
uses: actions/cache@v3
with:
path: ${{ matrix.opam-root }}/download-cache
key: ${{ matrix.dkml-host-abi }}

- name: Install/build/test
run: |
# Fix dependencies to work with MSVC
# - alcotest.1.4.0 works with MSVC; 1.5.0 does not
opamrun pin alcotest -k version 1.4.0 --no-action --yes
opamrun install . --deps-only --with-test --yes
opamrun exec -- dune build --display=short
opamrun exec -- dune runtest --display=short
16 changes: 10 additions & 6 deletions .ocamlformat
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
module-item-spacing=compact
break-struct=natural
break-infix=fit-or-vertical
parens-tuple=multi-line-only
wrap-comments=true
break-collection-expressions=wrap
version = 0.21.0
break-infix = fit-or-vertical
parse-docstrings = true
indicate-multiline-delimiters=no
nested-match=align
sequence-style=separator
break-before-in=auto
if-then-else=keyword-first
dock-collection-brackets=true
break-collection-expressions=wrap
10 changes: 10 additions & 0 deletions .test-mirage.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/bin/sh

set -ex

opam install -y mirage
(cd mirage && mirage configure -t unix && make depends && mirage build && ./digestif_test && mirage clean && cd ..) || exit 1
(cd mirage && mirage configure -t hvt && make depends && mirage build && mirage clean && cd ..) || exit 1
if [ $(uname -m) = "amd64" ] || [ $(uname -m) = "x86_64" ]; then
(cd mirage && mirage configure -t xen && make depend && mirage build && mirage clean && cd ..) || exit 1
fi
5 changes: 3 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -11,7 +11,8 @@ env:
- OCAML_VERSION=4.03
- OCAML_VERSION=4.04
- OCAML_VERSION=4.05
- OCAML_VERSION=4.05 DEPOPTS="mirage-xen-posix"
- OCAML_VERSION=4.06
- OCAML_VERSION=4.07
- OCAML_VERSION=4.07 DEPOPTS="ocaml-freestanding"
- OCAML_VERSION=4.08 DEPOPTS="ocaml-freestanding"
- OCAML_VERSION=4.08
- OCAML_VERSION=4.09 TESTS=false POST_INSTALL_HOOK=./.test-mirage.sh
91 changes: 91 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,94 @@
### Pending

- Use `CAMLextern` rather than `extern` in `caml_*` forward declarations to
support bytecode linking on Windows (@jonahbeckford, #157)

### v1.2.0 2024-03-18 Paris (France)

- Update the description to include SHA3 (@Leonidas-from-XIV, #146)
- Add a new type `hash'`, a polymorphic variant (@reynir, @dinosaure, #150)
- Lint `fmt` dependency lower-bound (@reynir, #152)
- Add `get_into_bytes` function and a fuzzer about it (@reynir, @dinosaure, #149)

### v1.1.4 2023-03-23 Paris (France)

- Add a test about CVE-2022-37454 (@dinosaure, #143)
- Lint the distribution and delete the `pkg-config` dependency (@dinosuare, 1eff5c5)
- Fix primitives used for bytes and fix the support of `js_of_ocaml` 5 (@hhugo, #144)

### v1.1.3 2022-10-20 Paris (France)

- Support MSVC compiler (@jonahbeckford, #137)
- Fix CI on Windows (`test_conv.ml` requires `/dev/urandom`) (@dinosaure, #138)
- Fix threads support (@dinosaure, #140)
- Delete the META trick needed for MirageOS 3 when we install `digestif` (@dinosaure, #141)
This version of `digestif` breaks the compatibility with MirageOS 3
and `ocaml-freestanding`. This PR should unlock the ability to
use `dune-cache`.

### v1.1.2 2022-04-08 Paris (France)

- Minor update on the README.md (@punchagan, #133)
- Support only OCaml >= 4.08, update with `ocamlformat.0.21.0` and remove `bigarray-compat`
dependency (@hannesm, #134)

### v1.1.1 2022-03-28 Paradou (France)

- Hide C functions (`sha3_keccakf`) (@hannesm, #125)
- Use `ocaml` to run `install.ml` instead of a shebang (@Nymphium, #127)
- Use `command -v` instead of `which` (@Numphium, #126)
- Add `@since` meta-data in documentation (@c-cube, @dinosaure, #128)
- Update the README.md (@dinosaure, @mimoo, #130)
- `ocaml-solo5` provides `__ocaml_solo5__` instead of `__ocaml_freestanding__` (@dinosaure, #131)

### v1.1.0 2021-10-11 Paris (France)

- Add Keccak256 module (ethereum padding) (@maxtori, @dinosaure, #118)
- Update README.md to include the documentation (@mimoo, @dinosaure, 65a5c12)
- Remove deprecated function from `fmt` library (@dinosaure, #121)
- **NOTE**: This version lost the support of OCaml 4.03 and OCaml 4.04.

### v1.0.1 2020-02-08 Paris (France)

- Fix `esy` support (@dinosaure, #115)
- Fix big-endian support (@dinosaure, #113)

### v1.0.0 2020-11-02 Paris (France)

- **breaking changes** Upgrade the library with MirageOS 3.9 (new layout of artifacts)
Add tests about compilation of unikernels (execution and link)
(#105, @dinosaure, @hannesm)
- Fix `esy` installation (#104, @dinosaure)
- **breaking changes** Better GADT (#103, @dinosaure)
As far as I can tell, nobody really use this part of `digestif`.
The idea is to provide a GADT which contains the type of the hash.
From third-part libraries point-of-view, it's better to _pattern-match_ with
such information instead to use a polymorphic variant (as before).
- **breaking changes** key used for HMAC is a constant `string` (#101, @dinosaure, @hannesm)
The key should not follow the same type as the digest value (`string`, `bytes`, `bigstring`).
This update restricts the user to user only constant key (as a `string`).

### v0.9.0 2020-07-10 Paris (France)

- Add sha3 implementation (#98), @lyrm, @dinosaure, @hannesm and @cfcs

### v0.8.1 2020-06-15 Paris (France)

- Move to `dune.2.6.0` (#97)
- Apply `ocamlformat.0.14.2` (#97)
- Fix tests according `alcotest.1.0.0` (#95)

### v0.8.0 2019-20-09 Saint Louis (Sénégal)

- Fake version to prioritize dune's variants instead of
old linking trick
- Use `stdlib-shims` to keep compatibility with < ocaml.4.07.0

### v0.7.3 2019-07-09 Paris (France)

- Fix bug about specialization of BLAKE2{B,S} (#85, #86)
reported by @samoht, fixed by @dinosaure, reviewed by @hannes and @cfcs

### v0.7.2 2019-05-16 Paris (France)

- Add conflict with `< mirage-xen-posix.3.1.0` packages (@hannesm)
60 changes: 48 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,12 @@
Digestif - Hash algorithms in C and OCaml
=========================================

[![Build Status](https://travis-ci.org/mirage/digestif.svg?branch=master)](https://travis-ci.org/mirage/digestif)

Digestif is a toolbox which implements hashes:

* MD5
* SHA1
* SHA224
* SHA256
* SHA384
* SHA512
* SHA2
* SHA3
* WHIRLPOOL
* BLAKE2B
* BLAKE2S
@@ -25,13 +21,52 @@ implementation he wants to use. We provide 2 implementations:
Both are well-tested. However, OCaml implementation is slower than the C
implementation.

**Note**: The linking trick requires `digestif.c` or `digestif.ocaml` to be the first of your dependencies.

**Note**: The linking trick requires `digestif.c` or `digestif.ocaml` to be the
first of your dependencies.

Home page: http://din.osau.re/
Documentation: https://mirage.github.io/digestif/

Contact: Romain Calascibetta `<romain.calascibet ta@gmail.com>`

## Install & Usage

The library is available on [OPAM](https://opam.ocaml.org/packages/digestif/). You can install it via:
```sh
$ opam install digestif
```

This is a simple program which implements `sha1sum`:
```sh
$ cat >sha1sum.ml <<EOF
let sum ic =
let tmp = Bytes.create 0x1000 in
let rec go ctx = match input ic tmp 0 0x1000 with
| 0 -> Digestif.SHA1.get ctx
| len ->
let ctx = Digestif.SHA1.feed_bytes ctx ~off:0 ~len tmp in
go ctx
| exception End_of_file -> Digestif.SHA1.get ctx in
go Digestif.SHA1.empty
let () = match Sys.argv with
| [| _; filename; |] when Sys.file_exists filename ->
let ic = open_in filename in
let hash = sum ic in
close_in ic ; print_endline (Digestif.SHA1.to_hex hash)
| [| _ |] ->
let hash = sum stdin in
print_endline (Digestif.SHA1.to_hex hash)
| _ -> Format.eprintf "%s [<filename>]\n%!" Sys.argv.(0)
EOF
$ cat >dune <<EOF
(executable
(name sha1sum)
(libraries digestif))
EOF
$ dune exec ./sha1sum.exe -- sha1sum.ml
fe6e6639a817c23857b507e2d833ec776f23f327
```

## API

For each hash, we implement the same API which is referentially transparent.
@@ -77,7 +112,8 @@ This work is from the [nocrypto](https://github.com/mirleft/nocrypto) library
and the Vincent hanquez's work in
[ocaml-sha](https://github.com/vincenthz/ocaml-sha).

All credits appear in the begin of files and this library is motivated by two reasons:

* delete the dependancy with `nocrypto` if you don't use the encryption (and common) part
All credits appear in the begin of files and this library is motivated by two
reasons:
* delete the dependancy with `nocrypto` if you don't use the encryption (and
common) part
* aggregate all hashes functions in one library
Loading