[ci] Add a lint to check for unrunnable tests

We have discovered that some unittest have become unrunnable because
we added my mistake dependencies that are marked as only compatible
with the OT platforms. This means that those unittests are then skipped
when running all tests using //...
This commit adds a lint to make sure that all tests are compatible with
the host platform. For efficiency reasons, this lint is run in the sw_test
step. The reason is that running a cquery is quite expensive as it requires
building the graph and even compiling things for certain repositories. By
running it after building the tests, we are sure not to duplicate this work.

Signed-off-by: Amaury Pouly <[email protected]>

azure-pipelines.yml

@@ -278,6 +278,10 @@ jobs:
         --target_pattern_file="${TARGET_PATTERN_FILE}"
     displayName: Build & test SW
   - template: ci/publish-bazel-test-results.yml
+  - bash: ci/scripts/check-unrunnable-tests.sh
+    displayName: Check for unrunnable tests
+    condition: succeededOrFailed()
+    continueOnError: True
 
 - job: chip_englishbreakfast_verilator
   displayName: Verilated English Breakfast (Build)

ci/scripts/check-unrunnable-tests.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+
+# Copyright lowRISC contributors (OpenTitan project).
+# Licensed under the Apache License, Version 2.0, see LICENSE for details.
+# SPDX-License-Identifier: Apache-2.0
+
+set -e
+
+# Look for all tests are incompatible with host, i.e. unrunnable.
+# The cquery will output an empty string for compatible tests and
+# a non-empty string (test name) for the incompatible ones. Therefore
+# we filter out the empty lines.
+ci/bazelisk.sh cquery 'tests(//...)' \
+    --noinclude_aspects \
+    --define DISABLE_VERILATOR_BUILD=true \
+    --output=starlark \
+    --starlark:file=ci/scripts/incompatible_targets.cquery \
+    | sed '/^$/d' > output.txt
+if [ -s output.txt ]; then
+    echo "The following tests are incompatible with the host platform:"
+    cat output.txt
+    exit 1
+fi

ci/scripts/incompatible_targets.cquery

@@ -0,0 +1,6 @@
+# Find all targets that are incompatible with the specified platform
+# (i.e. host by default).
+def format(target):
+  if "IncompatiblePlatformProvider" in providers(target):
+    return target.label
+  return ""