Merge pull request #15 from Koen1999/docs

Improved documentation website

.github/workflows/python-pr.yml

@@ -103,7 +103,6 @@ jobs:
 
       - name: Lint with flake8
         run: |
-          # stop the build if there are Python syntax errors or undefined names
           flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
 
       - name: Type checking with Pyright
@@ -117,3 +116,32 @@ jobs:
       - name: Lint with Ruff
         run: |
           ruff check --output-format=github .
+
+  docs:
+    name: Docs
+    runs-on: self-hosted
+    needs:
+      - lint
+
+    strategy:
+      fail-fast: false
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python 3.13
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.13"
+          cache: "pip"
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip setuptools wheel build
+          pip install -r requirements.txt
+
+      - name: Generate docs with sphinx
+        run: |
+          cd docs
+          make clean
+          make html

docs/Makefile

@@ -17,4 +17,4 @@ help:
 # Catch-all target: route all unknown targets to Sphinx using the new
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
 %: Makefile
-	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) -v -T

docs/api_usage.md

@@ -1,3 +1,9 @@
+---
+myst:
+    html_meta:
+        "description lang=en": "suricata-check is a command line utility to provide feedback on Suricata rules to by detecting issues through static analysis."
+        "keywords": "Suricata, rules, ruleset, suricata-check, API, Python"
+---
 # API Usage
 
 Sometimes it may be more convenient to avoid the CLI and instead use the module directly, which exposes more functionality and may be easier te extend if your project also uses Python. Below, we will characterize several use-cases you may encounter and how to address them using the functionality exposed by `suricata-check`.

docs/checker.md

@@ -1,3 +1,9 @@
+---
+myst:
+    html_meta:
+        "description lang=en": "suricata-check is a command line utility to provide feedback on Suricata rules to by detecting issues through static analysis."
+        "keywords": "Suricata, suricata-check, Extend, Checker"
+---
 # Writing checkers
 
 ## CheckerInterface
@@ -24,7 +30,7 @@ class ExampleChecker(CheckerInterface):
 
 ## Detecting issues
 
-To detect issues, you can use utility functions provided in `suricata_check.utils.checker`. A lot of utility functions exist, and you are encouraged to check out the [Checker API Reference](./_build/html/_modules/suricata_check/utils/checker/index.html) for a complete overview. For example, it contains utility functions to check whether a Suricata option is (not) set, and enables asserting that atleast one or all option values are (not) equal to a certain value or regular expression.
+To detect issues, you can use utility functions provided in `suricata_check.utils.checker`. A lot of utility functions exist, and you are encouraged to check out the [Checker API Reference](https://suricata-check.teuwen.net/autoapi/suricata_check/utils/checker/index.html) for a complete overview. For example, it contains utility functions to check whether a Suricata option is (not) set, and enables asserting that atleast one or all option values are (not) equal to a certain value or regular expression.
 
 All you have to do to add new issue types is, to add the desired issue code (e.g. `E000`) to the `codes` field of the class, and append a new `Issue` to the list of `issues` that is returned at the end of `_check_rule` depending on the output of the utlity function called from `suricata_check.utils.checker`. For example, we can add two new issue types as follows:
 

docs/ci_cd.md

@@ -1,3 +1,9 @@
+---
+myst:
+    html_meta:
+        "description lang=en": "suricata-check is a command line utility to provide feedback on Suricata rules to by detecting issues through static analysis."
+        "keywords": "Suricata, rules, ruleset, suricata-check, Continuous Integration, Continuous Deployment, Workflow, Action, GitHub, GitLab, CodeClimate"
+---
 # CI/CD Integration
 
 If you maintain a large rulebase in through version-control managed platform, you may be interested in integrating `suricata-check` with your Continuous Integration and Continuous Deployment workflows.

docs/cli_usage.md

@@ -1,3 +1,9 @@
+---
+myst:
+    html_meta:
+      "description lang=en": "suricata-check is a command line utility to provide feedback on Suricata rules to by detecting issues through static analysis."
+      "keywords": "Suricata, rules, ruleset, suricata-check, CLI, Command Line, rules, rule"
+---
 # CLI Usage
 
 After having installed `suricata_check` as described in [Installation](./readme.md#installation) and having ensured it is included in the `PATH` variable (check your operating system specific instructions for this), you can invoke it using by running `suricata-check`.
@@ -44,7 +50,7 @@ suricata-check -o /path/to/my/output/dir
 
 `suricata-check-fast.log` contains output that is easy to read for humans. The structure is similar to that of `suricata.fast` as generated by `suricata` when rules trigger alerts. When `suricata-check` detects issues with a rule, it will write one line for each issue in `suricata-check-fast.log` similar to:
 
-```log
+```
 [C100] Line 1, sid 2260000: The rule does not use the `target` Suricata meta option. Consider adding the `target` option to specify which IP address is the target of the attack.
 ```
 
@@ -150,7 +156,7 @@ suricata-check -o /path/to/my/output/dir
 
 If you are interested in an overview of how many rules have issues and which issues are prevelant, this is also provided in `suricata-check-stats.log`. The output is tabular in structure, such as the following example snippet:
 
-```log
+```
  MetadataChecker
 -----------------
         Count  Percentage of Rules
@@ -209,7 +215,7 @@ suricata-check -i "M.*|S8.." -e S802
 
 You can inspect the `suricata-check.log` output to verify if the intended checkers are indeed enabled:
 
-```log
+```
 2024-12-14 15:26:43,332 - suricata_check.suricata_check - INFO - include_all: False
 2024-12-14 15:26:43,332 - suricata_check.suricata_check - INFO - include: ('M.*|S8..',)
 2024-12-14 15:26:43,332 - suricata_check.suricata_check - INFO - exclude: ('S802',)

docs/conf.py

@@ -26,11 +26,11 @@
 
 extensions = [
     "myst_parser",
-    "sphinx.ext.autodoc",
     "autoapi.extension",
     "sphinx.ext.intersphinx",
     "sphinx.ext.viewcode",
     "sphinx_click",
+    "sphinx_sitemap",
 ]
 
 templates_path = ["_templates"]
@@ -49,46 +49,28 @@
 root_doc = "index"
 master_doc = "index"
 
+suppress_warnings = []
+
 # -- Options for HTML output -------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
 
 html_theme = "sphinx_rtd_theme"
+html_theme_options = {
+    "display_version": True,
+    "prev_next_buttons_location": "both",
+    "style_external_links": True,
+}
 html_static_path = ["static"]
+html_favicon = "https://docs.readthedocs.io/favicon.ico"
 
-html_js_files = ["js/umami.js"]
+html_js_files = ["js/script.js"]
 
 # -- Options for MyST     -------------------------------------------------
 # https://myst-parser.readthedocs.io/en/latest/
 
 myst_enable_extensions = ["linkify"]
 myst_heading_anchors = 5
 
-# -- Options for Autodoc     -------------------------------------------------
-# https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#configuration
-
-autoclass_context = "both"
-autodoc_class_signature = "separated"
-autodoc_member_order = "groupwise"
-autodoc_default_flags = [
-    "members",
-    "special-members",
-    "inherited-members",
-    "show-inheritance",
-    "ignore-module-all",
-    "maxdepth",
-]
-autodoc_default_options = {
-    "member-order": "bysource",
-    "special-members": "__init__",
-    "undoc-members": False,
-    "maxdepth": 2,
-}
-autodoc_typehints = "both"
-autodoc_typehints_description_target = "all"
-
-# -- Options for AutoAPI     -------------------------------------------------
-# https://sphinx-autoapi.readthedocs.io/en/latest/
-
 autoapi_dirs = ["../suricata_check"]
 autoapi_options = [
     "members",
@@ -106,3 +88,12 @@
 # https://www.sphinx-doc.org/en/master/usage/extensions/viewcode.html
 
 viewcode_line_numbers = True
+
+# -- Options for sphinx-sitemap    -------------------------------------------------
+# https://sphinx-sitemap.readthedocs.io/en/latest/index.html
+
+html_baseurl = "https://suricata-check.teuwen.net/"
+sitemap_url_scheme = "{link}"
+sitemap_locales = ["en"]
+sitemap_excludes = ["search.html", "genindex.html"]
+html_extra_path = ["robots.txt"]

docs/contributing.md

@@ -1,2 +1,8 @@
+---
+myst:
+    html_meta:
+        "description lang=en": "suricata-check is a command line utility to provide feedback on Suricata rules to by detecting issues through static analysis."
+        "keywords": "Suricata, suricata-check, Python, Contributing"
+---
 ```{include} ../CONTRIBUTING.md
 ```

docs/make.bat

@@ -25,7 +25,7 @@ if errorlevel 9009 (
 
 if "%1" == "" goto help
 
-%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% -v -T
 goto end
 
 :help

docs/readme.md

@@ -1,2 +1,8 @@
+---
+myst:
+    html_meta:
+        "description lang=en": "suricata-check is a command line utility to provide feedback on Suricata rules to by detecting issues through static analysis."
+        "keywords": "Suricata, rules, ruleset, suricata-check"
+---
 ```{include} ../README.md
 ```

docs/robots.txt

@@ -0,0 +1,3 @@
+User-agent: *
+
+Sitemap: https://suricata-check.teuwen.net/sitemap.xml

docs/static/js/script.js

@@ -0,0 +1,33 @@
+var script = document.createElement("script");
+
+script.setAttribute("async", "");
+script.setAttribute("src", "https://umami.suricata-check.teuwen.net/script.js");
+script.setAttribute("data-website-id", "122e579a-0a3b-41c2-8b20-9a3cbec8dc76");
+script.setAttribute("data-do-not-track", "true");
+script.setAttribute("data-domains", "suricata-check.teuwen.net");
+
+window.addEventListener('load', function () {
+    document.body.appendChild(script);
+
+    (() => {
+        const name = 'internal-link-click';
+        document.querySelectorAll('a').forEach(a => {
+            if (a.host === window.location.host && !a.getAttribute('data-umami-event')) {
+                a.setAttribute('data-umami-event', name);
+                a.setAttribute('data-umami-event-source', window.location.href);
+                a.setAttribute('data-umami-event-target', a.href);
+            }
+        });
+    })();
+
+    (() => {
+        const name = 'outbound-link-click';
+        document.querySelectorAll('a').forEach(a => {
+            if (a.host !== window.location.host && !a.getAttribute('data-umami-event')) {
+                a.setAttribute('data-umami-event', name);
+                a.setAttribute('data-umami-event-source', window.location.href);
+                a.setAttribute('data-umami-event-target', a.href);
+            }
+        });
+    })();
+});

pyproject.toml

@@ -71,6 +71,7 @@ dev = [
   "myst-parser[linkify]>=3.0.1",
   "sphinx-click>=6.0.0",
   "sphinx-autoapi>=3.4.0",
+  "sphinx-sitemap>=2.6.0",
 ]
 
 [project.urls]

requirements.txt

@@ -12,10 +12,11 @@ ruff>=0.4.9
 black>=24.4.2
 pyright>=1.1.367
 sphinx>=7.3.7
-sphinx-rtd-theme>=2.0.0
+sphinx-rtd-theme>=3.0.0
 myst-parser[linkify]>=3.0.1
 sphinx-click>=6.0.0
 sphinx-autoapi>=3.4.0
+sphinx-sitemap>=2.6.0
 numpy>=1.26.4
 pandas>=2.0.3
 scikit-learn==1.5.1

suricata_check/_version.py

@@ -25,7 +25,7 @@ def get_version() -> str:
         try:
             import setuptools_git_versioning
 
-            v = str(setuptools_git_versioning.get_version())
+            v = str(setuptools_git_versioning.get_version(root=SURICATA_CHECK_DIR))
             _logger.debug(
                 "Detected suricata-check version using setuptools_git_versioning: %s", v
             )