[multitop] Implement HW rules RFC

[pamaury]

This PR implements the Multitop HW desc rule RFC. It replaces #24791 which is the original implementation.

There is one notable difference between this implementation and the one suggested in the RFC.

RFC suggested implementation

The RFC suggested to build the top description entire out of rules and providers:

# In hw/ip/uart/BUILD:
# This target exports an OpenTitanIpInfo provider
opentitan_ip(
  name = "uart",
  hjson = "//path/to/uart.hjson"
)

# In hw/top_earlgrey/BUILD:
# This target exports an OpenTitanTopInfo provider
opentitan_top(
  name = "top_earlgrey_desc",
  hjson = "//path/to/top_earlgrey.gen.hjson",
  ips = [
    "//hw/ip/uart",
   ...
  ]
)

The information is then used to e.g. build headers:

# In hw/top:
opentitan_ip_c_headers(
  name = "uart_c_regs",
  top = "//hw/top_earlgrey:top_earlgrey_desc",
  ip = "uart",
)

Problem with this approach

While this approach works fine, we need to consider what happens when we mix this multitop. Suppose that we have a way to selecting a top though a bazel config and we use that to point to the relevant top description:

# In hw/top
alias(
  name = "top_desc",
  actual = select({
    "//hw/top:is_earlgrey": "//hw/top_earlgrey:top_earlgrey_desc",
    "//hw/top:is_darjeeling": "//hw/top_earlgrey:top_darjeeling_desc",
  })
)

opentitan_ip_c_headers(
  name = "uart_c_regs",
  top = "//hw/top:top_desc",   # <-- NOTE: not pointing to earlgrey anymore
  ip = "uart",
)

This works fine because both Earlgrey and Darjeeling have a UART. Now consider an IP block that only exists in Earlgrey or Darjeeling:

opentitan_ip_c_headers(
  name = "dma_c_regs",
  top = "//hw/top:top_desc", 
  ip = "dma",
)

opentitan_ip_c_headers(
  name = "usb_c_regs",
  top = "//hw/top:top_desc", 
  ip = "usb",
)

We run into an issue: the rule opentitan_ip_c_headers cannot work for ip = dma if run on earlgrey because Earlgrey does not have a DMA and therefore does not even know where the DMA hjson file is. One solution could be to output an empty header file but this would just push the problem onto the users: if they try to use it and depend on DMA register offsets, they will fail to compile. Therefore, the proper definition should be:

opentitan_ip_c_headers(
  name = "dma_c_regs",
  top = "//hw/top:top_desc", 
  ip = "dma",
  target_compatible_with = opentitan_require_top("darjeeling") # Express that we want darjeeling
)

opentitan_ip_c_headers(
  name = "usb_c_regs",
  top = "//hw/top:top_desc", 
  ip = "usb",
  target_compatible_with = opentitan_require_top("earlgrey") # Express that we want earlgrey
)

Where opentitan_require_top is some unspecified-as-of-yet macro, for example:

def opentitan_require_top(topname):
  return select({
    "//hw/top:is_{}".format(topname): [],
    "//conditions:default": ["@platforms//:incompatible"],
  })

But now run into another issue: we have to manually specify for each IP the list of tops that it is compatible with! This means that adding a top require adding 35+ conditions everywhere! Worse, if we change the definition of a top (e.g. Darjeeling which is supposed to be easily tweakable) then we have to change this as well. This defeats the entire point of have a top description generated by bazel.

Formalizing the problem

Thinking back, what we really would like to write is this:

opentitan_ip_c_headers(
  name = "dma_c_regs",
  top = "//hw/top:top_desc", 
  ip = "dma",
  target_compatible_with = opentitan_require_ip("dma") # Express that we only accept tops with the DMA IP
)

where opentitan_require_ip is a magical macro that somehow expands to the list of tops with the DMA block. In order to write such a macro, we need to remember that macros are expanded during Bazel's loading phase, i.e. before the build graph is even constructed. During the loading phase, the only "symbols" available to macros are:

• other macros defined in .bzl files
• variables defines in .bzl files

It is clear that in order to work, opentitan_require_ip needs access to a (possibly scaled-down) description of every top. Therefore the conclusion is that the top descriptions needs to be available in the loading phase. With the rule+provider approach from the RFC, it is only available in the analysis phase.

Solution

The solution proposed in the RFC is therefore to still use the idea of the RFC but at the loading phase. Here opentitan_ip and opentitan_top become macros returning struct. Those need to be placed in newly created .bzl files to be usable in the loading phase:

# In hw/ip/uart/defs.bzl:  (not a BUILD file)

UART = opentitan_ip(
  name = "uart",
  hjson = "//path/to/uart.hjson"
)

# In hw/top_earlgrey/defs.bzl:  (not a BUILD file)
load("//hw/ip/uart:defz.bzl", "UART")
EARLGREY = opentitan_top(
  name = "top_earlgrey_desc",
  hjson = "//path/to/top_earlgrey.gen.hjson",
  ips = [
    UART,
   ...
  ]
)

# In hw/top/defs.bzl:
load("//hw/top_earlgrey:defz.bzl", "EARLGREY")
load("//hw/top_darjeeling:defz.bzl", "DARJEELING")

ALL_TOPS = [EARLGREY, DARJEELING]

A simple implementation of those macros is to simply wrap the information in a struct:

def opentitan_ip(name, hjson):
    return struct(
        name = name,
        hjson = hjson,
    )

def opentitan_top(name, hjson, ips):
    return struct(
        name = name,
        hjson = hjson,
        top_lib = top_lib,
        top_ld = top_ld,
        ips = ips,
    )

This allows use to implement opentitan_require_ip simply as follows:

def opentitan_require_ip(ip):
    compatible_tops = []
    for top in ALL_TOPS:
        for _ip in top.ips:
            if _ip.name == ip:
                compatible_tops.append(top.name)
                break

    return select({
        "//hw/top:is_{}".format(top): []
        for top in compatible_tops
    } | {
        "//conditions:default": ["@platform//:incompatible"],
    })

Of course, we do not want to carry those struct around for the analysis phase so we still want to create a target that exports an OpenTitanTopInfo provider as before. This is implemented in the PR and a simplified version of the code is the following:

# In hw/top/BUILD:

# Create one description per top
[
    describe_top(
        name = "top_{}_desc".format(top),
        all_tops = ALL_TOPS,
        top = top,
    )
    for top in ALL_TOP_NAMES
]

# In hw/top/defs.bzl:
def _describe_top(ctx):
    return [
        OpenTitanTopInfo(
            name = ctx.attr.topname,
            hjson = ctx.file.hjson,
            ip_hjson = ctx.attr.ip_hjson, # simplified: ctx.attr.ip_hjson has the wrong "type" and needs a bit of massaging...
        ),
    ]

describe_top_rule = rule(
    implementation = _describe_top,
    doc = """Create a target that provides the description of a top in the form of an OpenTitanTopInfo provider.""",
    attrs = {
        "hjson": attr.label(mandatory = True, allow_single_file = True, doc = "toplevel hjson file generated by topgen"),
        "ip_hjson": attr.label_keyed_string_dict(allow_files = True, doc = "mapping from hjson files to tops"),
        "topname": attr.string(mandatory = True, doc = "Name of the top"),
    },
)


def describe_top(name, all_tops, top):
    """
    Create a target that provides an OpenTitanTopInfo corresponding to the
    requested top.

    - all_tops: list of tops (created by opentitan_top).
    - top: name of the top to use.
    """

    # Details omitted: go through the all_tops struct and extract the relevant information

    describe_top_rule(
        name = name,
        hjson = top_hjson,
        ip_hjson = all_hjson,
        topname = top,
    )

[cfrantz]

Are we also templating BUILD files?

[pamaury]

Yes some (all?) ipgen IP template BUILD file. Well technically they are not templates because their content does not depend on the template but we can't name them just BUILD because then bazel considers them as a package and this can cause all sorts of weird issues potentially. Therefore calling BUILD.tpl seems like an easy way to avoid this issue. Unless you have a better idea on how to handle this?

[matutem]

Will we be able to dispense with the "all_files" target? That may make these BUILD and BUILD.tpl files unnecessary.

[pamaury]

Unfortunately, there was quite a lot of pushback against touching the "all_files" targets so I have given up on touching them for the time being.

[cfrantz]

nit:

if __name__ == '__main__':
    main()

[pamaury]

Ah yes, although I should point out that this script will disappear at the end of the PR (I just forgot to add it yet). Still good practice though.

[cfrantz]

Would this be better as a comprehension? (also, maybe use an integer as the value, rather than empty dict?)

names = {ip.name: 1 for ip in top.ips for top in ALL_TOPS}

[a-will]

Too bad the set is only in bazel nightly, hehe.

[cfrantz]

Agree. Longer term, we probably need to distinguish between chip-specific constants in opentitantool, stick them into something like a hashmap by chip, and add a --chip=... cmdline argument to select which set of constants we care about.

[pamaury]

Yes that's a big issue. I would like to raise it in the SW WG tomorrow, there is clearly some work to be done here. I think the multitop SW PR is a good starting point to generate such a data structure.

[a-will]

One route: I could revive my devicetree generator, and opentitantool could consume its output for a description of the system. That would also leave open support for out-of-tree tops (instead of hard-coding entire known chips).

[a-will]

Out of curiosity, did you already consider a bazel module extension to create a repo that fills out the information from the hjson files? From the description, this starts to sound rather similar to a "foreign build system" or "foreign packaging" sort of problem. The hjson files seem to contain critical information about the build graph, such that it exists outside of bazel's BUILD files.

It looks like this PR goes the route of requiring the user to generate the graph with a separate call, but could we do it on the fly (in a bazel module extension) with what we already have with topgen, reggen, and the hjson files?

[Razer6]

This looks good to me. Thanks.

[pamaury]

The CI failures are unrelated (flaky tests, flaky CW305 bitstream build).