[prim] Remove primgen and replace with virtual cores

[a-will]

Add back prim_pkg enum for legacy support, and mention that it is deprecated. Some IPs reference the enums directly.

[a-will]

Interesting, it looks like mixing with a newer fusesoc might require at least Python 3.9 to work with the other dependencies.

ERROR: Ignored the following versions that require a different python version: 1.0.0 Requires-Python >=3.9,<4.0; 1.1.0 Requires-Python >=3.9
ERROR: Could not find a version that satisfies the requirement jsonschema2md==1.1.0 (from versions: 0.1.0, 0.1.1, 0.2.0.post1, 0.2.1, 0.3.0, 0.4.0, 0.9.0)
ERROR: No matching distribution found for jsonschema2md==1.1.0

[a-will]

Newer fusesoc also has a different directory layout compared to the OT fork. This was the OT fork's layout:

• {--build-root}/
  • sim-vcs/
    • fusesoc-created metadata, Makefile, and file list
  • src/
    • Exported source files

But newer fusesoc has this directory layout:

• {--work-root}/
  • Contains fusesoc-created metadata, Makefile, and file list
  • src/
    • Exported source files

Accommodating the new tree needed fixes to various paths, including CFLAGS definitions in hjson files. In the current example, {work-root} is set to dvsim's {build_dir}/fusesoc-work.

In addition, for ralgen, the position field was needed to be set to prepend for generated cores, else the files would come after the *_env_pkg.sv that would depend on them.

Now, some simulations work, but there is still a gap...

[a-will]

This PR is way out of date now, but since @olofk has released fusesoc 2.4, I think we don't need my random development tag anymore. 2.4 should be able to handle virtual cores as needed.

[olofk]

@a-will I think that depends on the OT requirements. If you are fine with pulling in cores from one vendor during a build, then virtual cores should do the trick.

However, as I understand it, OT wants to be able to select implementation at compile-time, and in that case you still need something like primgen.

With that said, I believe FuseSoC 2.4 should have all the functionality needed to implement what you need. I have done some experimenting implementing the equivalent to primgen as a FuseSoC filter instead of a generator and it looks promising.

Happy to discuss this further.

[a-will]

@a-will I think that depends on the OT requirements. If you are fine with pulling in cores from one vendor during a build, then virtual cores should do the trick.

However, as I understand it, OT wants to be able to select implementation at compile-time, and in that case you still need something like primgen.

With that said, I believe FuseSoC 2.4 should have all the functionality needed to implement what you need. I have done some experimenting implementing the equivalent to primgen as a FuseSoC filter instead of a generator and it looks promising.

Happy to discuss this further.

It's unclear to me if OT actually needs that, though. All of our in-tree top-level fusesoc cores end up binding a specific prim library, and an out-of-tree integrator would almost certainly do the same for their actual chip. The build recipes for synthesis often can't be reused across technologies, and across integrations for the same opentitan IP, the top-level generally uses different RTL, so we end up with independent top-level cores anyway. We'd end up with different YAML for each real target, so sharing that core file across implementations doesn't seem to provide a benefit.

There may be a bit of a rub with gate-level simulation applications, though, especially if you just want to redo the existing block-level simulations with your own prim library. I'd guess that the full integration level still uses its own core file for GLS, but is doing block-level simulation with a different prim library a supported activity? I'm not sure.

We'd probably achieve the original setup's capabilities if only we could provide fusesoc (as a parameter to its invocation) which implementations of virtual cores to include for the specified top-level core file. For example, it could be arguments to provide additional VLNVs to add to the build (using their default target). Then the top-level core file wouldn't need to explicitly pull them in.

[a-will]

For example, it could be arguments to provide additional VLNVs to add to the build (using their default target). Then the top-level core file wouldn't need to explicitly pull them in.

@olofk If that bit above is interesting, the specifying of additional VLNVs would be akin to how hierarchical synthesis flows work. The top-level (and other sub-cores) may have dangling references, but the missing netlists would get specified as additional sources, then linked in during the build. For fusesoc, this would mean writing in a "depends" arrow from the top-level VLNV to the additional VLNVs, and the solver should handle them as though they were written in the core file Edit: Actually, this might be just additional cores thrown into the Requirement, with no need to modify the dependency graph.

[a-will]

Looks like fusesoc 2.4 isn't being used in the sim runs in CI, and something broke with the CI file changes. I guess the API is a bit different for github actions. Maybe s/parameters/inputs?

[marnovandermaas]

Amazing PR! Left a few comments from looking at the code.

[marnovandermaas]

Maybe good to put this in a separate commit, just to separate it from the prepend changes.

[marnovandermaas]

Should this go in the previous commit?

[marnovandermaas]

Will we create an issue to track this?

[marnovandermaas]

This change is similar to a change made in an earlier commit.

[marnovandermaas]

Are these CDC waivers tested in CI? If not, have we tested this ourselves?

[marnovandermaas]

Should this have a todo that this will be removed in the future?

[marnovandermaas]

Also true for other core files in top_earlgrey

[marnovandermaas]

Add todo?

[olofk]

Regarding paths, I haven't checked exactly what the problem is, but I would recommend using --work-root instead of --build-root since that gives you more deterministic behaviour. You might also want to use --system-name to get a fixed name for the build artifacts.

[a-will]

Regarding paths, I haven't checked exactly what the problem is, but I would recommend using --work-root instead of --build-root since that gives you more deterministic behaviour.

That is what I did in this PR: https://github.com/lowRISC/opentitan/pull/23555/files#diff-8ed9c1256f9282d31ab3a7f150e2e95eaae62a546151c7203117aaa70451129e

[a-will]

If FuseSoC uses padring, physical_pads, and top_pkg from Englishbreakfast instead of Earlgrey, that can be ignored because the former symlink to the latter.

I think we should get rid of those symlinks, though. We ought to eliminate duplicate real VLNVs inside the repo. Virtual VLNVs can define common interfaces, and otherwise, englishbreakfast should become a proper top of its own.

The warnings in which FuseSoC uses cores under bazel-opentitan instead of those in the repo is a bit weird; maybe a FUSESOC_IGNORE in bazel-opentitan is missing?

No, the bazel-opentitan symlink is pointing to a directory of symlinks that point to exactly the same set of files. That is a link to bazel's execroot, which includes all the files from the repo + the external repos pulled in and/or generated + the output directory. We cannot create a FUSESOC_IGNORE file there, and it also wouldn't have any practical effect.

If we really wanted to get rid of the confusion, though, we could play with where the symlinks get created. We could create a directory with this structure:

${REPO_TOP}/
    .bazel/
        FUSESOC_IGNORE

Commit that to the repo, then add --symlink_prefix=.bazel/ to the common options. Then, all the symlinks will be created in the .bazel/ folder alongside that FUSESOC_IGNORE, so non-bazel calls to fusesoc won't exhibit these warnings. (bazel calls do not exhibit these warnings, since fusesoc is run in a sandbox, not in the source directory)

In scripts, we probably foolishly use uncontrolled symlink locations to get to output files, though, so those would have to be fixed up.

While I agree that it doesn't make a difference for Earlgrey and Englishbreakfast (for the mentioned reason), I am more concerned that the wrong packages are selected in general.

Is there a general issue here?

So far, nothing raised points to a general issue (except, arguably, that ascentlint should only be used in sign-off flows, since contributors get very low visibility into what's happening in CI with that tool).

[a-will]

I got this this PR to address the problem. olofk/fusesoc#713 This always builds non-cached generators in work_root which seems to be a reasonable solution to me. Intend to follow up with another fix to lock the directory of cached generators before trying to write those, but wanted to get this to you first for comments.

@olofk Thanks for writing that! I created a test release for it (with wrong version numbers, surely) at https://github.com/a-will/fusesoc/releases/tag/2.4.1.dev0%2Ba-will , and the results look good here. 🙂

[a-will]

The ascentlint issues were just more of the missing changes from prim_generic_* to prim_* (in a couple waiver files).

For CI, we're probably very close now. I'll have to check what's happening with the CW305 bitstream--It seems to be failing somehow on the MMI generation, this time in the search for BRAM cells for otp_ctrl.

Even once we get CI to be all green, there will still be issues with the flows that are only activated for sign-off activities (e.g. FPV, CDC, RDC, UNR, coverage collection / waivers, etc.). At least some docs will need to be updated as well.

[olofk]

I got this this PR to address the problem. olofk/fusesoc#713 This always builds non-cached generators in work_root which seems to be a reasonable solution to me. Intend to follow up with another fix to lock the directory of cached generators before trying to write those, but wanted to get this to you first for comments.

@olofk Thanks for writing that! I created a test release for it (with wrong version numbers, surely) at https://github.com/a-will/fusesoc/releases/tag/2.4.1.dev0%2Ba-will , and the results look good here. 🙂

I released FuseSoC 2.4.2 yesterday with this as well as the warning about non-deterministic virtual cores. Could it be that we have now reached the point where you can use a released version of upstream FuseSoC?