Which lib is used for unlisted -mabi=rv32imac_zicsr_zifencei_zba_zbb?

[ZhekaS]

I am looking to compile a project for architecture supporting Zicsr, Zifencei and Zba/Zba. The corresponding march string would be something like rv32imac_zicsr_zifencei_zba_zbb. I am looking at the output of riscv-none-elf-gcc --print-multi-lib:

.;
rv32e/ilp32e;@march=rv32e@mabi=ilp32e
rv32ea/ilp32e;@march=rv32ea@mabi=ilp32e
rv32em/ilp32e;@march=rv32em@mabi=ilp32e
rv32eac/ilp32e;@march=rv32eac@mabi=ilp32e
rv32emac/ilp32e;@march=rv32emac@mabi=ilp32e
rv32i/ilp32;@march=rv32i@mabi=ilp32
rv32ia/ilp32;@march=rv32ia@mabi=ilp32
rv32im/ilp32;@march=rv32im@mabi=ilp32
rv32if_zicsr/ilp32f;@march=rv32if_zicsr@mabi=ilp32f
rv32ifd_zicsr/ilp32d;@march=rv32ifd_zicsr@mabi=ilp32d
rv32iaf_zicsr/ilp32f;@march=rv32iaf_zicsr@mabi=ilp32f
rv32iafd_zicsr/ilp32d;@march=rv32iafd_zicsr@mabi=ilp32d
rv32imf_zicsr/ilp32f;@march=rv32imf_zicsr@mabi=ilp32f
rv32imfd_zicsr/ilp32d;@march=rv32imfd_zicsr@mabi=ilp32d
rv32iac/ilp32;@march=rv32iac@mabi=ilp32
rv32imafc_zicsr/ilp32f;@march=rv32imafc_zicsr@mabi=ilp32f
rv32imafdc_zicsr/ilp32d;@march=rv32imafdc_zicsr@mabi=ilp32d
rv64i/lp64;@march=rv64i@mabi=lp64
rv64ia/lp64;@march=rv64ia@mabi=lp64
rv64im/lp64;@march=rv64im@mabi=lp64
rv64if_zicsr/lp64f;@march=rv64if_zicsr@mabi=lp64f
rv64ifd_zicsr/lp64d;@march=rv64ifd_zicsr@mabi=lp64d
rv64iaf_zicsr/lp64f;@march=rv64iaf_zicsr@mabi=lp64f
rv64iafd_zicsr/lp64d;@march=rv64iafd_zicsr@mabi=lp64d
rv64imf_zicsr/lp64f;@march=rv64imf_zicsr@mabi=lp64f
rv64iac/lp64;@march=rv64iac@mabi=lp64
rv64imac/lp64;@march=rv64imac@mabi=lp64
rv64imafc_zicsr/lp64f;@march=rv64imafc_zicsr@mabi=lp64f
rv64imafdc_zicsr/lp64d;@march=rv64imafdc_zicsr@mabi=lp64d

None of these list either Zifencei or Zba/Zba extensions. Does it mean these aren't supported (on the prebuilt release at least) or I am missing something?

[ilg-ul]

Good question. I would say that the _zifencei_zba_zbb are not used for libraries, so passing -mabi=rv32imac_zicsr_zifencei_zba_zbb would use the default rv32imac/ilp32 libraries.

[TommyMurphyTM1234]

I imagined it might be the case, for zifencei at least, as it is only relevant for some ...exotic functionality. But Zba/Zbb are for bit manipulation, and I guess libraries might benefit from using it in some cases.

The problem that @ilg-ul and anybody else building binary RISC-V toolchains is that it's almost impossible at this stage to build multilibs for absolutely every possible combination of extensions. In fact, I think that it may actually be impossible right now due to limitations within the riscv-gnu-toolchain build scripts/Makefiles. And even if it was possible the resulting binary toolchain distro would be absolutely humungous.

[ZhekaS]

Yeah, I get it... perhaps multiple distros could do though.. for example split rv32 vs rv64. It is probably documented somewhere how to build from sources for a specific config, I am yet to find it :)

[ilg-ul]

it's almost impossible at this stage to build multilibs for absolutely every possible combination of extensions.

That's correct. The goal of the current multilib selection is to cover as many combinations as possible, but it is not possible to cover every possible combination, and some of those covered might lack some optimisations.

If you want to squeeze the very last cycle from your device, I suggest you build those library functions that would benefit from the new extensions together with your application.

Perhaps at a later time I'll consider a source package with a fork of newlib and all the build options to allow an easy integrating into applications.

[TommyMurphyTM1234]

Yeah, I get it... perhaps multiple distros could do though.. for example split rv32 vs rv64. It is probably documented somewhere how to build from sources for a specific config, I am yet to find it :)

Not sure if you mean xPack or the "generic"/upstream riscv-gnu-toolchain approach but...

For xPack you can tweak this script to modify the list of multilibs built:

• riscv-none-elf-gcc-xpack/build-assets/scripts/versioning.sh

  Lines 139 to 237 in 89586bd

  	# Releases from 14.2.*-2 up (September 2024) use the Debian list.
  	if [ "${XBB_APPLICATION_WITHOUT_MULTILIB:-""}" != "y" ]
  	then
  	if [ "${XBB_IS_DEVELOPMENT}" != "y" ]
  	then
  	# Possible future 'light' releases should include:
  	# - rv32imac, rv64imafdc: for QEMU tests
  	# - rv32imac: for SiFive HiFive1 (16K RAM only!)
  	# - rv32ec: for the small WCH CH32V003
  	# - rv32imac, rv32imacf: for WCH QingKe V4 series (xw!)
  	# like CH32V20[38]*, CH32V30[357]*
  	# https://www.wch-ic.com/downloads/QingKeV4_Processor_Manual_PDF.html
  	# https://codeberg.org/20-100/Awesome_RISC-V/src/branch/master/RISC-V_MCU_development_boards.pdf
  	# https://www.gigadevice.com/product/mcu/risc-v
  	# https://www.renesas.com/us/en/products/microcontrollers-microprocessors/risc-v/product-selector
  	# https://products.espressif.com/#/product-selector?language=en&names=
  	# --with-arch=rv64imafdc
  	# Debian 12 $ riscv64-unknown-elf-gcc -print-multi-lib
  	# rv32e/ilp32e;@march=rv32e@mabi=ilp32e
  	# rv32ea/ilp32e;@march=rv32ea@mabi=ilp32e
  	# rv32em/ilp32e;@march=rv32em@mabi=ilp32e
  	# rv32eac/ilp32e;@march=rv32eac@mabi=ilp32e
  	# rv32emac/ilp32e;@march=rv32emac@mabi=ilp32e
  	# rv32i/ilp32;@march=rv32i@mabi=ilp32
  	# rv32ia/ilp32;@march=rv32ia@mabi=ilp32
  	# rv32im/ilp32;@march=rv32im@mabi=ilp32
  	# rv32if/ilp32f;@march=rv32if@mabi=ilp32f
  	# rv32ifd/ilp32d;@march=rv32ifd@mabi=ilp32d
  	# rv32iaf/ilp32f;@march=rv32iaf@mabi=ilp32f
  	# rv32iafd/ilp32d;@march=rv32iafd@mabi=ilp32d
  	# rv32imf/ilp32f;@march=rv32imf@mabi=ilp32f
  	# rv32imfd/ilp32d;@march=rv32imfd@mabi=ilp32d
  	# rv32iac/ilp32;@march=rv32iac@mabi=ilp32
  	# rv32imac/ilp32;@march=rv32imac@mabi=ilp32
  	# rv32imafc/ilp32f;@march=rv32imafc@mabi=ilp32f
  	# rv32imafdc/ilp32d;@march=rv32imafdc@mabi=ilp32d
  	# rv64i/lp64;@march=rv64i@mabi=lp64
  	# rv64ia/lp64;@march=rv64ia@mabi=lp64
  	# rv64im/lp64;@march=rv64im@mabi=lp64
  	# rv64if/lp64f;@march=rv64if@mabi=lp64f
  	# rv64ifd/lp64d;@march=rv64ifd@mabi=lp64d
  	# rv64iaf/lp64f;@march=rv64iaf@mabi=lp64f
  	# rv64iafd/lp64d;@march=rv64iafd@mabi=lp64d
  	# rv64imf/lp64f;@march=rv64imf@mabi=lp64f
  	# rv64iac/lp64;@march=rv64iac@mabi=lp64
  	# rv64imac/lp64;@march=rv64imac@mabi=lp64
  	# rv64imafc/lp64f;@march=rv64imafc@mabi=lp64f
  	# ./multilib-generator rv32e-ilp32e--c rv32ea-ilp32e--m rv32em-ilp32e--c rv32eac-ilp32e-- rv32emac-ilp32e-- rv32i-ilp32--c rv32ia-ilp32--m rv32im-ilp32--c rv32if-ilp32f-rv32ifd-c rv32iaf-ilp32f-rv32imaf,rv32iafc-d rv32imf-ilp32f-rv32imfd-c rv32iac-ilp32-- rv32imac-ilp32-- rv32imafc-ilp32f-rv32imafdc- rv32ifd-ilp32d--c rv32imfd-ilp32d--c rv32iafd-ilp32d-rv32imafd,rv32iafdc- rv32imafdc-ilp32d-- rv64i-lp64--c rv64ia-lp64--m rv64im-lp64--c rv64if-lp64f-rv64ifd-c rv64iaf-lp64f-rv64imaf,rv64iafc-d rv64imf-lp64f-rv64imfd-c rv64iac-lp64-- rv64imac-lp64-- rv64imafc-lp64f-rv64imafdc- rv64ifd-lp64d--m,c rv64iafd-lp64d-rv64imafd,rv64iafdc- rv64imafdc-lp64d--
  	# Default rv32imac-ilp32--
  	XBB_GCC_MULTILIB_LIST=${XBB_APPLICATION_GCC_MULTILIB_LIST:-"\
  	rv32e-ilp32e--c \
  	rv32ea-ilp32e--m \
  	rv32em-ilp32e--c \
  	rv32eac-ilp32e-- \
  	rv32emac-ilp32e-- \
  	\
  	rv32i-ilp32--c \
  	rv32ia-ilp32--m \
  	rv32im-ilp32--c \
  	rv32if-ilp32f-rv32ifd-c \
  	rv32iaf-ilp32f-rv32imaf,rv32iafc-d \
  	rv32imf-ilp32f-rv32imfd-c \
  	rv32iac-ilp32-- \
  	rv32imac-ilp32-- \
  	rv32imafc-ilp32f-rv32imafdc- \
  	rv32ifd-ilp32d--c \
  	rv32imfd-ilp32d--c \
  	rv32iafd-ilp32d-rv32imafd,rv32iafdc- \
  	rv32imafdc-ilp32d-- \
  	\
  	rv64i-lp64--c \
  	rv64ia-lp64--m \
  	rv64im-lp64--c \
  	rv64if-lp64f-rv64ifd-c \
  	rv64iaf-lp64f-rv64imaf,rv64iafc-d \
  	rv64imf-lp64f-rv64imfd-c \
  	rv64iac-lp64-- rv64imac-lp64-- \
  	rv64imafc-lp64f-rv64imafdc- \
  	rv64ifd-lp64d--m,c \
  	rv64iafd-lp64d-rv64imafd,rv64iafdc- \
  	rv64imafdc-lp64d-- \
  	"}
  	echo
  	echo "Use Debian multi-libs: ${XBB_GCC_MULTILIB_LIST}"
  	else
  	# Short list used during development to save time.
  	XBB_GCC_MULTILIB_LIST=${XBB_APPLICATION_GCC_MULTILIB_LIST:-"\
  	rv32emac-ilp32e-- \
  	rv32imac-ilp32-- \
  	rv64imac-lp64-- \
  	"}
  	fi
  	fi
  	fi

and then use the usual build instructions:

• https://xpack-dev-tools.github.io/riscv-none-elf-gcc-xpack/docs/developer/

For riscv-gnu-toolchain:

# Assuming a recent Ubuntu distro
git clone https://github.com/riscv-collab/riscv-gnu-toolchain
sudo ./github/setup-apt.sh
./configure --prefix=`pwd`/installed-tools --with-arch=rv32imac_zicsr_zifencei_zba_zbb --with-abi=ilp32
make 

# Or alternatively...
# https://github.com/riscv-collab/riscv-gnu-toolchain?tab=readme-ov-file#build-with-customized-multi-lib-configure
# make distclean && rm -rf installed-tools
./configure --prefix=`pwd`/installed-tools --with-arch=rv32imac --with-abi=ilp32 --with-multilib-generator="rv32imac_zicsr_zifencei_zba_zbb-ilp32--; <other-multilibs-if-necessary>"
make

[ilg-ul]

perhaps multiple distros could do though..

Good luck with this. The RISC-V Foundation does not provide any guidelines with this, on the contrary, they encourage the creation of new extensions, even of vendor extensions, and it is no wonder that each toolchain maintainer chooses a particular set.

I initially tried an exhaustive approach, but it proved unrealistic. Therefore I recently decided to restart and follow the Debian configuration. These days I added two more combinations, for some of the very small devices. If there are good reasons I can add more, but I won't go back to the combinatorial craziness.

[TommyMurphyTM1234]

perhaps multiple distros could do though..

Good luck with this. The RISC-V Foundation does not provide any guidelines with this, on the contrary, they encourage the creation of new extensions, even of vendor extensions, and it is no wonder that each toolchain maintainer chooses a particular set.

Plus - in my opinion and experience (and I suspect that @ilg-ul will agree) embedded/bare-metal class devices are second class citizens as far as a lot of the RISC-V community is concerned with much more focus and attention on OS/Linux capable platforms.

[ilg-ul]

second class citizens

Second class? You're too kind. The initial core design team had zero interest in working on a bare-metal embedded specification, and even today there is no standard RISC-V microcontroller profile, no standard modern interrupt controller, no standard Embedded ABI, no standard system software (like Arm CMSIS), etc.

[ZhekaS]

Ok, after playing around with multilib-generator and building different multi-lib toolchain I have a follow-up question. When we specify something like -march=*_zicsr - does it even affect anything in the code compiled with the resulting toolchain? I mean, binutils GAS should recognize the CSR instructions and names, but what about GCC and it's libraries? Are they affected in any way?

[TommyMurphyTM1234]

As a (possible incorrect) opinion, I would say that _zicsr only enables the specific instructions in the assembler.

That's my understanding too. I.e. that specifying zicsr in the architecture string enables the use of the instructions specified by this extension so that you can use them in your code and the toolchain won't complain about them - but I very much doubt that anything in the toolchain will unilaterally generate CSR access instructions. But I guess that it's feasible that some toolchain bundled library could contain code that uses these instructions if the zicsr extension is "enabled"...

[ZhekaS]

Indeed.. it looks like code compiled with -march=rv32imac_zicsr_zifencei_zba_zbb (with multilib not having this exact config) is picking the library corresponding to rv32imac_zba_zbb, and it would accept assembly CSR instructions and fence.i. If compiled with -march=rv32imac_zba_zbb only, then the assembler will throw errors on these instructions.
I wonder why this behavior is not the same as with, say, Smrnmi extension, which does not need explicit march string. I can only guess, that it is because this extension is merely adding few CSR names to be recognized, but not new instructions.

[TommyMurphyTM1234]

I wonder why this behavior is not the same as with, say, Smrnmi extension, which does not need explicit march string. I can only guess, that it is because this extension is merely adding few CSR names to be recognized, but not new instructions.

It does - namely the nmret instruction.

[ZhekaS]

Right... well, my guess is wrong then :D Zifencei is adding a single instruction, but still needs to to be specified to be supported.

[TommyMurphyTM1234]

Right... well, my guess is wrong then :D Zifencei is adding a single instruction, but still needs to to be specified to be supported.

Yes, and Zifencei seems comparable to smrnmi in that the single instruction added by each extension is (I believe) one that will only be coded explicitly by the user and not generated unilaterally by the compiler toolchain. I have no idea why there isn't a Z[i?]smrnmi option for the architecture string and this situation seems inconsistent and anomalous. I couldn't find any clarification by searching and reading material that I could find related to these extensions. Some aspects of RISC-V are unfortunately very confusing in my opinion...