CNN Example - How to input data to the synthesised block

[thundertwonk001]

Hi folks,

HLS4ML noob here! For the CNN example project from HLS4ML's tutorial repo on github < https://github.com/fastmachinelearning/hls4ml-tutorial/blob/main/part6_cnns.ipynb >, I've synthesised the model in Vivado HLS and opened the generated IP in Vivado.

This is probably a very basic question but how do you actually input data into the model?

As there are 3x 16 bit input vectors, I imagine one needs to enter 16 bits of the "Red" part of the pixel, 16 bits of the "Green" part of the pixel, and 16 bits of the "Blue" part of the pixel into the input vector ports, clock in all the pixels like with a raster, and drive the ready and done (etc.) signals like regular AXI transactions, but I don't know if there's a normal way to get an image into a CNN when doing it embedded, or if there's just a known way with these sorts of cores.

Thanks in advance,
Roland

[dimdano]

The model accepts 3x16-bit input vectors using AXI4 interface. If there is a single input port, you basically pack the RGB pixel data into a 48-bit input. Set ap_start to high. Then you drive the input in synchronization with the clock signal (ap_clk). You also need to assert tvalid signals to '1' to indicate that valid data is ready on the input ports.

Data transfer occurs on every clock cycle where both tvalid (from your side) and tready (from the CNN IP block) are high, as per the AXI4-Stream handshaking protocol. On each clock cycle where tvalid and tready are both high, the current input data can be consumed and you can provide the next pixel's data on the following cycle. You repeat this until all pixel data has been sent to the CNN. The output is ready when you see an ap_done signal going high or a tvalid signal on the output data port, depending on the design.

[thundertwonk001]

Thanks for the response, your answer makes a lot of sense, and glad to see I wasn't too far away. I'll try it out in the simulator!

[thundertwonk001]

Yep, exactly as you said, the simulation works very well. Thanks for your help!

[DrBwts]

Can I ask how you managed to get the IP into Vivado in the first place?

There appears to be a step missing in the tutorials, even though I explicitly asked for an IP to be exported & the folders appear in the directory I have been unsuccessful importing it into Vivado.

[thundertwonk001]

I made a walk through guide as I went along but it's on my work computer, so I'll get back to you on Monday

…

On Sat, 1 Feb 2025, 15:56 DrBwts, ***@***.***> wrote: Can I ask how you managed to get the IP into Vivado in the first place? There appears to be a step missing in the tutorials, even though I explicitly asked for an IP to be exported & the folders appear in the directory I have been unsuccessful importing it into Vivado. — Reply to this email directly, view it on GitHub <#1173 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOUUSHZOIW5YHTV3UHRZSTL2NTVCZAVCNFSM6AAAAABVW2ARU2VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMBSGY4DCOA> . You are receiving this because you authored the thread.Message ID: ***@***.*** .com>

[DrBwts]

thanks it'll be much appreciated

[thundertwonk001]

Morning,

Doing < hls_model.build(csim=False, export=True) > in Jupyter doesn't work, so you need to (sort-of) do it in the CLI to allow Vivado HLS to synthesise the model.

In "build_prj.tcl" in /home//<project_path>/ at the top of the file, there should be a short list of options that look something like this:

array set opt {
reset 0
csim 0
synth 1
cosim 0
validation 0
export 1
vsynth 1
fifo_opt 1
}

What do yours say? Set whatever it says in your file to what I have above. I have found that it's important that csim, cosim and validation are not set to 1 - synthesis might work when they're set to 1, but it hasn't ever worked for me.

Also, make sure you have a lot of RAM allocated to the VM before starting the synthesis process (I have 32GB for my VM) otherwise you may encounter an error saying something like: "/tools/Xilinx/Vivado/2019.2/bin/rdiArgs.sh: line 280: 46180 Killed" which means it dies because of lack of RAM.

Finally, before you do synthesise the model, in the same terminal window where you want to trigger the synthesis command, do:

source /tools/Xilinx/Vivado/2019.2/settings64.sh

To launch synthesis, either do so by clicking the green arrow in Vivado HLS, or in the CLI (and in the same directory as the "build_prj.tcl" script above, do:

vivado_hls -f build_prj.tcl

If it does build, there should be a vivado .xpr project at:

/home//<project_path>/solution_1/impl/vhdl/project.xpr (or something similar to this)

Hopefully this helps!