Idea - Group output presentations by service payload (Data download only)

[rnd-ash]

OK, so this is something I've done for OVD's CBF Parser, and has dramatically reduced loading time of JSON and the mess of 1000s of data download functions present within the CBF.

It seems DAS has service groups, in which it knows if it sends a particular payload to the ECU, multiple services can be executed at once. For example, reading the majority of this table below can be executed with 1 payload sent to EGS52:

On the other hand, CBF seems to contain 1 output param per function, meaning there is no grouping of data based on what data is sent to the ECU, which can end up creating 20-30 different functions that all end up doing the same thing.

I've been able to group services together by comparing the payload of download functions. This has reduced by JSON size by approx 25%, since repeated data about the payload is no longer written to the files. Also, this can be reversed in software to re-construct the individual functions if needed.

Here is a gist of CRD ECU showing how things are grouped:
https://gist.github.com/rnd-ash/4a7d8edde72be61ebf939f0f984d2421

[jglim]

At this time, I would prefer to preserve as much original data in the CBF, therefore lossy coalescing of identical commands for parser performance does not fit the current strategy. However it is still a good idea, and it will be very useful in reducing the load on the already slow canbus; I'll still look into fitting this idea somewhere in the project.

On performance, c32s and its dependents like Vediamo have many paths for optimization which is unavailable to us:

• The entire file does not need to be loaded in a single pass; content can be lazy-loaded in stages
• For typical usage, only the variant's communication parameters have to be loaded first. When the variant is identified, its specific underlying content can then be loaded.

---

Still, I put the project through a profiler and worked on some of the major bottlenecks. Prior to optimization the performance looked something like:

CRD3 : 2793ms
CRD3S2 : 985ms
VGSNAG2 : 51ms
IC_204 : 181ms

In the main branch now, load times have been shortened to:

CRD3 : 570ms
CRD3S2 : 234ms
VGSNAG2 : 33ms
IC_204 : 90ms

The CRD3 file is about 10MB and should be representative of most large CBF files. The above timings include loading the entire CBF (so that they can be optionally serialized later), and should be tolerable for now.