Skip to content

Commit

Permalink
bitbake: data_smart: Improve performance for VariableHistory
Browse files Browse the repository at this point in the history
Fixed:
- BBMULTICONFIG = "qemux86-64 qemuarm64" and more than 70 layers in BBLAYERS
$ bitbake -p -P
Check profile.log.processed, the record() cost more than 20 seconds, it is less
than 1 second when multiconfig is not enabled, and there would be the following
error when more muticonfigs are enabled:

Timeout while waiting for a reply from the bitbake server

Don't change the type of loginfo['detail'] or re-assign it can make record()
back to less than 1 second, this won't affect COW since loginfo is a mutable
type.

The time mainly affected by two factors:
1) The number of enabled layers, nearly 1 second added per layer when the
   number is larger than 50.

2) The global var such as USER_CLASSES, about 1 ~ 2 seconds added per layer
   when the layers number is larger than 50.

(Bitbake rev: d251668d9a7a8dd25bd8767efb30d6d9ff8b1ad3)

Signed-off-by: Robert Yang <[email protected]>
Signed-off-by: Richard Purdie <[email protected]>
(cherry picked from commit 0596aa0d5b0e4ed3db11b5bd560f1d3439963a41)
Signed-off-by: Steve Sakoman <[email protected]>
  • Loading branch information
robertlinux authored and sakoman committed Aug 16, 2024
1 parent 0b37512 commit 6fddf66
Showing 1 changed file with 3 additions and 5 deletions.
8 changes: 3 additions & 5 deletions bitbake/lib/bb/data_smart.py
Original file line number Diff line number Diff line change
Expand Up @@ -272,12 +272,9 @@ def record(self, *kwonly, **loginfo):
return
if 'op' not in loginfo or not loginfo['op']:
loginfo['op'] = 'set'
if 'detail' in loginfo:
loginfo['detail'] = str(loginfo['detail'])
if 'variable' not in loginfo or 'file' not in loginfo:
raise ValueError("record() missing variable or file.")
var = loginfo['variable']

if var not in self.variables:
self.variables[var] = []
if not isinstance(self.variables[var], list):
Expand Down Expand Up @@ -336,7 +333,8 @@ def emit(self, var, oval, val, o, d):
flag = '[%s] ' % (event['flag'])
else:
flag = ''
o.write("# %s %s:%s%s\n# %s\"%s\"\n" % (event['op'], event['file'], event['line'], display_func, flag, re.sub('\n', '\n# ', event['detail'])))
o.write("# %s %s:%s%s\n# %s\"%s\"\n" % \
(event['op'], event['file'], event['line'], display_func, flag, re.sub('\n', '\n# ', str(event['detail']))))
if len(history) > 1:
o.write("# pre-expansion value:\n")
o.write('# "%s"\n' % (commentVal))
Expand Down Expand Up @@ -390,7 +388,7 @@ def get_variable_items_files(self, var):
if isset and event['op'] == 'set?':
continue
isset = True
items = d.expand(event['detail']).split()
items = d.expand(str(event['detail'])).split()
for item in items:
# This is a little crude but is belt-and-braces to avoid us
# having to handle every possible operation type specifically
Expand Down

0 comments on commit 6fddf66

Please sign in to comment.