Datetime not properly adjusted with timezone for syslog parser

[ptitdoc]

Description of problem:

When using the syslog plugin, I found a discrepancy between two systems that pointed me to a potential timezone bug in log2timeline.

The two analyzed system images are mounted in Linux before being parsed with log2timeline.py (instead of parsing a raw image).

Running pinfo.py shows proper detection of the timezone even if the analyzed system has been mounted in /mnt: CET for system1, UTC for system2.

However if I look at an arbitrary syslog event on the system1, psort output the event with the same timestamp than the original syslog entry (traditionnal syslog entry), showing that the datetime has not been adjusted to UTC.

Command line and arguments:

$ log2timeline.py --skip-compressed-streams --hashers none system1.plaso /mnt/system1/

$ pinfo.py -v system1.plaso
[...]
           Product version : 20200717
--------------------------------------------------------------------------------

********** System configuration: 35a94712-e9eb-437e-8664-f1b4f269975d **********
                Hostname : system1
        Operating system : Linux
Operating system product : CentOS Linux release 7.4.1708 (Core)
Operating system version : N/A
               Code page : cp1252
         Keyboard layout : N/A
               Time zone : CET
--------------------------------------------------------------------------------

********** Available time zones: 35a94712-e9eb-437e-8664-f1b4f269975d **********
Name :
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------

************* User accounts: 35a94712-e9eb-437e-8664-f1b4f269975d **************
       Username : User directory

[...]

$ head /mnt/system1/var/log/messages
Jul 26 03:25:48 system1 timeseries-rrd-updates: Update error(1): /usr/local/nagios/var/rrd/ctrl-3220-bet/Disk%20Write/Bytes_Disk_Write_/value.rrd: not a simple unsigned integer: '8.31200537659802e+15'

$ psort.py --slice "2020-07-26T01:25:48" system1.plaso -w /tmp/test.out
cat /tmp/test.out
2020-07-26T03:25:48+00:00,Content Modification Time,LOG,Log File,[timeseries-rrd-updates] Update error(1): /usr/local/nagios/var/rrd/ctrl-3220-bet/Disk%20Write/Bytes_Disk_Write_/value.rrd: not a simple unsigned integer: '8.31200537659802e+15',syslog,-,

Source data:

Please provide the source data you used when you experienced the problem.
For publicly available data please provide an URL or path of the source data.

Plaso version:

20200717

Operating system Plaso is running on:

Archlinux

Installation method:

• built from pip in a python3 virtualenv

[ptitdoc]

I'm looking at the logic to understand where it can come from.

TimeElements is created after detecting a traditional syslog record:

plaso/plaso/parsers/syslog.py

Line 343 in 0bb6a19

date_time = dfdatetime_time_elements.TimeElements(

In TimeElements initializer, self._number_of_seconds is set with self._time_zone_offset, however no information on the system timezone has been provided at that point:
https://github.com/log2timeline/dfdatetime/blob/5c0531d0502b429d19e9b148f6f113fe086417be/dfdatetime/time_elements.py#L105

Immediately afterward, the property is_local_time is set:

plaso/plaso/parsers/syslog.py

Line 345 in 0bb6a19

date_time.is_local_time = True

This property seems to used there:

plaso/plaso/containers/time_events.py

Line 30 in 98e302b

if date_time.is_local_time and time_zone:

But apparently DateTimeValuesEvent is never initialized with a timezone.

In other plugins, the KB timezone is retrieved from the parser mediator, is it what is missing ?

plaso/plaso/parsers/winjob.py

Line 250 in f74f649

time_zone=parser_mediator.timezone)

[joachimmetz]

In other plugins, the KB timezone is retrieved from the parser mediator, is it what is missing ?

Thx, was about to update this. Yes that is what is missing, fix pending

[ptitdoc]

Are there maybe other affected plugins ? Timezones are critical and I always screw up when I deal with it manually. Thanks for the reactivity anyway.

[joachimmetz]

Are there maybe other affected plugins ? Timezones are critical and I always screw up when I deal with it manually. Thanks for the reactivity anyway.

We are working on changes to preserve the time zone information (#910) so that corrections can be done easier after processing.

Timezones are critical and I always screw up when I deal with it manually.

Timezones are very complicated, e.g. should Nov 25, 2020 02:00 (AM) or 03:00 (AM) be parsed as CEST or as CET ?

[ptitdoc]

See, we always screw-up when dealing manually with timezones. November is definitively Winter time ;-)

[joachimmetz]

November is definitively Winter time ;-)

not really, kind of depends on which hemisphere you are ;)

[ptitdoc]

True, it is inverted in South Africa. What a mess.

[joachimmetz]

Changes merged, added follow up action item to assess potential other affected parsers / plugins #3287