Self healing: Parity stuck syncing a specific block number

[itamararjuan]

As part of the active monitoring tools on the machine I think it's imperative to recognize this condition and restart the process once every hour to make sure we are not stuck on a specific block.

When this happenes the log looks similar to this:

2019-05-15 08:27:53 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers     45 KiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    2 req/s,   13 µs
2019-05-15 08:27:58 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers      4 MiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    4 req/s,   13 µs
2019-05-15 08:28:03 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers      4 MiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    5 req/s,   13 µs
2019-05-15 08:28:08 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers      4 MiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    3 req/s,   13 µs
2019-05-15 08:28:13 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers      4 MiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    3 req/s,   13 µs
2019-05-15 08:28:18 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers      4 MiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    4 req/s,   14 µs
2019-05-15 08:28:23 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers      4 MiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    5 req/s,   14 µs
2019-05-15 08:28:28 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers      4 MiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    4 req/s,   15 µs
2019-05-15 08:28:33 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers      8 MiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    3 req/s,   15 µs
2019-05-15 08:28:38 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers    239 KiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    3 req/s,   15 µs
2019-05-15 08:28:43 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers    239 KiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    4 req/s,   14 µs
2019-05-15 08:28:48 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers    239 KiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    4 req/s,   15 µs
2019-05-15 08:28:53 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers    239 KiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    2 req/s,   14 µs
2019-05-15 08:28:58 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers    239 KiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    4 req/s,   14 µs
2019-05-15 08:29:03 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers      7 MiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    2 req/s,   14 µs
2019-05-15 08:29:08 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers     14 MiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    4 req/s,   14 µs
2019-05-15 08:29:13 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers      7 MiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    2 req/s,   15 µs
2019-05-15 08:29:18 UTC Syncing #7761572 0x2497…a811     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #7761079   46/60 peers     11 MiB chain    7 GiB db  0 bytes queue  174 MiB sync  RPC:  0 conn,    4 req/s,   15 µs