add UCB bonus to planning progress explorer

[tomsilver]

Git commit hashes seen in results/:
cada9828a9876217ae926523440ae5d0697e0ef0


AGGREGATED DATA OVER SEEDS:
                                                         NUM_TRAIN_TASKS    NUM_SOLVED AVG_NUM_PREDS AVG_TEST_TIME AVG_NODES_CREATED     LEARNING_TIME   AVG_REF_COST  NUM_SEEDS
EXPERIMENT_ID                                      CYCLE                                                                                                                        
ball_and_cup_sticky_table-planning_progress_exp... 0        50.00 (0.00)   4.80 (1.60)  19.92 (1.95)   0.42 (0.01)      56.52 (5.82)       1.12 (0.08)    0.96 (0.50)          5
                                                   1        50.00 (0.00)   3.40 (1.36)  22.56 (2.33)   0.42 (0.02)      64.00 (6.24)     26.34 (48.34)    1.35 (0.54)          5
                                                   2        50.00 (0.00)   4.80 (1.17)  20.64 (2.45)   0.46 (0.05)      58.56 (6.56)    122.37 (83.28)    0.86 (0.26)          5
                                                   3        50.00 (0.00)   5.20 (1.33)  19.92 (3.09)   0.44 (0.01)      56.52 (8.46)    218.38 (62.96)    0.80 (0.26)          5
                                                   4        50.00 (0.00)   4.60 (2.65)  21.36 (6.32)   0.44 (0.00)     60.60 (17.81)    314.54 (68.15)    1.44 (1.29)          5
                                                   5        50.00 (0.00)   4.60 (1.85)  21.84 (3.90)   0.48 (0.06)     61.38 (10.42)    437.95 (64.60)    0.97 (0.32)          5
                                                   6        50.00 (0.00)   7.60 (2.06)  15.84 (3.51)   0.45 (0.02)     44.98 (10.40)    553.18 (46.26)    0.55 (0.22)          5
                                                   7        50.00 (0.00)   6.00 (1.90)  18.48 (3.09)   0.45 (0.03)      52.44 (8.62)    641.74 (36.77)    0.73 (0.31)          5
                                                   8        50.00 (0.00)   8.00 (1.41)  14.88 (2.23)   0.45 (0.03)      42.26 (5.82)    743.90 (59.61)    0.49 (0.10)          5
                                                   9        50.00 (0.00)   7.20 (3.71)  15.36 (4.45)   0.46 (0.04)      37.00 (2.99)    854.53 (56.49)    0.42 (0.05)          5
                                                   None     50.00 (0.00)   0.00 (0.00)  24.72 (0.59)     nan (nan)      35.96 (0.78)       0.32 (0.04)      nan (nan)          5
ball_and_cup_sticky_table-planning_progress_exp... 0        50.00 (0.00)   4.80 (1.60)  19.92 (1.95)   0.40 (0.00)      56.52 (5.82)       0.95 (0.04)    0.96 (0.50)          5
                                                   1        50.00 (0.00)   3.60 (1.20)  22.32 (2.23)   0.41 (0.03)      63.32 (6.00)     30.40 (56.85)    1.22 (0.44)          5
                                                   2        50.00 (0.00)   5.00 (0.89)  20.16 (1.59)   0.44 (0.02)      57.20 (4.11)    126.16 (90.69)    0.79 (0.15)          5
                                                   3        50.00 (0.00)   4.60 (1.85)  21.36 (3.97)   0.44 (0.00)     60.60 (10.80)    221.00 (60.04)    1.04 (0.53)          5
                                                   4        50.00 (0.00)   5.60 (2.80)  19.68 (6.81)   0.44 (0.01)     55.84 (19.35)    312.67 (65.97)    1.28 (1.34)          5
                                                   5        50.00 (0.00)   3.80 (2.14)  22.80 (4.02)   0.46 (0.05)     64.10 (10.91)    393.38 (65.72)    1.26 (0.47)          5
                                                   6        50.00 (0.00)   7.60 (1.62)  15.84 (2.56)   0.45 (0.02)      44.98 (7.68)    486.20 (82.55)    0.52 (0.13)          5
                                                   7        50.00 (0.00)   6.60 (2.06)  17.76 (3.34)   0.44 (0.01)      50.40 (9.15)    577.04 (99.61)    0.67 (0.32)          5
                                                   8        50.00 (0.00)   6.40 (3.38)  16.80 (4.55)   0.44 (0.00)      40.54 (5.25)   669.52 (114.24)    0.48 (0.09)          5
                                                   9        50.00 (0.00)   6.40 (3.61)  16.80 (4.62)   0.42 (0.02)      41.08 (8.97)   778.84 (133.32)    0.50 (0.16)          5
                                                   None     50.00 (0.00)   0.00 (0.00)  24.72 (0.59)     nan (nan)      35.96 (0.78)       0.28 (0.02)      nan (nan)          5
grid_row-planning_progress_explore_with_bonus      0        50.00 (0.00)   3.40 (1.36)  13.10 (0.92)   1.22 (0.26)     105.86 (0.55)      11.98 (0.40)  20.68 (15.27)          5
                                                   1        50.00 (0.00)   3.80 (2.14)  11.90 (1.91)   1.09 (0.21)     105.14 (1.14)    182.92 (92.85)  21.27 (16.07)          5
                                                   2        50.00 (0.00)   6.40 (3.14)   9.60 (3.20)   1.02 (0.14)     103.76 (1.92)    431.16 (78.59)  15.67 (17.67)          5
                                                   3        50.00 (0.00)   8.80 (0.75)   7.90 (0.86)   1.07 (0.06)     102.74 (0.52)    661.22 (98.08)    5.76 (0.49)          5
                                                   4        50.00 (0.00)   9.60 (0.80)   5.90 (1.80)   0.94 (0.11)     101.54 (1.08)    927.37 (84.06)    5.26 (0.52)          5
                                                   5        50.00 (0.00)  10.00 (0.00)   5.10 (0.20)   0.89 (0.02)     101.06 (0.12)   1164.63 (99.73)    5.00 (0.00)          5
                                                   6        50.00 (0.00)  10.00 (0.00)   5.30 (0.40)   0.92 (0.04)     101.18 (0.24)  1407.11 (128.59)    5.00 (0.00)          5
                                                   7        50.00 (0.00)  10.00 (0.00)   5.10 (0.20)   0.90 (0.04)     101.06 (0.12)  1688.86 (167.10)    5.00 (0.00)          5
                                                   8        50.00 (0.00)  10.00 (0.00)   5.30 (0.40)   0.93 (0.02)     101.18 (0.24)  1967.43 (174.90)    5.00 (0.00)          5
                                                   9        50.00 (0.00)  10.00 (0.00)   5.10 (0.20)   0.92 (0.04)     101.06 (0.12)  2240.40 (198.20)    5.00 (0.00)          5
                                                   None     50.00 (0.00)   0.00 (0.00)  25.00 (0.00)     nan (nan)     121.00 (0.00)       1.02 (0.03)      nan (nan)          5
grid_row-planning_progress_explore_without_bonus   0        50.00 (0.00)   3.40 (1.36)  13.10 (0.92)   1.15 (0.19)     105.86 (0.55)      11.88 (0.41)  20.68 (15.27)          5
                                                   1        50.00 (0.00)   3.80 (2.14)  11.90 (1.91)   1.05 (0.15)     105.14 (1.14)   186.81 (101.04)  21.27 (16.07)          5
                                                   2        50.00 (0.00)   6.40 (3.14)   9.60 (3.20)   1.01 (0.14)     103.76 (1.92)   418.81 (115.38)  15.67 (17.67)          5
                                                   3        50.00 (0.00)   8.80 (0.75)   7.90 (0.86)   1.04 (0.04)     102.74 (0.52)   638.03 (130.32)    5.76 (0.49)          5
                                                   4        50.00 (0.00)   9.60 (0.80)   5.90 (1.80)   0.94 (0.11)     101.54 (1.08)   881.55 (135.88)    5.26 (0.52)          5
                                                   5        50.00 (0.00)  10.00 (0.00)   5.10 (0.20)   0.90 (0.03)     101.06 (0.12)  1126.88 (146.11)    5.00 (0.00)          5
                                                   6        50.00 (0.00)  10.00 (0.00)   5.30 (0.40)   0.93 (0.04)     101.18 (0.24)  1391.76 (171.35)    5.00 (0.00)          5
                                                   7        50.00 (0.00)  10.00 (0.00)   5.10 (0.20)   0.91 (0.04)     101.06 (0.12)  1660.26 (202.93)    5.00 (0.00)          5
                                                   8        50.00 (0.00)  10.00 (0.00)   5.30 (0.40)   0.91 (0.04)     101.18 (0.24)  1939.40 (232.32)    5.00 (0.00)          5
                                                   9        50.00 (0.00)  10.00 (0.00)   5.10 (0.20)   0.90 (0.02)     101.06 (0.12)  2208.50 (246.59)    5.00 (0.00)          5
                                                   None     50.00 (0.00)   0.00 (0.00)  25.00 (0.00)     nan (nan)     121.00 (0.00)       0.98 (0.02)      nan (nan)          5

[NishanthJKumar]

LGTM! Very nice!

[tomsilver]

It turns out that this PR was actually quite bad. Running 10 seeds is important!

AGGREGATED DATA OVER SEEDS:
                                                         NUM_TRAIN_TASKS   NUM_SOLVED AVG_NUM_PREDS AVG_TEST_TIME AVG_NODES_CREATED    LEARNING_TIME AVG_REF_COST  NUM_SEEDS
EXPERIMENT_ID                                      CYCLE                                                                                                                    
ball_and_cup_sticky_table-planning_progress_exp... 0        50.00 (0.00)  4.40 (1.56)  20.40 (2.08)   0.41 (0.01)      57.55 (5.99)      0.93 (0.04)  1.04 (0.49)         10
                                                   1        50.00 (0.00)  3.90 (2.02)  21.72 (2.81)   0.42 (0.01)     58.05 (10.88)    22.04 (41.00)  1.08 (0.52)         10
                                                   2        50.00 (0.00)  3.60 (1.80)  22.20 (2.70)   0.43 (0.01)     59.41 (10.86)    98.17 (70.42)  1.13 (0.49)         10
                                                   3        50.00 (0.00)  3.60 (2.37)  21.60 (3.39)   0.44 (0.01)     55.55 (11.88)   185.67 (81.74)  1.22 (1.07)         10
                                                   4        50.00 (0.00)  3.30 (2.65)  22.56 (4.94)   0.44 (0.01)     58.29 (16.57)  273.80 (108.55)  1.41 (1.06)         10
                                                   5        50.00 (0.00)  3.40 (2.69)  21.96 (3.40)   0.44 (0.01)     51.90 (14.06)  378.29 (133.83)  0.91 (0.31)         10
                                                   6        50.00 (0.00)  6.10 (2.81)  18.00 (3.94)   0.43 (0.01)     47.61 (10.62)  481.78 (165.32)  0.62 (0.22)         10
                                                   7        50.00 (0.00)  5.70 (2.65)  18.84 (3.72)   0.43 (0.02)     50.08 (10.83)  587.73 (165.65)  0.68 (0.26)         10
                                                   8        50.00 (0.00)  5.40 (3.72)  18.12 (4.34)   0.44 (0.02)      42.40 (6.62)  696.33 (188.65)  0.51 (0.10)         10
                                                   9        50.00 (0.00)  5.20 (4.31)  18.12 (4.99)   0.43 (0.02)      37.97 (4.44)  806.36 (207.60)  0.44 (0.05)         10
                                                   None     50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.50 (0.72)      0.29 (0.02)    nan (nan)         10
111