investigating poor overall performance in simulated ball cup sampler learning

[tomsilver]

before hyperparameter changes:

Git commit hashes seen in results/:
7a5b8646457a5cf659da7581f6afba5d8c5c1a3a


AGGREGATED DATA OVER SEEDS:
                                                         NUM_TRAIN_TASKS   NUM_SOLVED AVG_NUM_PREDS AVG_TEST_TIME AVG_NODES_CREATED     LEARNING_TIME AVG_REF_COST  NUM_SEEDS
EXPERIMENT_ID                                      CYCLE                                                                                                                     
ball_and_cup_sticky_table-planning_progress_exp... 0        50.00 (0.00)  4.40 (1.56)  20.40 (2.08)   0.44 (0.02)      57.55 (5.99)       1.26 (0.17)  1.04 (0.49)         10
                                                   1        50.00 (0.00)  3.90 (2.02)  21.72 (2.81)   0.45 (0.01)     58.05 (10.88)     22.69 (40.81)  1.08 (0.52)         10
                                                   2        50.00 (0.00)  3.60 (1.80)  22.20 (2.70)   0.46 (0.02)     59.41 (10.86)     85.03 (74.01)  1.13 (0.49)         10
                                                   3        50.00 (0.00)  3.90 (2.07)  21.84 (3.63)   0.46 (0.02)     55.97 (11.60)    176.02 (77.34)  1.23 (1.01)         10
                                                   4        50.00 (0.00)  3.30 (2.65)  22.56 (4.94)   0.45 (0.03)     57.17 (17.70)   264.67 (103.32)  1.41 (1.06)         10
                                                   5        50.00 (0.00)  3.40 (2.69)  21.96 (3.40)   0.46 (0.02)     51.90 (14.06)   366.57 (129.30)  0.91 (0.31)         10
                                                   6        50.00 (0.00)  5.50 (2.97)  19.20 (4.77)   0.45 (0.01)     50.61 (13.22)   467.50 (160.11)  0.79 (0.46)         10
                                                   7        50.00 (0.00)  6.50 (1.91)  17.64 (3.04)   0.46 (0.02)      48.24 (9.24)   559.30 (160.17)  0.65 (0.25)         10
                                                   8        50.00 (0.00)  6.30 (3.38)  17.04 (4.08)   0.47 (0.03)      42.47 (6.19)   662.35 (189.90)  0.50 (0.10)         10
                                                   9        50.00 (0.00)  6.10 (4.06)  17.04 (4.76)   0.46 (0.02)      38.30 (4.28)   766.97 (223.69)  0.43 (0.04)         10
                                                   None     50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.50 (0.72)       0.30 (0.02)    nan (nan)         10
ball_and_cup_sticky_table-random_nsrts_explore     0        50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.50 (0.72)      9.86 (26.55)    nan (nan)         10
                                                   1        50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.48 (0.73)     22.34 (60.86)    nan (nan)         10
                                                   2        50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.56 (0.69)     34.98 (95.20)    nan (nan)         10
                                                   3        50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.52 (0.71)    47.74 (129.61)    nan (nan)         10
                                                   4        50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.52 (0.71)    60.58 (163.80)    nan (nan)         10
                                                   5        50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.54 (0.70)    84.72 (196.93)    nan (nan)         10
                                                   6        50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.52 (0.71)   112.10 (235.92)    nan (nan)         10
                                                   7        50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.54 (0.70)   137.28 (272.34)    nan (nan)         10
                                                   8        50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.50 (0.72)   162.67 (311.72)    nan (nan)         10
                                                   9        50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.56 (0.69)   190.60 (359.32)    nan (nan)         10
                                                   None     50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.50 (0.72)       0.27 (0.00)    nan (nan)         10
ball_and_cup_sticky_table-random_score_explore     0        50.00 (0.00)  4.30 (1.55)  21.00 (3.09)   0.41 (0.02)      59.18 (8.82)     12.49 (34.15)  1.10 (0.56)         10
                                                   1        50.00 (0.00)  3.10 (2.26)  23.40 (3.23)   0.45 (0.01)     58.83 (14.77)    132.10 (81.78)  1.42 (1.04)         10
                                                   2        50.00 (0.00)  1.40 (2.24)  23.64 (2.40)   0.44 (0.02)     42.95 (12.99)   285.47 (122.47)  0.90 (0.28)         10
                                                   3        50.00 (0.00)  1.80 (2.44)  23.88 (3.10)   0.42 (0.03)     49.90 (15.45)   440.21 (178.97)  1.45 (0.63)         10
                                                   4        50.00 (0.00)  1.30 (1.73)  24.00 (2.01)   0.44 (0.02)     44.84 (12.69)   579.23 (236.45)  1.34 (0.41)         10
                                                   5        50.00 (0.00)  0.00 (0.00)  25.32 (3.20)     nan (nan)     41.98 (18.61)   727.07 (306.51)    nan (nan)         10
                                                   6        50.00 (0.00)  0.20 (0.60)  24.72 (1.44)   0.42 (0.00)     39.73 (14.04)   885.60 (379.00)  1.95 (0.00)         10
                                                   7        50.00 (0.00)  2.30 (3.03)  21.72 (3.54)   0.43 (0.01)      40.78 (8.33)  1052.87 (464.31)  0.75 (0.31)         10
                                                   8        50.00 (0.00)  2.20 (2.93)  22.20 (3.14)   0.42 (0.02)     42.83 (11.28)  1214.11 (543.07)  0.80 (0.32)         10
                                                   9        50.00 (0.00)  2.50 (2.91)  22.20 (3.45)   0.44 (0.01)     46.26 (14.48)  1340.85 (624.37)  0.92 (0.37)         10
                                                   None     50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.50 (0.72)       0.28 (0.01)    nan (nan)         10
ball_and_cup_sticky_table-success_rate_explore_... 0        50.00 (0.00)  4.40 (1.56)  20.40 (2.08)   0.43 (0.02)      57.55 (5.99)       1.03 (0.10)  1.04 (0.49)         10
                                                   1        50.00 (0.00)  4.20 (1.47)  21.96 (3.22)   0.44 (0.03)      61.97 (9.01)     37.02 (53.25)  1.05 (0.40)         10
                                                   2        50.00 (0.00)  3.50 (1.12)  23.40 (2.59)   0.45 (0.03)      66.05 (7.04)    101.00 (88.54)  1.26 (0.48)         10
                                                   3        50.00 (0.00)  2.90 (1.22)  24.12 (2.76)   0.44 (0.01)      68.09 (7.61)   196.17 (112.71)  1.67 (0.89)         10
                                                   4        50.00 (0.00)  2.40 (1.11)  24.96 (2.62)   0.44 (0.03)      70.47 (7.24)   303.97 (140.90)  1.95 (0.81)         10
                                                   5        50.00 (0.00)  3.40 (1.96)  23.64 (4.29)   0.44 (0.02)     66.73 (12.14)   406.62 (175.99)  1.87 (1.41)         10
                                                   6        50.00 (0.00)  3.20 (0.98)  23.76 (2.26)   0.44 (0.01)      67.07 (6.27)   501.06 (202.36)  1.34 (0.43)         10
                                                   7        50.00 (0.00)  3.10 (1.30)  24.72 (3.18)   0.45 (0.02)      69.79 (8.79)   601.60 (210.05)  1.60 (0.92)         10
                                                   8        50.00 (0.00)  3.10 (1.51)  24.00 (3.52)   0.43 (0.01)     67.75 (10.06)   713.20 (209.17)  1.65 (0.93)         10
                                                   9        50.00 (0.00)  3.90 (1.70)  22.68 (3.50)   0.43 (0.01)     64.01 (10.05)   821.80 (207.84)  1.22 (0.55)         10
                                                   None     50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.50 (0.72)       0.30 (0.04)    nan (nan)         10
ball_and_cup_sticky_table-success_rate_explore_ucb 0        50.00 (0.00)  4.40 (1.56)  20.40 (2.08)   0.40 (0.02)      57.55 (5.99)       1.09 (0.12)  1.04 (0.49)         10
                                                   1        50.00 (0.00)  4.20 (1.47)  21.96 (3.22)   0.44 (0.01)      61.97 (9.01)     44.27 (64.62)  1.05 (0.40)         10
                                                   2        50.00 (0.00)  3.50 (1.12)  23.40 (2.59)   0.44 (0.01)      66.05 (7.04)   111.26 (105.91)  1.26 (0.48)         10
                                                   3        50.00 (0.00)  2.90 (1.22)  24.12 (2.76)   0.44 (0.01)      68.09 (7.61)   198.80 (124.66)  1.67 (0.89)         10
                                                   4        50.00 (0.00)  2.40 (1.11)  24.96 (2.62)   0.45 (0.02)      70.47 (7.24)   300.30 (150.38)  1.95 (0.81)         10
                                                   5        50.00 (0.00)  3.40 (1.96)  23.64 (4.29)   0.44 (0.01)     66.73 (12.14)   399.27 (180.79)  1.87 (1.41)         10
                                                   6        50.00 (0.00)  3.20 (0.98)  23.76 (2.26)   0.44 (0.02)      67.07 (6.27)   490.24 (205.03)  1.34 (0.43)         10
                                                   7        50.00 (0.00)  3.10 (1.30)  24.72 (3.18)   0.44 (0.02)      69.79 (8.79)   589.88 (207.91)  1.60 (0.92)         10
                                                   8        50.00 (0.00)  3.10 (1.51)  24.00 (3.52)   0.45 (0.05)     67.75 (10.06)   701.04 (206.40)  1.65 (0.93)         10
                                                   9        50.00 (0.00)  3.90 (1.70)  22.68 (3.50)   0.43 (0.02)     64.01 (10.05)   809.38 (206.80)  1.22 (0.55)         10
                                                   None     50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.50 (0.72)       0.29 (0.01)    nan (nan)         10
ball_and_cup_sticky_table-task_repeat_explore      0        50.00 (0.00)  3.30 (1.42)  23.40 (2.29)   0.43 (0.03)      64.93 (6.07)     39.10 (38.08)  1.54 (0.94)         10
                                                   1        50.00 (0.00)  4.20 (1.60)  21.84 (2.98)   0.45 (0.04)      61.30 (8.45)    133.51 (51.27)  1.26 (0.98)         10
                                                   2        50.00 (0.00)  3.70 (2.49)  22.80 (4.71)   0.45 (0.01)     60.24 (15.66)    247.37 (60.78)  1.57 (1.33)         10
                                                   3        50.00 (0.00)  3.30 (2.15)  24.48 (4.90)   0.45 (0.01)     68.06 (12.88)    355.45 (58.50)  1.67 (1.28)         10
                                                   4        50.00 (0.00)  3.60 (1.74)  23.28 (3.14)   0.44 (0.02)      62.83 (9.63)    462.11 (72.91)  1.13 (0.49)         10
                                                   5        50.00 (0.00)  3.20 (2.27)  23.28 (4.02)   0.45 (0.01)     61.66 (11.68)    566.39 (95.18)  1.72 (1.26)         10
                                                   6        50.00 (0.00)  3.60 (2.37)  22.68 (3.41)   0.44 (0.01)     60.25 (12.73)   661.66 (123.18)  1.40 (1.02)         10
                                                   7        50.00 (0.00)  4.30 (3.13)  20.64 (4.55)   0.44 (0.01)     52.61 (12.30)   760.01 (126.62)  0.92 (0.49)         10
                                                   8        50.00 (0.00)  3.80 (2.36)  22.68 (4.37)   0.60 (0.44)     63.56 (12.47)   847.62 (148.74)  1.32 (1.02)         10
                                                   9        50.00 (0.00)  4.30 (3.03)  20.76 (4.43)   0.45 (0.04)     51.32 (14.09)   945.04 (148.65)  0.93 (0.52)         10
                                                   None     50.00 (0.00)  0.00 (0.00)  24.36 (0.55)     nan (nan)      35.50 (0.72)       0.29 (0.02)    nan (nan)         10