Is presence of information the right measure of representation qualities?

[Natithan]

Hi,

I just read the abstract, so forgive me if you answer this in the main text already, but I thought I would try the lazy approach and ask the authors :P

The idea is that a very complex and capable probe will be better at seeing if any information is present int the representation than a less powerful probe, which makes sense.
However, consider the language representation consisting of the bits and bytes used to encode the alphanumeric characters of a text. That representation contains all the information necessary for various linguistic tasks: with a powerful enough probe, we can predict the syntax etc. But of course it is not a USEFUL representation, because all the "work" still needs to be done by the probe.

So it seems like presence of information is less relevant than the ease with which we can access that information?