Remove semicolon from e_query_arg safe character list

[plambert]

For Issue 110...

[plambert]

The bug this fixes is that the user agent is "lost" when sending an agent with semicolons (which is most of them) and then following any links on the results page, where the semicolons are not url-escaped and end up being dropped from the URL, at least in my Chrome 20.something and Firefox 14.something.

[mnot]

Which links in particular are you seeing this happen on? Can you give me a redbot URL that demonstrates this?

Thanks

[plambert]

Go to redbot.org and set the user-agent to the second one in the popup.

Then enter http://news.yahoo.com/ and hit return.

Now, click the "check embedded" link: http://redbot.org/?descend=True&uri=http://news.yahoo.com/&req_hdr=User-Agent%3AMozilla/4.0%20(compatible;%20MSIE%208.0;%20Windows%20NT%206.1;%20Win64;%20x64;%20Trident/4.0)

You've now got a truncated user-agent in all the links on the resulting page.

For example: http://redbot.org/?uri=http://news.yahoo.com/&req_hdr=User-Agent%3AMozilla/4.0%20(compatible

Happens in both Firefox and Chrome on my Mac.

[mnot]

Thanks. This is actually a server-side issue; Python's parse_qs treats semicolon as a delimiter, as recommended by HTML4. Anyway, thanks again for digging into it.