Duplicate check fails against the example question?

[DHNishi]

I tried creating a MD file with the following text:

# Test Deck
## What's the Markdown?

Markdown is a lightweight markup language with plain-text-formatting syntax.
Its design allows it to be converted to many output formats,
but the original tool by the same name only supports HTML.

## Who created Markdown?

John Gruber created the Markdown language in 2004 in collaboration with
Aaron Swartz on the syntax.

I copied the questions directly our of the README.md. The "What's the Markdown?" question gets duplicated every time I run Anki: Send to Deck.

A quick diff check of the default notes's deck's JSON files for the questions shows that the duplicated questions only differ in the "id" field.

---

A minimal repro of this bug can be found by using the following test deck:

## Test'
Answer

## Test
Answer

The first question "Test'" will be duplicated but the second question "Test" will not. It looks like the apostrophe throws off the duplicate check somewhere and causes it to be re-added every time the question set is added to Anki.

Inspecting the generated card JSON file, I suspect that this is because the apostrophe converts to ' which triggers FooSoft/anki-connect#194 but I haven't done any deeper dive into this yet.

[jasonwilliams]

I think this is a duplicate (no pun intended) of #13

Certain characters cause he duplicate check to fail on AnkiConnect. I think it’s because we htmlEncode the final output and that causes problems.

[jasonwilliams]

Closing as duplicate