Implement ctc loss function

[antimora]

implement ctc loss function. Can complete: #1049

[pmalex]

Hi! I have a few questions regarding implementation details. Here is the article.

1. Does we need to manually implement backward pass? As I can see in torch it is manually implemented.
2. In the original article authors use rescaling mode (additional math operation) due to better numerical stability of results (because probabilities in normal mode become zeros so fast). Should it be used here? Or burn handles numerical stability automatically under the hood?
3. Is there some standard way to measure performance inside burn? Or should I use criterion and manually implement benchmarks?

[pmalex]

@antimora, @laggui ?

[laggui]

Sorry I was not subscribed to this older issue, didn't get notifications for the first question 😅

1. This is not usually required for loss functions since they are a combination of tensor ops to compute the result. For now we have tensor ops, activation ops and module ops which can have their own implementation, but no provision for loss functions.
2. Tricks for better numerical stability are usually at the implementation level (see log sigmoid for example).
3. We have the burn benchmarks to compare speed between backends, but what you're looking for probably just needs manually measuring the performance (e.g. speed and memory).