Error compiling bindings

[bratao]

While compiling bindings, a lot of error appears:

crfsuite_api.hpp:326:10: error: candidate is: bool CRFSuite::Tagger::open(const string&)
::string& name);

fsuite.cpp:1:0:
crfsuite.hpp: In member function 'CRFSuite::StringList CRFSuite::Tagger::viterbi()':
crfsuite.hpp:401:52: error: too few arguments to function
viterbi(tagger, path, &score))) {
                            ^
crfsuite.hpp: In member function 'double CRFSuite::Tagger::probability(const StringList&)':
crfsuite.hpp:474:48: error: too few arguments to function
lognorm(tagger, &lognorm))) {
                        ^
crfsuite.hpp: In member function 'double CRFSuite::Tagger::marginal(const string&, int)':
crfsuite.hpp:530:58: error: too few arguments to function
marginal_point(tagger, l, t, &prob))) {
                                  ^

Looks like crfsuite.hpp has not been updated with new code for semi-markov ( aux)

[bratao]

@WladimirSidorenko, Is this issue done ? Did you were able to the the Swig bindings to compile ?

Here I just tried and I got the error:

export_wrap.cpp:5108:25: error: redefinition of 'struct swig::traits<std::vector
<CRFSuite::Attribute> >'
     template <>  struct traits<CRFSuite::Item > {
                         ^
export_wrap.cpp:4995:22: error: previous definition of 'struct swig::traits<std:
:vector<CRFSuite::Attribute> >'
  template <>  struct traits<std::vector<CRFSuite::Attribute, std::allocator< CR
FSuite::Attribute > > > {
                      ^

[WladimirSidorenko]

not yet, I'm occasionally working through the errors, when I have spare time, but it's not yet ready. Is it urgent for your tasks?

[bratao]

Thank you for being so kind.

I just keep looking at this project, as I have a hypothesis that higher
order could improve my results.

But don't worry. If I get any free time I will try to fix it and contribute
as a patch.

However, I'm sure that you can fix this much faster than me. If you do it,
I will be forever grateful !

Thank you again for distributing this amazing improvement !
On Dec 29, 2015 18:45, "Wladimir Sidorenko" [email protected]
wrote:

not yet, I'm occasionally working through the errors, when I have spare
time, but it's not yet ready. Is it urgent for your tasks?

—
Reply to this email directly or view it on GitHub
#3 (comment)
.

[WladimirSidorenko]

I'm glad that it could be useful to someone. I have seemingly found the source of the errors (these are the %template lines in the file swig/export.i in the original @chokkan distribution. Actually, the original distribution had this kind of errors too, cf. https://github.com/chokkan/crfsuite/blob/master/swig/python/README at the very end) -- Once I remove them, the bindings get compiled, but then other errors occur during script execution. I'll try to finish them by the end of the next week and then update this bug.

Beware, however, that higher order models lead to an exponential increase of the training times. This is a general np-complete problem which is additionally aggravated by the fact that I haven't made any performance optimizations for the added higher-order models and don't provide support for training algorithms other than l-BFGS yet (though these algorithms typically converge faster and often yield better results, at least the preceptron). If you run into these problems, once the swig bindings are done, then you might either use an early stopping for l-BFGS (not wait for convergence) or possibly have a look at MarMot (https://github.com/muelletm/cistern/tree/master/marmot) or HOSemiCRF (https://github.com/nvcuong/HOSemiCRF) maybe they will work faster.

[bratao]

Thank you for the new references. But personally, Java is a big no, as I need to call from Python and Lua.

I got to compile the swig without errors using this export.i
https://raw.githubusercontent.com/abicky/crfsuite/develop/swig/export.i

But yeah, I got crashes when running the script =(

[bratao]

Hello @WladimirSidorenko , do you have any plans to look at this bug ?
It crashes when called from Python. I tried, but have no idea what is happening =(

I suppose that is some higher-order change that need to be made to the functions called from python.

[WladimirSidorenko]

I've just pushed further updates. The sample_train.py script works but throws sporadic seg faults which are difficult to trace. I assume there is some subtlety with SWIG's stream handling, which I don't know how to fix yet. You can look at the train script whether it works for you, I haven't tried the test script yet, but don't think that it will work out of the box.

[bratao]

I 💚 you !! Thank you so much.

I will try tonight. Thank you again !!!!

[WladimirSidorenko]

As I said, I can't promise anything yet. I did my best at the time that I had at my disposal, but unfortunately time is a very scarce resource for me. Try the updates out and post your output here. I assume there still are some things to be done.

[bratao]

@WladimirSidorenko , sorry to bother you again. I tested and the second-order linear-chain greatly improves my results !

However, I can´t find a way to get the python binds to work. Using both sample_train.py and sample_tag.py gave me segmentation faults at the very beginning.

Could you share what version of GCC, Operating System , Swag and Python you got it work ??

[bratao]

Debugging I can see that things are not implemented yet.
For example, sample_tag.py calls Tagger::viterbi(). At line 527 in crfsuite.hpp it calls "labels->release(labels);", but labels are set to NULL.

EDIT:
I guess it should be :
m_labels->release(m_labels);

Now the call to viterbi works. But I get a crash inside Tagger:probability() , I´m checking out.
-> Semim do not support probability for now. Removing this call, everything works =) !!!

[bratao]

With the latest set of commit, things are working now !
Thank you !!

[WladimirSidorenko]

Sorry for the delay. I was in a pretty hot publication phase and couldn't get any time for this project. The issue is still not solved yet, since the tree model still crashes during training, and there are neither tests nor auto-make rules to run the whole procedure automatically. Therefore, I'm re-opening the issue but you can ignore it, if the core functionality works for you. Hope, you'll still be able to get your results on time.

[bratao]

Awesome !!
I did got my results at time. However, not with your version of higher order =( It was very slow and not feasible to use. I ended using a LSTM architecture for tagging.

Do you have any idea of how to optimize the training speed ? I still haven't dig in the code. But with the end of my masters, this is something I'm very interested of !