Taskboard

Taskboard for VisualWord2Vec!

Code

• Port the relationship modeling code to Github (5 June)
• Read and understand both the modeling and training (MATLAB) code (12 June)
• Clean up and document the training code to replicate results in the ICCV paper (15 June)
• Install and setup Caffe framework (already setup) (12 June)
• Remove irrelevant parts for the current work (19 June)
• Search and setup tools for refining the neural network (22 June)
• Cluster the visual vectors, refine trained CNN to classify into one of the clusters, get new word2vec features (26 June)

---

Ideas

How to incorporate visual features in word2vec?

1. Group the visual features obtained for 213 verbs into N clusters. Refine the trained CNN to predict the cluster id. Obtain the features from this modified CNN.
2. ?

---

Resources

• Training deep neural nets in python using deeplearning4j
• Training deep neural nets using gensim