If you’re working on a NLP related task, you’ll need to decide which approach to take. Before starting building the code a project structure, it’s important for you to define the specific goals of the project in terms of explainability and accuracy.

Nowadays the NLP techniques that can offer more accuracy are the ones with less explainability, you don’t really now what’s the decision process that the model follows. Simpler techniques are more understandable, but you probably will end up getting less accuracy.

The most common advice would be to start with simpler and more explainable techniques, validate the model and get the accuracy of it and then evaluate if this accuracy meets your previously defined goals. If not, keep progressing on more complex techniques, knowing what’s the trade-off.

A practical example would be to follow this process:

• tf-idf/bag of words
• Word2Vec (vector embeddings)
• FastText
• Deep Learning