Look Who it Was Random video

Remaining Challenges in Deep Learning based NLP | Vered Shwartz

482 views · Published 21 December 2018 · 30:44 · Indexed 13 October 2026

Channel: WeAreDevelopers · 2018 · Science & Technology

Watch on YouTube

Deep Learning has changed the face of Natural Language Processing (NLP) over the recent years. The state-of-the-art performance in nearly every NLP task today is achieved by a neural model, often with a large gap from previous models, which were based on domain knowledge. The availability of pre-trained word embeddings (such as word2vec), trained on a massive amount of texts, has given our models a good starting point for meaning representation. The ability to learn vector representations of arbitrarily long texts using Recurrent Neural Networks (RNNs) has facilitated re-using the same architectures to solve various NLP tasks. Relying on latent neural representations obviated the need to hand-engineer features, allowing machine learning practitioners to join the party without any knowledge of the domain. Popular media announces every few days that AI has solved language. But looking beyond the performance metrics, did Deep Learning actually solve NLP? In this talk, I will present some of the remaining challenges in Deep Learning based NLP. Among which, the need for a large amount of training data creates algorithms that only address certain domains and languages well while performing much worse on others. How performance metrics can lie when you don't actually know what your black-box network is learning. The lack of robustness of our models. The shaky ground on which we base our meaning representation, distributional word embeddings. Our unreasonable expectations from sentence and document level representations. And finally, the lack of interpretability, that didn't only kill the debugging, but also causes major fairness issues.

Visit the largest developer playground in Europe!
https://www.wearedevelopers.com/

Facebook: https://www.facebook.com/wearedevelopers
Twitter: https://twitter.com/WeAreDevs
Instagram: https://www.instagram.com/_wearedevelopers/

#WeAreDevs

More from this channel