South England NLP Meetup@UCL

Piotr Mirowski (Bing)

Wednesday, 30th April 2014 at 4pm - 6pm

Location: 16-18, Gordon Square, Room 101 (UCL)

This event is in the past.

Directions Room 101, 16-18 Gordon Square are here: crf.casa.ucl.ac.uk/screenRoute.aspx

Recent developments in neural probabilistic language models

In this tutorial talk, I will talk about recent development in statistical language models that go beyond word n-grams and that build on distributional semantics. Language modelling consists in assigning probabilities to sentences by factorizing the joint likelihood of the sentence into conditional likelihoods of a word given the word’s history. In continuous space language models, each word is also “embedded” into a low-dimensional vector-space representation that is learned as the language model is trained. When they are trained on very large corpora, these models achieve state-of-the-art perplexity and word error rates. Starting from neural probabilistic language models, I will present their extensions, including recurrent neural networks, log-bilinear models and continuous bags of words, mention the Microsoft Sentence Completion challenge dataset and illustrate how these models can preserve semantic linguistic regularities such as: {king} – {man} + {woman} = {queen}.



Comments


South England NLP Meetup@UCL

Powered by GroupSpaces · Terms · Privacy Policy · Cookie Use · Create Your Own Group