nlp

(From Diary March 11, 2017)

reading a paper while at a banquet tonight (Bates 1993)

The paper starts with marking a distinction between SR and NLP. SR is a well defined problem and effectiveness of method can be clearly assessed. NLP is hard. It is about understanding. It is not a well defined problem and hard to evaluate effectiveness of any method.

It surveys historical development of the branch. Now I understand why AI articles tend to be very “non-parametric”. It is because textual data itself is non-parametric–too many degrees of freedom. Any modelling of it will internally host a lot of degrees of freedom and thus could appear as adhoc and heuristic, but it is really reflecting the difficulty of finding a suitable parameterization of the text data itself.

Historically NLP researchers have attempted machine translation, question answering (which uses a database to store all answers and then the problem becomes interface design (querying)), application design such as grammar/style checkers, generative model.

There was the original attempt to always deep interpret every word, this conceptually overfits. Newer methodologies let go the temptation to interpret every word completely.

Broadly in NLP there are problem categories as syntactic processing, semantic processing, context modelling, and response generation.

The problems can be solve sequentially or independently using a joint probabilistic approach.