带有Python的AI –自然语言处理

带有Python的AI –自然语言处理 (AI with Python – Natural Language Processing)

Natural Language Processing (NLP) refers to AI method of communicating with intelligent systems using a natural language such as English.

自然语言处理(NLP)是指使用自然语言(例如英语)与智能系统进行通信的AI方法。

Processing of Natural Language is required when you want an intelligent system like robot to perform as per your instructions, when you want to hear decision from a dialogue based clinical expert system, etc.

当您希望像机器人这样的智能系统按照您的说明执行时，或者您希望听到基于对话的临床专家系统的决策等时，需要对自然语言进行处理。

The field of NLP involves making computers erform useful tasks with the natural languages humans use. The input and output of an NLP system can be −

NLP领域涉及使计算机利用人类使用的自然语言来执行有用的任务。 NLP系统的输入和输出可以是-

• Speech
  言语
• Written Text
  书面文字

NLP的组成部分 (Components of NLP)

In this section, we will learn about the different components of NLP. There are two components of NLP. The components are described below −

在本节中，我们将学习NLP的不同组成部分。 NLP有两个组成部分。 组件描述如下-

自然语言理解(NLU) (Natural Language Understanding (NLU))

It involves the following tasks −

它涉及以下任务-

• Mapping the given input in natural language into useful representations.

  将给定的自然语言输入映射为有用的表示形式。

• Analyzing different aspects of the language.

  分析语言的不同方面。

自然语言生成(NLG) (Natural Language Generation (NLG))

It is the process of producing meaningful phrases and sentences in the form of natural language from some internal representation. It involves −

它是从某种内部表示形式以自然语言形式生成有意义的短语和句子的过程。 它涉及-

• Text planning − This includes retrieving the relevant content from the knowledge base.

  文本计划 -这包括从知识库中检索相关内容。

• Sentence planning − This includes choosing the required words, forming meaningful phrases, setting tone of the sentence.

  句子计划 -这包括选择所需的单词，形成有意义的短语，设置句子的语气。

• Text Realization − This is mapping sentence plan into sentence structure.

  文本实现 -这是将句子计划映射到句子结构中。

NLU的困难 (Difficulties in NLU)

The NLU is very rich in form and structure; however, it is ambiguous. There can be different levels of ambiguity −

NLU的形式和结构非常丰富； 但是，这是模棱两可的。 可能存在不同程度的歧义-

词汇歧义 (Lexical ambiguity)

It is at a very primitive level such as the word-level. For example, treating the word “board” as noun or verb?

它处于非常原始的级别，例如单词级别。 例如，将“木板”一词视为名词还是动词？

语法级别歧义 (Syntax level ambiguity)

A sentence can be parsed in different ways. For example, “He lifted the beetle with red cap.” − Did he use cap to lift the beetle or he lifted a beetle that had red cap?

句子可以用不同的方式解析。 例如，“他戴红色帽子举起了甲虫。” −他是用帽子举起甲虫还是举起有红色帽子的甲虫？

参照歧义 (Referential ambiguity)

Referring to something using pronouns. For example, Rima went to Gauri. She said, “I am tired.” − Exactly who is tired?

用代词指代某物 例如，里玛去了高里。 她说：“我很累。” −究竟谁累了？

NLP术语 (NLP Terminology)

Let us now see a few important terms in the NLP terminology.

现在让我们来看一下NLP术语中的一些重要术语。

• Phonology − It is study of organizing sound systematically.

  语音学 -这是系统地组织声音的研究。

• Morphology − It is a study of construction of words from primitive meaningful units.

  形态 -这是从原始有意义单位构造单词的研究。

• Morpheme − It is a primitive unit of meaning in a language.

  语素 -它是语言中意义的原始单位。

• Syntax − It refers to arranging words to make a sentence. It also involves determining the structural role of words in the sentence and in phrases.

  语法 -指安排单词组成句子。 它还涉及确定单词在句子和短语中的结构作用。

• Semantics − It is concerned with the meaning of words and how to combine words into meaningful phrases and sentences.

  语义 -它关注单词的含义以及如何将单词组合为有意义的短语和句子。

• Pragmatics − It deals with using and understanding sentences in different situations and how the interpretation of the sentence is affected.

  语用学 -它处理和理解不同情况下的句子，以及如何影响句子的解释。

• Discourse − It deals with how the immediately preceding sentence can affect the interpretation of the next sentence.

  话语 -它处理紧接在前的句子如何影响下一个句子的解释。

• World Knowledge − It includes the general knowledge about the world.

  世界知识 -它包括有关世界的常识。

NLP中的步骤 (Steps in NLP)

This section shows the different steps in NLP.

本节显示了NLP中的不同步骤。

词法分析 (Lexical Analysis)

It involves identifying and analyzing the structure of words. Lexicon of a language means the collection of words and phrases in a language. Lexical analysis is dividing the whole chunk of txt into paragraphs, sentences, and words.

它涉及识别和分析单词的结构。 语言词典是指语言中单词和短语的集合。 词法分析将txt的整个块划分为段落，句子和单词。

句法分析(解析) (Syntactic Analysis (Parsing))

It involves analysis of words in the sentence for grammar and arranging words in a manner that shows the relationship among the words. The sentence such as “The school goes to boy” is rejected by English syntactic analyzer.

它涉及分析句子中的单词以进行语法分析以及以显示单词之间关系的方式排列单词。 英语句法分析器拒绝诸如“学校去男孩”之类的句子。

语义分析 (Semantic Analysis)

It draws the exact meaning or the dictionary meaning from the text. The text is checked for meaningfulness. It is done by mapping syntactic structures and objects in the task domain. The semantic analyzer disregards sentence such as “hot ice-cream”.

它从文本中得出确切的含义或字典的含义。 检查文本是否有意义。 它是通过在任务域中映射语法结构和对象来完成的。 语义分析器忽略诸如“热冰淇淋”之类的句子。

话语整合 (Discourse Integration)

The meaning of any sentence depends upon the meaning of the sentence just before it. In addition, it also brings about the meaning of immediately succeeding sentence.

任何句子的含义取决于其前面句子的含义。 另外，它还带来了紧接句子的含义。

语用分析 (Pragmatic Analysis)

During this, what was said is re-interpreted on what it actually meant. It involves deriving those aspects of language which require real world knowledge.

在此期间，所讲的内容将根据其实际含义进行重新解释。 它涉及派生需要真实世界知识的语言方面。

翻译自: https://www.tutorialspoint.com/artificial_intelligence_with_python/artificial_intelligence_with_python_natural_language_processing.htm