本文介绍了如何使用NLTK库进行文本预处理和Keras构建一个简单的问答聊天机器人。首先，从JSON文件加载数据，然后进行预处理，包括标记化和词形还原。接着，创建训练和测试数据，构建一个包含三层神经网络的模型，并进行训练。最后，通过图形用户界面接收用户输入，预测意图并返回相应回答。
Python聊天机器人项目-学习使用NLTK和Keras构建第一个聊天机器人
一个简单的问答机器人
下载数据集
准备好要用的数据集一般是xx.json文件，然后还需要准备tensorflow, keras, pickle, nltk这些工具包
这是json数据集的大概样子
以下是用Python从头创建聊天机器人的5个步骤:
- 导入并加载数据文件
- 数据进行预处理
- 创建培训和测试数据
- 构建模型
- 预测的回复
1 导入数据集
首先，创建一个名为train_chatbot.py的文件。
我们导入chatbot所需的包，并初始化Python项目中将要使用的变量。
数据文件是JSON格式的，所以我们使用JSON包来解析JSON文件到Python。
import nltk
from nltk.stem import WordNetLemmatizer
lemmatizer = WordNetLemmatizer()
import json
import pickle
import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Activation, Dropout
from keras.optimizers import SGD
import random
words=[]
classes = []
documents = []
ignore_words = ['?', '!']
data_file = open('intents.json').read()
intents = json.loads(data_file)
##2. 预处理数据
在处理文本数据时，在建立机器学习或深度学习模型之前，我们需要对数据进行各种预处理。
根据需求，我们需要应用各种操作对数据进行预处理。
Tokenizing(标记化)是对文本数据所能做的最基本也是最首要的事情。
标记化是将整个文本分解成单词等小部分的过程。
这里我们遍历模式并使用nltk.word_tokenize()函数对句子进行标记，并将每个单词添加到单词列表中。
我们还为标记创建了一个类列表。
for intent in intents['intents']:
for pattern in intent['patterns']:
#tokenize each word
w = nltk.word_tokenize(pattern)
words.extend(w)
#add documents in the corpus
documents.append((w, intent['tag']))
# add to our classes list
if intent['tag'] not in classes:
classes.append(intent['tag'])
print("classes ---:")
print(classes)
print("words: -------")
print(words)
print("documents:-------------")
print(documents)
classes ---:
['greeting', 'goodbye', 'thanks', 'options', 'adverse_drug', 'blood_pressure', 'blood_pressure_search', 'pharmacy_search', 'hospital_search']
words: -------
['Hi', 'there', 'How', 'are', 'you', 'Is', 'anyone', 'there', '?', 'Hey', 'Hola', 'Hello', 'Good', 'day', 'Bye', 'See', 'you', 'later', 'Goodbye', 'Nice', 'chatting', 'to', 'you', ',', 'bye', 'Till', 'next', 'time', 'Thanks', 'Thank', 'you', 'That', "'s", 'helpful', 'Awesome', ',', 'thanks', 'Thanks', 'for', 'helping', 'me', 'How', 'you', 'could', 'help', 'me', '?', 'What', 'you', 'can', 'do', '?', 'What', 'help', 'you', 'provide', '?', 'How', 'you', 'can', 'be', 'helpful', '?', 'What', 'support', 'is', 'offered', 'How', 'to', 'check', 'Adverse', 'drug', 'reaction', '?', 'Open', 'adverse', 'drugs', 'module', 'Give', 'me', 'a', 'list', 'of', 'drugs', 'causing', 'adverse', 'behavior', 'List', 'all', 'drugs', 'suitable', 'for', 'patient', 'with', 'adverse', 'reaction', 'Which', 'drugs', 'dont', 'have', 'adverse', 'reaction', '?', 'Open', 'blood', 'pressure', 'module', 'Task', 'related', 'to', 'blood', 'pressure', 'Blood', 'pressure', 'data', 'entry', 'I', 'want', 'to', 'log', 'blood', 'pressure', 'results', 'Blood', 'pressure', 'data', 'management', 'I', 'want', 'to', 'search', 'for', 'blood', 'pressure', 'result', 'history', 'Blood', 'pressure', 'for', 'patient', 'Load', 'patient', 'blood', 'pressure', 'result', 'Show', 'blood', 'pressure', 'results', 'for', 'patient', 'Find', 'blood', 'pressure', 'results', 'by', 'ID', 'Find', 'me', 'a', 'pharmacy', 'Find', 'pharmacy', 'List', 'of', 'pharmacies', 'nearby', 'Locate', 'pharmacy', 'Search', 'pharmacy', 'Lookup', 'for', 'hospital', 'Searching', 'for', 'hospital', 'to', 'transfer', 'patient', 'I', 'want', 'to', 'search', 'hospital', 'data', 'Hospital', 'lookup', 'for', 'patient', 'Looking', 'up', 'hospital', 'details', 'Hi', 'there', 'How', 'are', 'you', 'Is', 'anyone', 'there', '?', 'Hey', 'Hola', 'Hello', 'Good', 'day', 'Bye', 'See', 'you', 'later', 'Goodbye', 'Nice', 'chatting', 'to', 'you', ',', 'bye', 'Till', 'next', 'time', 'Thanks', 'Thank', 'you', 'That', "'s", 'helpful', 'Awesome', ',', 'thanks', 'Thanks', 'for', 'helping', 'me', 'How', 'you', 'could', 'help', 'me', '?', 'What', 'you', 'can', 'do', '?', 'What', 'help', 'you', 'provide', '?', 'How', 'you', 'can', 'be', 'helpful', '?', 'What', 'support', 'is', 'offered', 'How', 'to', 'check', 'Adverse', 'drug', 'reaction', '?', 'Open', 'adverse', 'drugs', 'module', 'Give', 'me', 'a', 'list', 'of', 'drugs', 'causing', 'adverse', 'behavior', 'List', 'all', 'drugs', 'suitable', 'for', 'patient', 'with', 'adverse', 'reaction', 'Which', 'drugs', 'dont', 'have', 'adverse', 'reaction', '?', 'Open', 'blood', 'pressure', 'module', 'Task', 'related', 'to', 'blood', 'pressure', 'Blood', 'pressure', 'data', 'entry', 'I', 'want', 'to', 'log', 'blood', 'pressure', 'results', 'Blood', 'pressure', 'data', 'management', 'I', 'want', 'to', 'search', 'for', 'blood', 'pressure', 'result', 'history', 'Blood', 'pressure', 'for', 'patient', 'Load', 'patient', 'blood', 'pressure', 'result', 'Show', 'blood', 'pressure', 'results', 'for', 'patient', 'Find', 'blood', 'pressure', 'results', 'by', 'ID', 'Find', 'me', 'a', 'pharmacy', 'Find', 'pharmacy', 'List', 'of', 'pharmacies', 'nearby', 'Locate', 'pharmacy', 'Search', 'pharmacy', 'Lookup', 'for', 'hospital', 'Searching', 'for', 'hospital', 'to', 'transfer', 'patient', 'I', 'want', 'to', 'search', 'hospital', 'data', 'Hospital', 'lookup', 'for', 'patient', 'Looking', 'up', 'hospital', 'details', 'Hi', 'there', 'How', 'are', 'you', 'Is', 'anyone', 'there', '?', 'Hey', 'Hola', 'Hello', 'Good', 'day', 'Bye', 'See', 'you', 'later', 'Goodbye', 'Nice', 'chatting', 'to', 'you', ',', 'bye', 'Till', 'next', 'time', 'Thanks', 'Thank', 'you', 'That', "'s", 'helpful', 'Awesome', ',', 'thanks', 'Thanks', 'for', 'helping', 'me', 'How', 'you', 'could', 'help', 'me', '?', 'What', 'you', 'can', 'do', '?', 'What', 'help', 'you', 'provide', '?', 'How', 'you', 'can', 'be', 'helpful', '?', 'What', 'support', 'is', 'offered', 'How', 'to', 'check', 'Adverse', 'drug', 'reaction', '?', 'Open', 'adverse', 'drugs', 'module', 'Give', 'me', 'a', 'list', 'of', 'drugs', 'causing', 'adverse', 'behavior', 'List', 'all', 'drugs', 'suitable', 'for', 'patient', 'with', 'adverse', 'reaction', 'Which', 'drugs', 'dont', 'have', 'adverse', 'reaction', '?', 'Open', 'blood', 'pressure', 'module', 'Task', 'related', 'to', 'blood', 'pressure', 'Blood', 'pressure', 'data', 'entry', 'I', 'want', 'to', 'log', 'blood', 'pressure', 'results', 'Blood', 'pressure', 'data', 'management', 'I', 'want', 'to', 'search', 'for', 'blood', 'pressure', 'result', 'history', 'Blood', 'pressure', 'for', 'patient', 'Load', 'patient', 'blood', 'pressure', 'result', 'Show', 'blood', 'pressure', 'results', 'for', 'patient', 'Find', 'blood', 'pressure', 'results', 'by', 'ID', 'Find', 'me', 'a', 'pharmacy', 'Find', 'pharmacy', 'List', 'of', 'pharmacies', 'nearby', 'Locate', 'pharmacy', 'Search', 'pharmacy', 'Lookup', 'for', 'hospital', 'Searching', 'for', 'hospital', 'to', 'transfer', 'patient', 'I', 'want', 'to', 'search', 'hospital', 'data', 'Hospital', 'lookup', 'for', 'patient', 'Looking', 'up', 'hospital', 'details', 'Hi', 'there', 'How', 'are', 'you', 'Is', 'anyone', 'there', '?', 'Hey', 'Hola', 'Hello', 'Good', 'day', 'Bye', 'See', 'you', 'later', 'Goodbye', 'Nice', 'chatting', 'to', 'you', ',', 'bye', 'Till', 'next', 'time', 'Thanks', 'Thank', 'you', 'That', "'s", 'helpful', 'Awesome', ',', 'thanks', 'Thanks', 'for', 'helping', 'me', 'How', 'you', 'could', 'help', 'me', '?', 'What', 'you', 'can', 'do', '?', 'What', 'help', 'you', 'provide', '?', 'How', 'you', 'can', 'be', 'helpful', '?', 'What', 'support', 'is', 'offered', 'How', 'to', 'check', 'Adverse', 'drug', 'reaction', '?', 'Open', 'adverse', 'drugs', 'module', 'Give', 'me', 'a', 'list', 'of', 'drugs', 'causing', 'adverse', 'behavior', 'List', 'all', 'drugs', 'suitable', 'for', 'patient', 'with', 'adverse', 'reaction', 'Which', 'drugs', 'dont', 'have', 'adverse', 'reaction', '?', 'Open', 'blood', 'pressure', 'module', 'Task', 'related', 'to', 'blood', 'pressure', 'Blood', 'pressure', 'data', 'entry', 'I', 'want', 'to', 'log', 'blood', 'pressure', 'results', 'Blood', 'pressure', 'data', 'management', 'I', 'want', 'to', 'search', 'for', 'blood', 'pressure', 'result', 'history', 'Blood', 'pressure', 'for', 'patient', 'Load', 'patient', 'blood', 'pressure', 'result', 'Show', 'blood', 'pressure', 'results', 'for', 'patient', 'Find', 'blood', 'pressure', 'results', 'by', 'ID', 'Find', 'me', 'a', 'pharmacy', 'Find', 'pharmacy', 'List', 'of', 'pharmacies', 'nearby', 'Locate', 'pharmacy', 'Search', 'pharmacy', 'Lookup', 'for', 'hospital', 'Searching', 'for', 'hospital', 'to', 'transfer', 'patient', 'I', 'want', 'to', 'search', 'hospital', 'data', 'Hospital', 'lookup', 'for', 'patient', 'Looking', 'up', 'hospital', 'details']
documents:-------------
[(['Hi', 'there'], 'greeting'), (['How', 'are', 'you'], 'greeting'), (['Is', 'anyone', 'there', '?'], 'greeting'), (['Hey'], 'greeting'), (['Hola'], 'greeting'), (['Hello'], 'greeting'), (['Good', 'day'], 'greeting'), (['Bye'], 'goodbye'), (['See', 'you', 'later'], 'goodbye'), (['Goodbye'], 'goodbye'), (['Nice', 'chatting', 'to', 'you', ',', 'bye'], 'goodbye'), (['Till', 'next', 'time'], 'goodbye'), (['Thanks'], 'thanks'), (['Thank', 'you'], 'thanks'), (['That', "'s", 'helpful'], 'thanks'), (['Awesome', ',', 'thanks'], 'thanks'), (['Thanks', 'for', 'helping', 'me'], 'thanks'), (['How', 'you', 'could', 'help', 'me', '?'], 'options'), (['What', 'you', 'can', 'do', '?'], 'options'), (['What', 'help', 'you', 'provide', '?'], 'options'), (['How', 'you', 'can', 'be', 'helpful', '?'], 'options'), (['What', 'support', 'is', 'offered'], 'options'), (['How', 'to', 'check', 'Adverse', 'drug', 'reaction', '?'], 'adverse_drug'), (['Open', 'adverse', 'drugs', 'module'], 'adverse_drug'), (['Give', 'me', 'a', 'list', 'of', 'drugs', 'causing', 'adverse', 'behavior'], 'adverse_drug'), (['List', 'all', 'drugs', 'suitable', 'for', 'patient', 'with', 'adverse', 'reaction'], 'adverse_drug'), (['Which', 'drugs', 'dont', 'have', 'adverse', 'reaction', '?'], 'adverse_drug'), (['Open', 'blood', 'pressure', 'module'], 'blood_pressure'), (['Task', 'related', 'to', 'blood', 'pressure'], 'blood_pressure'), (['Blood', 'pressure', 'data', 'entry'], 'blood_pressure'), (['I', 'want', 'to', 'log', 'blood', 'pressure', 'results'], 'blood_pressure'), (['Blood', 'pressure', 'data', 'management'], 'blood_pressure'), (['I', 'want', 'to', 'search', 'for', 'blood', 'pressure', 'result', 'history'], 'blood_pressure_search'), (['Blood', 'pressure', 'for', 'patient'], 'blood_pressure_search'), (['Load', 'patient', 'blood', 'pressure', 'result'], 'blood_pressure_search'), (['Show', 'blood', 'pressure', 'results', 'for', 'patient'], 'blood_pressure_search'), (['Find', 'blood', 'pressure', 'results', 'by', 'ID'], 'blood_pressure_search'), (['Find', 'me', 'a', 'pharmacy'], 'pharmacy_search'), (['Find', 'pharmacy'], 'pharmacy_search'), (['List', 'of', 'pharmacies', 'nearby'], 'pharmacy_search'), (['Locate', 'pharmacy'], 'pharmacy_search'), (['Search', 'pharmacy'], 'pharmacy_search'), (['Lookup', 'for', 'hospital'], 'hospital_search'), (['Searching', 'for', 'hospital', 'to', 'transfer', 'patient'], 'hospital_search'), (['I', 'want', 'to', 'search', 'hospital', 'data'], 'hospital_search'), (['Hospital', 'lookup', 'for', 'patient'], 'hospital_search'), (['Looking', 'up', 'hospital', 'details'], 'hospital_search'), (['Hi', 'there'], 'greeting'), (['How', 'are', 'you'], 'greeting'), (['Is', 'anyone', 'there', '?'], 'greeting'), (['Hey'], 'greeting'), (['Hola'], 'greeting'), (['Hello'], 'greeting'), (['Good', 'day'], 'greeting'), (['Bye'], 'goodbye'), (['See', 'you', 'later'], 'goodbye'), (['Goodbye'], 'goodbye'), (['Nice', 'chatting', 'to', 'you', ',', 'bye'], 'goodbye'), (['Till', 'next', 'time'], 'goodbye'), (['Thanks'], 'thanks'), (['Thank', 'you'], 'thanks'), (['That', "'s", 'helpful'], 'thanks'), (['Awesome', ',', 'thanks'], 'thanks'), (['Thanks', 'for', 'helping', 'me'], 'thanks'), (['How', 'you', 'could', 'help', 'me', '?'], 'options'), (['What', 'you', 'can', 'do', '?'], 'options'), (['What', 'help', 'you', 'provide', '?'], 'options'), (['How', 'you', 'can', 'be', 'helpful', '?'], 'options'), (['What', 'support', 'is', 'offered'], 'options'), (['How', 'to', 'check', 'Adverse', 'drug', 'reaction', '?'], 'adverse_drug'), (['Open', 'adverse', 'drugs', 'module'], 'adverse_drug'), (['Give', 'me', 'a', 'list', 'of', 'drugs', 'causing', 'adverse', 'behavior'], 'adverse_drug'), (['List', 'all', 'drugs', 'suitable', 'for', 'patient', 'with', 'adverse', 'reaction'], 'adverse_drug'), (['Which', 'drugs', 'dont', 'have', 'adverse', 'reaction', '?'], 'adverse_drug'), (['Open', 'blood', 'pressure', 'module'], 'blood_pressure'), (['Task', 'related', 'to', 'blood', 'pressure'], 'blood_pressure'), (['Blood', 'pressure', 'data', 'entry'], 'blood_pressure'), (['I', 'want', 'to', 'log', 'blood', 'pressure', 'results'], 'blood_pressure'), (['Blood', 'pressure', 'data', 'management'], 'blood_pressure'), (['I', 'want', 'to', 'search', 'for', 'blood', 'pressure', 'result', 'history'], 'blood_pressure_search'), (['Blood', 'pressure', 'for', 'patient'], 'blood_pressure_search'), (['Load', 'patient', 'blood', 'pressure', 'result'], 'blood_pressure_search'), (['Show', 'blood', 'pressure', 'results', 'for', 'patient'], 'blood_pressure_search'), (['Find', 'blood', 'pressure', 'results', 'by', 'ID'], 'blood_pressure_search'), (['Find', 'me', 'a', 'pharmacy'], 'pharmacy_search'), (['Find', 'pharmacy'], 'pharmacy_search'), (['List', 'of', 'pharmacies', 'nearby'], 'pharmacy_search'), (['Locate', 'pharmacy'], 'pharmacy_search'), (['Search', 'pharmacy'], 'pharmacy_search'), (['Lookup', 'for', 'hospital'], 'hospital_search'), (['Searching', 'for', 'hospital', 'to', 'transfer', 'patient'], 'hospital_search'), (['I', 'want', 'to', 'search', 'hospital', 'data'], 'hospital_search'), (['Hospital', 'lookup', 'for', 'patient'], 'hospital_search'), (['Looking', 'up', 'hospital', 'details'], 'hospital_search'), (['Hi', 'there'], 'greeting'), (['How', 'are', 'you'], 'greeting'), (['Is', 'anyone', 'there', '?'], 'greeting'), (['Hey'], 'greeting'), (['Hola'], 'greeting'), (['Hello'], 'greeting'), (['Good', 'day'], 'greeting'), (['Bye'], 'goodbye'), (['See', 'you', 'later'], 'goodbye'), (['Goodbye'], 'goodbye'), (['Nice', 'chatting', 'to', 'you', ',', 'bye'], 'goodbye'), (['Till', 'next', 'time'], 'goodbye'), (['Thanks'], 'thanks'), (['Thank', 'you'], 'thanks'), (['That', "'s", 'helpful'], 'thanks'), (['Awesome', ',', 'thanks'], 'thanks'), (['Thanks', 'for', 'helping', 'me'], 'thanks'), (['How', 'you', 'could', 'help', 'me', '?'], 'options'), (['What', 'you', 'can', 'do', '?'], 'options'), (['What', 'help', 'you', 'provide', '?'], 'options'), (['How', 'you', 'can', 'be', 'helpful', '?'], 'options'), (['What', 'support', 'is', 'offered'], 'options'), (['How', 'to', 'check', 'Adverse', 'drug', 'reaction', '?'], 'adverse_drug'), (['Open', 'adverse', 'drugs', 'module'], 'adverse_drug'), (['Give', 'me', 'a', 'list', 'of', 'drugs', 'causing', 'adverse', 'behavior'], 'adverse_drug'), (['List', 'all', 'drugs', 'suitable', 'for', 'patient', 'with', 'adverse', 'reaction'], 'adverse_drug'), (['Which', 'drugs', 'dont', 'have', 'adverse', 'reaction', '?'], 'adverse_drug'), (['Open', 'blood', 'pressure', 'module'], 'blood_pressure'), (['Task', 'related', 'to', 'blood', 'pressure'], 'blood_pressure'), (['Blood', 'pressure', 'data', 'entry'], 'blood_pressure'), (['I', 'want', 'to', 'log', 'blood', 'pressure', 'results'], 'blood_pressure'), (['Blood', 'pressure', 'data', 'management'], 'blood_pressure'), (['I', 'want', 'to', 'search', 'for', 'blood', 'pressure', 'result', 'history'], 'blood_pressure_search'), (['Blood', 'pressure', 'for', 'patient'], 'blood_pressure_search'), (['Load', 'patient', 'blood', 'pressure', 'result'], 'blood_pressure_search'), (['Show', 'blood', 'pressure', 'results', 'for', 'patient'], 'blood_pressure_search'), (['Find', 'blood', 'pressure', 'results', 'by', 'ID'], 'blood_pressure_search'), (['Find', 'me', 'a', 'pharmacy'], 'pharmacy_search'), (['Find', 'pharmacy'], 'pharmacy_search'), (['List', 'of', 'pharmacies', 'nearby'], 'pharmacy_search'), (['Locate', 'pharmacy'], 'pharmacy_search'), (['Search', 'pharmacy'], 'pharmacy_search'), (['Lookup', 'for', 'hospital'], 'hospital_search'), (['Searching', 'for', 'hospital', 'to', 'transfer', 'patient'], 'hospital_search'), (['I', 'want', 'to', 'search', 'hospital', 'data'], 'hospital_search'), (['Hospital', 'lookup', 'for', 'patient'], 'hospital_search'), (['Looking', 'up', 'hospital', 'details'], 'hospital_search'), (['Hi', 'there'], 'greeting'), (['How', 'are', 'you'], 'greeting'), (['Is', 'anyone', 'there', '?'], 'greeting'), (['Hey'], 'greeting'), (['Hola'], 'greeting'), (['Hello'], 'greeting'), (['Good', 'day'], 'greeting'), (['Bye'], 'goodbye'), (['See', 'you', 'later'], 'goodbye'), (['Goodbye'], 'goodbye'), (['Nice', 'chatting', 'to', 'you', ',', 'bye'], 'goodbye'), (['Till', 'next', 'time'], 'goodbye'), (['Thanks'], 'thanks'), (['Thank', 'you'], 'thanks'), (['That', "'s", 'helpful'], 'thanks'), (['Awesome', ',', 'thanks'], 'thanks'), (['Thanks', 'for', 'helping', 'me'], 'thanks'), (['How', 'you', 'could', 'help', 'me', '?'], 'options'), (['What', 'you', 'can', 'do', '?'], 'options'), (['What', 'help', 'you', 'provide', '?'], 'options'), (['How', 'you', 'can', 'be', 'helpful', '?'], 'options'), (['What', 'support', 'is', 'offered'], 'options'), (['How', 'to', 'check', 'Adverse', 'drug', 'reaction', '?'], 'adverse_drug'), (['Open', 'adverse', 'drugs', 'module'], 'adverse_drug'), (['Give', 'me', 'a', 'list', 'of', 'drugs', 'causing', 'adverse', 'behavior'], 'adverse_drug'), (['List', 'all', 'drugs', 'suitable', 'for', 'patient', 'with', 'adverse', 'reaction'], 'adverse_drug'), (['Which', 'drugs', 'dont', 'have', 'adverse', 'reaction', '?'], 'adverse_drug'), (['Open', 'blood', 'pressure', 'module'], 'blood_pressure'), (['Task', 'related', 'to', 'blood', 'pressure'], 'blood_pressure'), (['Blood', 'pressure', 'data', 'entry'], 'blood_pressure'), (['I', 'want', 'to', 'log', 'blood', 'pressure', 'results'], 'blood_pressure'), (['Blood', 'pressure', 'data', 'management'], 'blood_pressure'), (['I', 'want', 'to', 'search', 'for', 'blood', 'pressure', 'result', 'history'], 'blood_pressure_search'), (['Blood', 'pressure', 'for', 'patient'], 'blood_pressure_search'), (['Load', 'patient', 'blood', 'pressure', 'result'], 'blood_pressure_search'), (['Show', 'blood', 'pressure', 'results', 'for', 'patient'], 'blood_pressure_search'), (['Find', 'blood', 'pressure', 'results', 'by', 'ID'], 'blood_pressure_search'), (['Find', 'me', 'a', 'pharmacy'], 'pharmacy_search'), (['Find', 'pharmacy'], 'pharmacy_search'), (['List', 'of', 'pharmacies', 'nearby'], 'pharmacy_search'), (['Locate', 'pharmacy'], 'pharmacy_search'), (['Search', 'pharmacy'], 'pharmacy_search'), (['Lookup', 'for', 'hospital'], 'hospital_search'), (['Searching', 'for', 'hospital', 'to', 'transfer', 'patient'], 'hospital_search'), (['I', 'want', 'to', 'search', 'hospital', 'data'], 'hospital_search'), (['Hospital', 'lookup', 'for', 'patient'], 'hospital_search'), (['Looking', 'up', 'hospital', 'details'], 'hospital_search')]
现在我们将对每个单词进行词形化，并从列表中删除重复的单词。
Lemmatizing 是将一个单词转换为lemma形式，然后创建一个pickle文件来存储我们在预测时将要使用的Python对象的过程。
# lemmatize, lower each word and remove duplicates
words = [lemmatizer.lemmatize(w.lower()) for w in words if w not in ignore_words]
words = sorted(list(set(words)))
# sort classes
classes = sorted(list(set(classes)))
# documents = combination between patterns and intents
print (len(documents), "documents")
# classes = intents
print (len(classes), "classes", classes)
# words = all words, vocabulary
print (len(words), "unique lemmatized words", words)
pickle.dump(words,open('words.pkl','wb'))
pickle.dump(classes,open('classes.pkl','wb'))
188 documents
9 classes ['adverse_drug', 'blood_pressure', 'blood_pressure_search', 'goodbye', 'greeting', 'hospital_search', 'options', 'pharmacy_search', 'thanks']
88 unique lemmatized words ["'s", ',', 'a', 'adverse', 'all', 'anyone', 'are', 'awesome', 'be', 'behavior', 'blood', 'by', 'bye', 'can', 'causing', 'chatting', 'check', 'could', 'data', 'day', 'detail', 'do', 'dont', 'drug', 'entry', 'find', 'for', 'give', 'good', 'goodbye', 'have', 'hello', 'help', 'helpful', 'helping', 'hey', 'hi', 'history', 'hola', 'hospital', 'how', 'i', 'id', 'is', 'later', 'list', 'load', 'locate', 'log', 'looking', 'lookup', 'management', 'me', 'module', 'nearby', 'next', 'nice', 'of', 'offered', 'open', 'patient', 'pharmacy', 'pressure', 'provide', 'reaction', 'related', 'result', 'search', 'searching', 'see', 'show', 'suitable', 'support', 'task', 'thank', 'thanks', 'that', 'there', 'till', 'time', 'to', 'transfer', 'up', 'want', 'what', 'which', 'with', 'you']
utput_empty = [0] * 10
print(utput_empty)
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
3.创建培训和测试数据
现在，我们将创建训练数据，在其中提供输入和输出。
我们的输入是模式，输出是我们的输入模式所属的类。
但是计算机不理解文本，所以我们将文本转换成数字。
# create our training data
training = []
# create an empty array for our output
output_empty = [0] * len(classes)
# training set, bag of words for each sentence
for doc in documents:
# initialize our bag of words
bag = []
# list of tokenized words for the pattern
pattern_words = doc[0]
# lemmatize each word - create base word, in attempt to represent related words
pattern_words = [lemmatizer.lemmatize(word.lower()) for word in pattern_words]
# create our bag of words array with 1, if word match found in current pattern
for w in words:
bag.append(1) if w in pattern_words else bag.append(0)
# output is a '0' for each tag and '1' for current tag (for each pattern)
output_row = list(output_empty)
output_row[classes.index(doc[1])] = 1
training.append([bag, output_row])
# shuffle our features and turn into np.array
random.shuffle(training)
training = np.array(training)
# create train and test lists. X - patterns, Y - intents
train_x = list(training[:,0])
train_y = list(training[:,1])
print("Training data created")
4. 构建模型
我们已经准备好了训练数据，现在我们将建立一个有三层的深度神经网络。
为此，我们使用Keras顺序API。
经过200个epoch的训练，我们的模型达到了100%的精度。
让我们将模型保存为’ chatbot_model.h5 '。
# Create model - 3 layers. First layer 128 neurons, second layer 64 neurons and 3rd output layer contains number of neurons
# equal to number of intents to predict output intent with softmax
model = Sequential()
model.add(Dense(128, input_shape=(len(train_x[0]),), activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(len(train_y[0]), activation='softmax'))
# Compile model. Stochastic gradient descent with Nesterov accelerated gradient gives good results for this model
sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
#fitting and saving the model
hist = model.fit(np.array(train_x), np.array(train_y), epochs=200, batch_size=5, verbose=1)
model.save('chatbot_model.h5', hist)
print("model created")
5. 预测响应(图形用户界面)
为了预测句子，并从用户那里获得响应，让我们创建一个新文件“chatapp.py”。
我们将加载经过训练的模型，然后使用一个图形用户界面来预测机器人的响应。
模型只会告诉我们它所属的类，因此我们将实现一些函数，这些函数将识别类，然后从响应列表中检索一个随机响应。
我们再次导入必要的包并加载’ words。
pkl”和“类。
我们在训练模型时创建的Pkl ’ pickle文件:
import nltk
from nltk.stem import WordNetLemmatizer
lemmatizer = WordNetLemmatizer()
import pickle
import numpy as np
from keras.models import load_model
model = load_model('chatbot_model.h5')
import json
import random
intents = json.loads(open('intents.json').read())
words = pickle.load(open('words.pkl','rb'))
classes = pickle.load(open('classes.pkl','rb'))
为了预测class，我们需要提供与培训时相同的输入。
因此，我们将创建一些函数来执行文本预处理，然后预测类。
def clean_up_sentence(sentence):
# tokenize the pattern - split words into array
sentence_words = nltk.word_tokenize(sentence)
# stem each word - create short form for word
sentence_words = [lemmatizer.lemmatize(word.lower()) for word in sentence_words]
return sentence_words
# return bag of words array: 0 or 1 for each word in the bag that exists in the sentence
def bow(sentence, words, show_details=True):
# tokenize the pattern
sentence_words = clean_up_sentence(sentence)
# bag of words - matrix of N words, vocabulary matrix
bag = [0]*len(words)
for s in sentence_words:
for i,w in enumerate(words):
if w == s:
# assign 1 if current word is in the vocabulary position
bag[i] = 1
if show_details:
print ("found in bag: %s" % w)
return(np.array(bag))
def predict_class(sentence, model):
# filter out predictions below a threshold
p = bow(sentence, words,show_details=False)
res = model.predict(np.array([p]))[0]
ERROR_THRESHOLD = 0.25
results = [[i,r] for i,r in enumerate(res) if r>ERROR_THRESHOLD]
# sort by strength of probability
results.sort(key=lambda x: x[1], reverse=True)
return_list = []
for r in results:
return_list.append({"intent": classes[r[0]], "probability": str(r[1])})
return return_list
在预测类之后，我们将从意图列表中得到一个随机响应。
def getResponse(ints, intents_json):
tag = ints[0]['intent']
list_of_intents = intents_json['intents']
for i in list_of_intents:
if(i['tag']== tag):
result = random.choice(i['responses'])
break
return result
def chatbot_response(text):
ints = predict_class(text, model)
res = getResponse(ints, intents)
return res
现在我们将开发一个图形用户界面。
让我们使用Tkinter库，它附带了大量用于GUI的有用库。
我们将从用户获取输入消息，然后使用我们创建的助手函数从bot获取响应并将其显示在GUI上。
下面是GUI的完整源代码。
#Creating GUI with tkinter
import tkinter
from tkinter import *
def send():
msg = EntryBox.get("1.0",'end-1c').strip()
EntryBox.delete("0.0",END)
if msg != '':
ChatLog.config(state=NORMAL)
ChatLog.insert(END, "You: " + msg + '\n\n')
ChatLog.config(foreground="#442265", font=("Verdana", 12 ))
res = chatbot_response(msg)
ChatLog.insert(END, "Bot: " + res + '\n\n')
ChatLog.config(state=DISABLED)
ChatLog.yview(END)
base = Tk()
base.title("Hello")
base.geometry("400x500")
base.resizable(width=FALSE, height=FALSE)
#Create Chat window
ChatLog = Text(base, bd=0, bg="white", height="8", width="50", font="Arial",)
ChatLog.config(state=DISABLED)
#Bind scrollbar to Chat window
scrollbar = Scrollbar(base, command=ChatLog.yview, cursor="heart")
ChatLog['yscrollcommand'] = scrollbar.set
#Create Button to send message
SendButton = Button(base, font=("Verdana",12,'bold'), text="Send", width="12", height=5,
bd=0, bg="#32de97", activebackground="#3c9d9b",fg='#ffffff',
command= send )
#Create the box to enter message
EntryBox = Text(base, bd=0, bg="white",width="29", height="5", font="Arial")
#EntryBox.bind("<Return>", send)
#Place all components on the screen
scrollbar.place(x=376,y=6, height=386)
ChatLog.place(x=6,y=6, height=386, width=370)
EntryBox.place(x=128, y=401, height=90, width=265)
SendButton.place(x=6, y=401, height=90)
base.mainloop()
6. 运行聊天机器人
为了运行聊天机器人，我们有两个主要文件;
train_chatbot.py chatapp.py。
首先，我们使用终端中的命令对模型进行训练:
python train_chatbot.py
如果我们在训练期间没有看到任何错误，我们就成功地创建了模型。
然后运行应用程序，我们运行第二个文件。
python chatgui.py
扫一扫
2372
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
