elasticsearch分析教程

目录(Table of Contents)

• Introduction

  介绍

• Connecting to Elasticsearch

  连接到Elasticsearch

• Querying With Search-Based Analytics

  使用基于搜索的分析进行查询

• Analyzing and Visualizing Your Data

  分析和可视化您的数据

• Summary

  概要

介绍(Introduction)

Elasticsearch is a scalable full-text search engine with an HTTP web interface and schema-based JSON documents. Elasticsearch shines brightest when it is used in the background as the fundamental engine powering applications with convoluted search features and many requirements.

Elasticsearch是可扩展的全文本搜索引擎，具有HTTP Web界面和基于架构的JSON文档。 当将Elasticsearch用作具有复杂搜索功能和许多要求的基本引擎来为应用程序提供动力时，Elasticsearch在后台使用时最为耀眼。

At it stands, Kibana’s position in Elastic’s popular ELK stack makes it the most common tool used for the purpose of visualizing and analyzing data from Elasticsearch. While Kibana certainly stands on its own merit, it also features some shortcomings; namely, it does not support integration with any other data source, nor does it offer user management features or support for raising proactive alerts.

就目前而言，Kibana在Elastic流行的ELK堆栈中的位置使其成为用于可视化和分析Elasticsearch数据的最常用工具。 虽然Kibana当然有自己的优点，但它也有一些缺点。 也就是说，它不支持与任何其他数据源集成，也不提供用户管理功能或不支持发出主动警报。

At some point, Elasticsearch users may decide that one or more of these shortcomings are a deal breaker, and opt for a different visualization platform that addresses these shortcomings. That is where Knowi comes in. Knowi offers broad native integration to 35 other data sources in addition to Elasticsearch, plenty of user management features, and support for alerts, as well as search-based analytics and machine learning. If you’re interested in learning about how to use Knowi to visualize data from Elasticsearch, you’ve come to the right place.

在某个时候，Elasticsearch用户可能会确定这些缺点中的一个或多个是破坏交易的行为，并选择解决这些缺点的另一种可视化平台。 那就是Knowi的用处。Knowi除了Elasticsearch，大量的用户管理功能，对警报的支持以及基于搜索的分析和机器学习之外，还向35个其他数据源提供了广泛的本机集成。 如果您对学习如何使用Knowi可视化Elasticsearch数据感兴趣，那么您来对地方了。

连接到Elasticsearch (Connecting to Elasticsearch)

Once you’re logged in to your free Knowi trial account, you’ll need to set up your Elasticsearch datasource by following these steps:

登录到免费的Knowi试用帐户后，您需要按照以下步骤设置Elasticsearch数据源：

1. Head to the upper middle of the panel on the left side of your screen and click on “Data sources.”
   转到屏幕左侧面板的中上方，然后单击“数据源”。
2. Head to the top right section of the NoSQL Datasources that are listed and click on Elasticsearch.
   转到列出的NoSQL数据源的右上部分，然后单击Elasticsearch。
3. Click on “Test Connection” at the bottom of your screen.
   单击屏幕底部的“测试连接”。
4. Once you’ve ensured that your connection is successful, click on “Save.”
   确保连接成功后，请单击“保存”。

演示地址

Image from Author

图片来自作者

使用基于搜索的分析进行查询(Querying With Search-Based Analytics)

Now that we’ve set up a datasource, it’s time to run a query on our data. Our goal is to analyze transit data from Georgia, and to specifically focus only on data which pertains to westbound vehicles on New Year’s Day a certain data set. Follow these steps in order to do so:

现在我们已经建立了数据源，是时候对该数据运行查询了。 我们的目标是分析佐治亚州的过境数据，并专门关注与某些数据集在元旦当天西行车辆有关的数据。 请按照以下步骤操作：

1. When you saved your datasource, you should’ve received an alert at the top of your screen saying “Datasource Added. Configure Queries.” Get started by clicking on “Queries.” Once you’ve done this, you will be taken to a query builder and you will see a new alert at the top of your screen which reads “Getting Indexes.” This means that Knowi is automatically indexing tables from the Elasticsearch datasource that you just connected to. Before you do anything else, name your query “Westbound Transit on New Year’s Day” under “Report Name*.”
   保存数据源后，您应该在屏幕顶部收到一条警告，提示“已添加数据源。 配置查询。” 点击“查询”开始使用。 完成此操作后，您将被带到查询构建器，您将在屏幕顶部看到一个新警报，显示为“获取索引”。 这意味着Knowi会自动为刚连接到的Elasticsearch数据源中的表建立索引。 在执行其他任何操作之前，请在“报告名称*”下将查询命名为“元旦西行过境”。
2. Hover down to “Indexes.” Click inside the bar, and you will see every index that exists in your Elasticsearch Database. Click on “transit.” This will prompt Knowi’s broad native integration to generate an Elasticsearch JSON Query which calls the first 10,000 rows from all columns of the transit table.
   将鼠标悬停在“索引”上。 单击栏内，您将看到Elasticsearch数据库中存在的每个索引。 点击“公交”。 这将提示Knowi进行广泛的本机集成，以生成Elasticsearch JSON查询，该查询调用转换表的所有列中的前10,000行。
3. Select “Preview” at the bottom left corner of your screen. This will show you preview data, but if you look at the visualization, you’ll notice Knowi automatically chose a map view. Because our preview data contains longitude and latitude coordinates, Know is smart enough to automatically turn the preview visualization into a Geo-Clusters/Custom Map visualization. If you click on one of the dots in the visualization, it will expand the map. This is exactly the type of visualization that we want to see, but remember, we’re only interested in westbound activity from New Year’s Day.
   选择屏幕左下角的“预览”。 这将向您显示预览数据，但是如果您查看可视化效果，则会注意到Knowi自动选择了地图视图。 由于我们的预览数据包含经度和纬度坐标，因此Know足够聪明，可以自动将预览可视化效果转换为Geo-Clusters / Custom Map可视化效果。 如果单击可视化中的点之一，它将扩展地图。 这正是我们想要看到的可视化类型，但请记住，我们只对元旦起的西行活动感兴趣。
4. In order to filter our data, we’re going to use Knowi’s search-based analytics feature which is simple, intuitive, and appeals to non-technical users by allowing them to ask questions in plain English and receive results in real time. Simply type “show me westbound on 2017–01–01” and give it a second.
   为了过滤数据，我们将使用Knowi的基于搜索的分析功能，该功能简单，直观，并且允许非技术用户以普通英语提问并实时接收结果，从而吸引他们。 只需键入“显示我在2017-01-01日向西行驶”，然后再输入一秒钟即可。
5. If you look back to your visualization, you should see that there are fewer observations on your map as you filtered it down to a specific subset of it. This is your cue that you’ve done everything right, and that now you just need to click “Save & Run Now” to complete your query.
   如果回头看一下可视化，您应该看到在将地图过滤为特定子集时，地图上的观测值减少了。 这表明您已正确完成了所有操作，现在只需单击“保存并立即运行”即可完成查询。

演示地址

Image from Author

图片来自作者

Congratulations on setting up your first Elasticsearch query and visualization with Knowi!

祝贺您使用Knowi设置了您的第一个Elasticsearch查询和可视化！

分析和可视化您的数据 (Analyzing and Visualizing Your Data)

As soon as you saved and ran your query, Knowi saved the raw data behind it as a dataset within Knowi’s elastic data warehouse, and it also saved your preview visualization as a widget within your Knowi account. In order to further analyze and visualize your data, follow these steps:

保存并运行查询后，Knowi将原始数据作为数据集保存在Knowi的弹性数据仓库中，并且还将预览可视化形式保存为Knowi帐户中的小部件。 为了进一步分析和可视化您的数据，请按照下列步骤操作：

1. Widgets are built to exist on dashboards. Right now, you’ve got a widget without a home. Give your widget a home by maneuvering over to the left side panel and clicking “Dashboards.” Then, click on the “+” icon to create a new dashboard and name it “Westbound Transit Dashboard.” Click “OK” to save this dashboard; you will be immediately taken to it once you do.
   窗口小部件构建为存在于仪表板上。 现在，您有了一个没有家的小部件。 移至左侧面板，然后单击“仪表板”，为小部件提供一个家。 然后，单击“ +”图标创建一个新的仪表板，并将其命名为“ Westbound Transit Dashboard”。 点击“确定”保存该仪表盘； 完成后，您将立即被接受。
2. Head back the panel on the left side of your screen, and just below “Dashboards,” click on “Widgets.” Here, you will see the “Westbound Transit on New Year’s Day” Widget which you just made. Add it to your dashboard by simply dragging it over and letting go.
   返回屏幕左侧的面板，然后在“仪表盘”下方，单击“窗口小部件”。 在这里，您将看到刚刚制作的“元旦西行过境”小部件。 只需将其拖动并放开，即可将其添加到仪表板。
3. Now that your widget is at home on your dashboard, you can conduct more analysis and visualization on your data. Click the 3 dot icon at the top right corner of your widget and select “Analyze” in order to be directed to the raw dataset which is currently empowering your widget.
   现在您的小部件已放在仪表板上，您可以对数据进行更多的分析和可视化。 单击窗口小部件右上角的3点图标，然后选择“分析”，以定向到当前正在为您的窗口小部件赋权的原始数据集。
4. Our goal here is to add visualize the distribution of different routes within our data. In other words, we want to see which routes were most frequently being traversed by westbound vehicles on New Year’s Day. So, from the left pane where the column names are listed, we’re going to drag “route_name” (make sure you choose this and not “route”) over to “Grouping/Dimensions.” Then, even though it already exists in “Fields/Metrics,” we’re going to drag it over there once more, but this time, we’re going to change the “Operation” in this second “route_name” field to “Count.” If you do this correctly, you will see six route names and the frequency with which they appear.
   我们的目标是在数据中可视化不同路线的分布。 换句话说，我们想知道在元旦那天最常被西行车穿越的路线。 因此，从列出列名称的左窗格中，我们将“ route_name”(确保您选择此名称而不是“ route”)拖到“ Grouping / Dimensions”上。 然后，即使它已经存在于“字段/指标”中，我们也将其再次拖到那儿，但是这次，我们要将第二个“ route_name”字段中的“ Operation”更改为“ Count” 。” 如果正确执行此操作，将看到六个路由名称及其出现的频率。
5. Drag the “Count of route_name” field from “Fields/Metrics” over to “Sort By” and then make sure to sort this field by descending order.
   将“路由名称计数”字段从“字段/指标”拖到“排序依据”，然后确保按降序对该字段进行排序。
6. Now, head to the top of your screen, and next to “Data,” click on “Visualization.” Then, change “Visualization Type” to “Pie.”
   现在，转到屏幕顶部，然后在“数据”旁边，单击“可视化”。 然后，将“可视化类型”更改为“饼图”。
7. Because we want to view our new pie chart side by side with our geo cluster map, we’re not going to save it; we’re going to clone it. To do this, head to the top right corner of your screen, find the “Clone” icon which looks like one piece of paper being dropped on top of another, and click on it. Then, name your new widget “Westbound Transit on New Year’s Day — Route Frequency.” Click “Clone” to complete this process and then click “Add to Dashboard” to add your new widget to your dashboard.
   因为我们要与地理簇地图同时查看新的饼图，所以我们将不保存它。 我们将克隆它。 为此，请转到屏幕的右上角，找到“克隆”图标，该图标看起来像是一张纸放在另一张纸上，然后单击它。 然后，将新的小部件命名为“元旦那天西行过境-路线频率”。 单击“克隆”完成此过程，然后单击“添加到仪表板”将新的小部件添加到仪表板。

演示地址

As you can see, Glenwood Road was our most traversed, with Moreland/Candler Park not far behind. There was a sizable drop between Moreland/Cander Park and Perry Blvd/West Highlands, and then a massive drop between Perry Blvd/West Highlands and the next three routes.

如您所见，格伦伍德路(Glenwood Road)是我们穿越得最多的地方，莫兰德/坎德勒公园(Moreland / Candler Park)紧随其后。 Moreland / Cander Park和Perry Blvd / West Highlands之间有很大的下降，然后Perry Blvd / West Highlands与接下来的三条路线之间有很大的下降。

概要 (Summary)

To summarize, we began this tutorial by establishing a successful connection to an Elasticsearch database and querying a specific subset of our transit dataset within our database using a helping hand from Knowi’s search-based analytics feature to make querying easier. When we successfully ran this query, Knowi stored the results of our query as a dataset in Knowi’s elastic data warehouse, and stored the preview visualization which we used as a widget in our Knowi account. We then created a new dashboard to serve as a home for our new widget, and created another widget to further analyze our dataset and answer an important question.

总而言之，我们通过建立与Elasticsearch数据库的成功连接并使用Knowi基于搜索的分析功能的帮助在查询数据库中查询我们的运输数据集的特定子集来开始本教程，以使查询更加轻松。 成功运行此查询后，Knowi将查询结果作为数据集存储在Knowi的弹性数据仓库中，并将预览可视化结果存储为Knowi帐户中的小部件。 然后，我们创建了一个新的仪表板作为新窗口小部件的主页，并创建了另一个窗口小部件以进一步分析我们的数据集并回答一个重要问题。

翻译自: https://towardsdatascience.com/elasticsearch-analytics-tutorial-7d47029804a0