appbase_使用Appbase.io和JavaScript构建类似于Live Search Feed的Twitter

appbase

Ever tried Twitter Live Search and wondered how it works? On the surface, you search for a #hashtag or a keyword and Twitter shows you a continuous stream of results, including new tweets that appear after you searched!

是否曾经尝试过Twitter Live Search，并想知道它如何工作？ 从表面上看，您搜索#hashtag或关键字，并且Twitter向您显示连续的结果流，包括搜索后出现的新tweet！

Image: We search for 'Trump' in the Twitter Live Mode. Notice the popup showing 20 new updates while we are on the screen.

图片：我们在Twitter Live模式中搜索“ Trump”。 注意，当我们在屏幕上时，弹出窗口显示20个新更新。

How is Twitter able to return a dynamic result feed for any searched keyword? One way to implement a Twitter Search style live feed is to return the original search results from a database (SQL, ElasticSearch, Mongo etc.) and then have a separate feed component using a realtime channel (think socket.io).

Twitter如何为搜索到的任何关键字返回动态结果供稿？ 实施Twitter搜索样式实时供稿的一种方法是从数据库（SQL，ElasticSearch，Mongo等）返回原始搜索结果，然后使用实时通道（如socket.io）拥有单独的供稿组件。

Image: Architecture for a naive realtime feed

图片 ：天真实时提要的体系结构

We can't know for sure how Twitter internally implements the realtime feed, but the approach described above, while easy to get started, requires scaling two different components and suffers from data consistency issues while offering an okay-ish O(MxN) time complexity (where M=data insertions, N=open keyword searches).

我们不能确切知道Twitter的内部如何实现实时的饲料，但上述的方法，而容易上手，需要从缩放数据一致性问题两种不同成分和遭受损失，而提供一个okay- ISH O（M×N个）的时间复杂度（其中M =数据插入，N =开放关键字搜索）。

In this post, we describe a scalable approach to building realtime feeds using Meetup's RSVPs as an example with appbase.io, a hosted data streams API. On the backend, we use a Node.JS worker to fetch Meetup's RSVP API and insert them into an appbase.io app. On the frontend, we use jQuery (an alternate post with React.JS instead coming soon) to create the feed UI and query appbase.io in realtime with different keyword filters.

在本文中，我们将介绍一种可扩展的方法，该方法使用Meetup的RSVP作为托管数据流API appbase.io的示例来构建实时提要。 在后端，我们使用Node.JS工作程序来获取Meetup的RSVP API并将其插入到appbase.io应用程序中。 在前端，我们使用jQuery（使用React.JS的另一篇替代文章，而不是即将发布）来创建feed UI并使用不同的关键字过滤器实时查询appbase.io。

输入meetupblast！ ( Enter meetupblast! )

Meetupblast shows a live feed of meetup RSVPs searchable by cities and topics. Like Twitter Live Search, it returns a dynamic feed of RSVP data by the selected cities and topics. It's a great way to find cool meetups other people are attending in your city.

Meetupblast显示了可以按城市和主题搜索的Meetup RSVP的实时提要。 像Twitter Live Search一样，它返回选定城市和主题的RSVP数据动态提要。 这是找到其他人正在您城市参加的酷聚会的好方法。

(

http://appbaseio.github.io/meetupblast/)

http://appbaseio.github.io/meetupblast/ ）

Image: Screenshot of meetupblast, tap on the image for redirection to the live demo.

图片： meetupblast的屏幕截图，点击图片可重定向到实时演示。

Or continue reading to see how we built this.

或继续阅读以了解我们是如何构建的。

关键成分 (Key Ingredients)

The meetupblast recipe can be broken into two key pieces: the backend worker and the user interface.

metupblast配方可以分为两个关键部分：后端工作程序和用户界面。

1. Backend Worker:
   • We use the Meetup's long-polling RSVP stream endpoint to get realtime RSVPs.
     Note: This demo wouldn't have been possible without meetup supporting an API endpoint to extract the RSVP information. So, cheers to them!
   • We then store this data in appbase.io, which provides us a convenient way to query both historical data and realtime feeds - we call it the streaming DB ;)
   后端工作者：
   • 我们使用Meetup的长轮询RSVP流端点来获取实时RSVP 。
     Note ：如果没有聚会支持API端点来提取RSVP信息，则无法进行此演示。 因此，为他们加油！
   • 然后，我们将这些数据存储在appbase.io中 ，这为我们提供了一种方便的方式来查询历史数据和实时提要-我们将其称为流式数据库;）
2. User Interface:
   • Querying appbase.io for Meetup RSVP feeds by cities and topics.
   • The UI / UX logic maintained with a jQuery frontend. And we use typeahead for displaying the search interface.
   用户界面：
   • 通过城市和主题查询appbase.io以获取Meetup RSVP提要。
   • 用jQuery前端维护的UI / UX逻辑。 我们使用typeahead来显示搜索界面。

什么是metupblast？ ( What is meetupblast? )

Meetupblast shows live RSVPs from people across the globe. It has some neat filters that allow you to select cities and the topics of interest, to see what meetups are happening in your city.

Meetupblast展示了来自全球各地的实时RSVP 。 它具有一些简洁的过滤器，可让您选择城市和感兴趣的主题，以查看您的城市正在发生的聚会。

Image: Let's check for RSVPs in the all the singles tagged meetups in San Francisco.

图片 ：让我们检查旧金山所有带有singles标签的聚会中的RSVP。

Now when a person RSVPs for a singles meetup, you would see it show up on the meetupblast feed. Now don't get any ideas for stalking unsuspecting people. It's just a cool way to find the most happening meetups!

现在，当某人通过RSVP进行singles聚会时，您会看到它出现在metupblast饲料中。 现在没有任何想法来跟踪毫无戒心的人。 这是找到发生率最高的聚会的好方法！

A fun fact: While building this app, we noticed that there are about 120 RSVPs on meetup.com every minute, that's 2 every second. Or in 30 days, we would see over 5 million RSVPs - that's a good data set size to query in realtime.

一个有趣的事实：在构建此应用程序时，我们注意到metup.com上每分钟大约有120个RSVP ，即每秒2个。 或在30天之内，我们将看到超过500万个RSVP-这是一个可以实时查询的不错的数据集大小。

深潜 ( Deep Dive )

Now that we know what meetupblast does, let's get into the meat of how the app works.

现在我们知道metupblast的作用，让我们深入了解该应用程序的工作原理。

后端工作者 (Backend Worker)

Our backend worker is a simple Node.JS code that keeps running forever on a DigitalOcean droplet.

我们的后端工作人员是一个简单的Node.JS代码，该代码可以在DigitalOcean Droplet上永久运行。

Let's see how our worker consumes meetup RSVP data and then stores it into appbase.io.

让我们看看我们的工作人员如何使用Meetup RSVP数据，然后将其存储到appbase.io中。

http.get(meetup_url, function(res) {
    res.on('data', function(chunk) { // 'data' event handler is called everytime there is a new meetup RSVP
        var data = JSON.parse(chunk);
        meetup_data.push(data);  // capture the RSVP responses in an array
    });
});

appbaseRef.index({
    type: DATA_TABLE,            // collection to store the data into
    body: meetup_data[counter]
}).on('data', function(res) {
    console.log("successfully indexed one meetup RSVP");
})

Meetup provides us a JSON object for every RSVP. We then write this data into appbase.io as soon as it arrives. appbase.io is built as a streaming layer on ElasticSearch, and provides a continuous query interface for streaming JSON results.

Meetup为每个RSVP提供了一个JSON对象。 然后，我们会在数据到达后立即将其写入appbase.io。 appbase.io被构建为ElasticSearch上的流传输层，并提供用于流传输JSON结果的连续查询接口。

An RSVP JSON data looks like this:

RSVP JSON数据如下所示：

"visibility": "public",
"response": "yes",
"guests": 0,
"member": {
    "member_id": 185034988,
    "photo": "http://photos1.meetupstatic.com/photos/member/d/0/0/0/thumb_248333248.jpeg",
    "member_name": "Wilma"
},
"rsvp_id": 1566804180,
"mtime": 1440266230993,
"event": {
      "event_name": "Wednesday Badminton @ Flying Dragon",
      "event_id": "224809211",
      "time": 1440630000000,
    "event_url": "http://www.meetup.com/Richmond-Hill-Friends/events/224809211/"
},
"group": {
    "group_topics": [
    {
        "urlkey": "social",
        "topic_name": "Social"
    },
    {
        "urlkey": "board-games",
        "topic_name": "Board Games"
    },
    {
        "urlkey": "movie-nights",
        "topic_name": "Movie Nights"
    }
    ],
"group_city": "Richmond Hill",
"group_country": "ca",
"group_id": 1676043,
"group_name": "Richmond Hill Friends",
"group_lon": -79.4,
"group_urlname": "Richmond-Hill-Friends",
"group_state": "ON",
"group_lat": 43.84
}

用户界面 (User Interface)

The User Interface is a small frontend that queries appbase.io for realtime meetups based on specific cities and tags and displays them in a neat list.

用户界面是一个小型前端，可以根据特定的城市和标签向 appbase.io查询实时聚会，并在整洁的列表中显示它们。

Image: Showing meetups by city and topic tags - try it live here.

图片 ：按城市和主题标签显示聚会- 在此处尝试生活 。

The app directory structure looks like this:

应用程序目录结构如下所示：

meetupblast/
|_ _ _ _ app/
|        |_ _ _ _ app.js
|        |_ _ _ _ request.js
|        |_ _ _ _ variable.js
|_ _ _ _ index.html

The codebase can be accessed at the meetupblast github repo.

可以在metupblast github repo上访问该代码库。

app.js

app.js

This is the entry point of the codebase and brokers responsibility to variable.js and request.js.

这是代码库的入口点，由代理承担起variable.js和request.js的责任。

request.js

request.js

Creates a meetup_request prototype which is responsible for three different queries.

创建一个meetup_request原型，该原型负责三个不同的查询。

1. Fetching the list of the 1000 most popular cities and topic where meetups happen,
   获取举行聚会的1000个最受欢迎的城市和主题的列表，
2. A realtime query fetching live meetup data from the selected cities and topics,
   实时查询可从选定的城市和主题中获取实时聚会数据，
3. Pagination showing older RSVPs when a user scrolls to the bottom of the meetup feed.
   当用户滚动到聚会提要的底部时，分页显示较旧的RSVP。

variable.js

variable.js

Instantiates a meetup prototype object which is responsible for the tag selection logic when a user ticks a city / topic and displaying a new meetup RSVP inside the feed.

实例化一个Meetup原型对象，该对象在用户勾选一个城市/主题并在Feed中显示新的Meetup RSVP时负责标签选择逻辑。

潜入连续查询 (A sneakpeak into continuous queries)

Here's a sneakpeak into how the continuous query for fetching live meetup data works.

这是一个偷偷摸摸的话题，用于获取实时聚会数据的连续查询是如何工作的。

We use the appbase-js library (available via bower and npm) for making continuous queries over the meetup feed data.

我们使用appbase-js库 （可通过bower和npm ）对metup feed数据进行连续查询。

appbaseRef= new Appbase({
    url: "https://qz4ZD8xq1:a0edfc7f-5611-46f6-8fe1-d4db234631f3@scalr.api.appbase.io",
    appname: "meetup2"
})

/* A query object for meetups in the cities 'San Francisco' and 'Boston' */
var $composeQuery = {
  "filtered": {
    "filter" : {
      "terms" : { 
        "group_city_simple" : ["San Francisco", "Boston"]
      }
    }
  }
}

/* A query request to fetch initial results */
appbaseRef.search({
  type: "meetup",
  size: 100,
  body: {
    query: $composeQuery
  }
}).on('data', function(res) {
  // initial matches are returned in the array 'res.hits.hits'
  console.log("initial search results: ", res.hits.hits);
});

/* Same query request for streaming realtime results */
appbaseRef.searchStream({
  type: "meetup",
  body: {
    query: $composeQuery
  }
}).on('data', function(res) {
    // realtime updates with one new RSVP
    console.log("new data feed: ", res);
})

search() and searchStream() are alter-egos, the former returns the initial query matches and the latter streams RSVP updates in realtime.

search()和searchStream()是alter- searchStream() ，前者返回初始查询匹配项，后者实时流式传输RSVP更新。

摘要 ( Summary )

We created meetupblast as a way to find happening meetups by looking up what other people are RSVPing to.

我们创建了metupblast，是通过查找其他人向RSVP发送的内容来查找正在发生的聚会的方式。

1. We use the Meetup's long-polling API to get the live data of meetups.
   我们使用Meetup的长轮询API来获取Meetup的实时数据。
2. We use appbase.io to transform this into queryable live data.
   我们使用appbase.io将其转换为可查询的实时数据。
3. We put together a small frontend that allows neat filtering based on cities and topics.
   我们汇总了一个小的前端，可以根据城市和主题进行整洁的过滤。

Without further ado, here are the important links: meetup worker code, meetupblast and the demo.

事不宜迟 ，这里是重要的链接： Meetup 工作人员代码 ， metupblast和演示 。

For the curious, try building a date filter which only shows RSVP feeds between a specific date range.

出于好奇，请尝试构建日期过滤器，该过滤器仅显示特定日期范围之间的RSVP提要。

翻译自: https://scotch.io/tutorials/build-a-twitter-like-live-search-feed-with-appbase-io-and-javascript

appbase