Liferay And Twitter

Requirements: In our Liferay instance, we need to display organizations' tweets in twitter portlet. Those organizations' twitter account is configured in organization's twitter custom field.

Before I develop our twitter portlet, I did some research on Twitter portlet(6.1.10) because we use Liferay 6.1.10. You can go tohttp://www.liferay.com/marketplace to download the newest one. 

Let's deploy and try it first!

• Put the war file into your bundles/deploy package if you use Tomcat.
• Start your Tomcat, login in to your Liferay.
• Select Add->Social->Twitter, add it to your public or private page.
• Twitter Portlet will let you set up an Twitter account, this will go to Liferay component which is under Control Panel->My Account->Social Network. Here you can configure Twitter Account(Your TwitterScreenName). 

For example: 

When I login in Twitter and Click my account photo or my name.  The link below will be shown in the browser bar.

https://twitter.com/RujuanXing, Then the TwitterScreenName will  be RujuanXing

• Then you go back to the page you've added Twitter portlet. It'll say "Your activities will be shown in Activities Portlet". Now you need to add Activities which is under Social category in your portal.
• If you haven't done anything in your portal like posting a blog. Activities Portlet will display something like "No activities Now". Next, you log in Twitter, and Tweet something and go back to your portal. Refresh it, Activities portlet still complain "No activities"! If you're lucky, your tweets will be displayed in Activities portlet. I was not the lucky one. I spent a lot of time on figuring it out.

Let me explain how it works!

1. When we deploy twitter-portlet.war. It'll create a table which is named twitter_feed. The table is used to store twitter screen names which we configure in Control Panel.

2. Twitter portlet schedule a job to fetch Tweets from Twitter REST API. The job is configured in liferay-portlet.xml

3. When the user in the table tweets in Twitter, Twitter Portlet will get the data and store in socialactivitiy table which is a default table in Liferay.

4. When we add Activities portlet in our page, Activities portlet will read socialactivitiy table and display data.

Obviously, the Twitter portlet doesn't suit my case. It has some shortcomings.

1. Twitter portlet is using REST API. REST API has rate limiting. Unauthenticated user calls are permitted 150 request one hour. OAuth calls are permitted 350 requests per hour. For more details, please refer to https://dev.twitter.com/docs/rate-limiting/1.1. Obviously, this is a key issue. 

2. REST call URL is hard coded in class. In case, Twitter changes API, we need to change the code and compile and deploy. Actually, this is the issue why twitter portlet didn't work on my case. It uses legacy Twitter REST API like(https://twitter.com/statuses/user_timeline/rujuanxing.json) which is used to fetch tweets from Twitter.com. In March of 2012, Twitter announced their new API and in early October 2012, they turned off old endpoints. Since we use Liferay EE 6.1.10, I can't use the newest version of Twitter portlet. I did change the code and compile and redeploy. It's time consuming. If we could put the API in properties file, what we need to do is only to change the properties file.

3. From the api twitter portlet uses, it doesn't fetch the tweets which users retweet others.

4. We're not able to configure two or more twitter account in the twitter portlet. That means we only see one user's tweets. 

5. All tweets are shown in Activities portlet. We know if we blog or do other activities, those activities also are shown in Activities portlet. The twitter portlet doesn't provide a clean page to only display tweets.

Because of those shortcomings, we decided to develop our own twitter-portlet. Here I just explain the idea of how to do it.

First, we need to solve rate limiting issue. Fortunately, Twitter.com provides another API: Streaming API which means we set up a long live connection to Twitter.com. Whenever our listened twitter account tweets on Twitter.com, Twitter.com will push the tweets to us. Then we can do whatever we want. For more info, refer to https://dev.twitter.com/docs/streaming-apis

When we use REST API, tweets are stored in Twitter.com. If we use Streaming API, we need a place to store those tweets. In order to make those things work, we need to design some tables to store those tweets. We design three tables: organizationstatus, tweet and feed table.

• organizationstatus table: it is used to store twitter screen name and organization id which is configured in Liferay.
• feed table: it is used to store twitter user id and organization id.
• tweet table: it is used to store tweet from Twitter.

For our applications, we have three projects: twitter-persistence, twitter standalone app and twitter portlet.

• twitter-persistence: This is app is used to do all database operations.
• twitter standalone application: This app is used to set up a long live connection to Twitter.com. It listens to a list of twitter accounts. When those accounts tweet on Twitter.com, Twitter.com will push those tweets into our connection. Our standalone app will store those tweets in tweet table. Because our listening list is dynamic, when user adds, updates or deletes organization's twitter account in Liferay, we need to change our listening list. In order to do that, we have a job to check organizationstatus table every 30 minutes(you can specify the time you want), when it finds the table is changed, it'll make the corresponding change into feed table including get the new twitter user id and then change the listening list.
• twitter-portlet: It has two functions: one is to display tweets, the other is actually a hook which stores organization's twitter screen into organizationstatus table. When user logs in Liferay, we'll get the user's organizations, if there's no tweets with current organization, we'll display the organization's parent organization's tweets. When admin add/update/delete organization's twitter account, we make a update to organizationstatus table.

Second, how can we set up a long live connection to Twitter.com? Previously, I was thinking I need to start coding from scratch. The good thing is I found twitter4j which is an unofficial Java Library for Twitter API. You can download jar files and demo here.

Let me share a small demo with you. This small demo is used to get the tweets which we specify in the filter. In the example, I only listen to RujuanXing and CNN.

public class PrintFilterStream {

	public static void printTweets(long[] followArray) {

		StatusListener listener = new StatusListener() {
			//This method is used to get tweets including the tweets which user retweets.
			@Override
			public void onStatus(Status status) {
				System.out.println("@" + status.getUser().getScreenName() + " - "
						+ status.getText());
			}

			@Override
			public void onDeletionNotice(StatusDeletionNotice statusDeletionNotice) {
				System.out.println("Got a status deletion notice id:"
						+ statusDeletionNotice.getStatusId());
			}

			@Override
			public void onTrackLimitationNotice(int numberOfLimitedStatuses) {
				System.out.println("Got track limitation notice:" + numberOfLimitedStatuses);
			}

			@Override
			public void onScrubGeo(long userId, long upToStatusId) {
				System.out.println("Got scrub_geo event userId:" + userId + " upToStatusId:"
						+ upToStatusId);
			}

			@Override
			public void onStallWarning(StallWarning warning) {
				System.out.println("Got stall warning:" + warning);
			}

			@Override
			public void onException(Exception ex) {
				ex.printStackTrace();
			}
		};

		TwitterStream twitterStream = new TwitterStreamFactory().getInstance();
		twitterStream.addListener(listener);
		// filter() method internally creates a thread which manipulates
		// TwitterStream and calls these adequate listener methods continuously.
		twitterStream.filter(new FilterQuery(followArray));
	}

	public static void main(String[] args) throws TwitterException {
		//We have to pass twitter user id into FilterQuerey method.
		long[] followArray = { 935445715, 759251 }; // RujuanXing 935445715, CNN 759251
		printTweets(followArray);
	}

}

Another question is how to get twitter userId based on twitter screen name. It's really easy to do that when we use twitter4j.

public static void main(String[] args) throws TwitterException {
		String twitterScreenName = "RujuanXing";
		Twitter twitter = new TwitterFactory().getInstance();
        User user = twitter.showUser(twitterScreenName); 
        long twitterUserId = user.getId();
}

Now we've gotten everything we want, you can start to build your own twitter portlet!