Configuring and running TwitLogic

TwitLogic is a Java application which can be executed at the command line. It takes one argument: a configuration file. This tells the application:

• which credentials to use when interacting with Twitter’s API
• which Twitter users and lists to follow, and which keywords to track for new tweets. You may follow up to 400 users at a time.
• where to put the triple store, and which triple store implementation to use
• if you would like to publish the stream as Linked Data, the base URI of the server and the port to be used
• if you would like to create periodic dump files, where to put them and how often to generate them

There is an example configuration in TwitLogic’s main directory, which you’ll find after unpacking twitlogic-core-xxx.zip. If you are running ’nix or Mac, you can navigate into the twitlogic-core directory at the command line and type:

./twitlogic.sh example.properties

However, you will need to edit example.properties first, so that these four lines contain actual Twitter OAuth credentials:

net.fortytwo.twitlogic.twitter.consumerKey = [application key]
net.fortytwo.twitlogic.twitter.consumerSecret = [application secret]
net.fortytwo.twitlogic.twitter.accessToken = [user token]
net.fortytwo.twitlogic.twitter.accessTokenSecret = [user secret]

In order for TwitLogic to communicate with Twitter’s Streaming API, you will need to register a Twitter application, which you can do here, and give the application permission to access the API on behalf of a specific user. Once you have registered the application, Twitter will give you the consumer key and consumer secret values you need for the first two properties above. Next, add a user to get the access token and access token secret values. Currently, Twitter provides a “Create my access token” button at the bottom of the application page, which you can use to grant permissions for the user who registered the application.

Note that there is also a utility, TwitterCredentials, which you can use to grant permissions to your application for any given user. Just copy your configuration (with the app’s OAuth info) to /tmp/twitlogic.properties and run the utility.

After you have filled in your credentials, start the script, and TwitLogic will create a triple store and begin listening to the Twitter stream. To customize the users TwitLogic follows and the keywords it tracks, change these properties and restart:

net.fortytwo.twitlogic.followList0 = twit_logic/listenlist
net.fortytwo.twitlogic.followUser0 = twit_logic
net.fortytwo.twitlogic.trackTerms0 = #semanticweb, #linkeddata

You will receive tweets from any Twitter user and all users in any Twitter list, as well as tweets containing any of the keyword terms, specified here. You can remove these lists, individuals and terms or just add some more:

net.fortytwo.twitlogic.followList2 = joshsh/twcrpi
net.fortytwo.twitlogic.followUser1 = abevigoda
net.fortytwo.twitlogic.followUser2 = conanobrien
net.fortytwo.twitlogic.trackTermsX = FIFA, #worldcup, soccer, football

TwitLogic’s logging output will allow you to watch the tweets as they come in. But where do the RDF triples end up? At the moment, they end up in the /tmp directory:

net.fortytwo.twitlogic.persistence.nativeStoreDirectory = /tmp/twitlogic/data

It’s best to change this to a more permanent location. Finally, if you prefer RDF documents to the NativeStore triple store, pay attention to these properties, which tell TwitLogic to create a dump file every five minutes:

net.fortytwo.twitlogic.persistence.dump.file = /tmp/twitlogic/website/dump/twitlogic-full.trig.gz
net.fortytwo.twitlogic.persistence.dump.interval = 300000

This causes TwitLogic to output a dump file in one of several common RDF formats. You can change both the location of the file and the time (in milliseconds) between dumps. The file extension determines the format of the resulting file. For example, this will cause an uncompressed RDF/XML file to be created:

net.fortytwo.twitlogic.persistence.dump.file = /home/arthurdent/tweets.rdf

Finally, if you want to serve your aggregated data as Linked Data, you should change these properties:

net.fortytwo.twitlogic.server.baseURI = http://localhost:8182/
net.fortytwo.twitlogic.server.port = 8182
net.fortytwo.twitlogic.server.staticContentDirectory = /tmp/twitlogic/website

Now execute the script twitlogic-server.sh instead of twitlogic.sh. Note that the TwitLogic client demo and TwitLogic server demo rdf-ize all tweets, even when no embedded nanoformatting is detected. You can change this behavior, as well as build new applications entirely, by writing your own version of this file. The TwitLogic API should give you an idea of what is possible.