Jesse, Separated

Twittersphere Graph

So you wanna make a TwitterSphere graph, a la those made by acostoss and Gargron?

Requirements

  • A linux computer with a stable internet connection, hereby called the Fetcher
  • A twitter account
  • A computer (running Windows, OSX, or Linux) with a decent amount of RAM (suggested 4GB or more), hereby called the Renderer. Can be the same computer as the Fetcher
  • Java 7 (NOT 8!)
  • RVM

For the purposes of this guide I will assume you are using Debian on your Fetcher, though this works out for most any Linux distro under the sun. On the renderer I used Windows, though the guide is platform-agnostic.

Fetcher Prep

Ruby 2.2.0 and Bundler

At the time of writing, Debian stable has 1.8.x in it's repos. The fetcher requires Ruby 2.2.0, which we'll be installing via RVM.

First, we need to add mpiapis public key with the following (req gpg2)

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3

Then, we install the stable version of rvm

\curl -sSL https://get.rvm.io | bash -s stable

Finally, we install Ruby 2.2.0

rvm install 2.2.0

Once Ruby is installed, we need to install the bundler gem via

gem install bundler

Compiling Twittersphere

Now we're ready to compile twittersphere. If you don't already have git, install it via your distro's package manager. For example, in Debian:

apt-get install git

Next, we're going to clone Gargron's Twittersphere repo on Github. From you home directory (or wherever you have RW permissions), do:

git clone https://github.com/Gargron/twittersphere.git

This will clone the Twittersphere repo to the twittersphere folder in your current directory.

Change directories to that folder with

cd twittersphere

Next, just run

bundle install

and watch the text flow by.

Twitter Application

Once that is finished, we need to obtain some consumer keys nad access tokens from Twitter. We doing this by maknig a Twitter app. Navigate to Twitter Application Management and click the "Create a New App" button. Fill out the required fields on the form and Agree to the terms to make your new twitter app.

Navigate to your new twitter app, and click the "Keys and Access Tokens" tab. In here, you'll find your Consumer Key and Secret. Copy these somewhere safe for now. At the bottom, click the "Create my Access Token" button to get your Token and Token Secret.

While still in your Twittersphere directory, do

touch .env

then open this file with your favorite text editor (pico, vim, emacs, etc) and put in the following.

TW_CONSUMER_KEY:
TW_CONSUMER_SECRET:
TW_ACCESS_TOKEN:
TW_ACCESS_TOKEN_SECRET:

Add your Key, Secret, Token, and Token Secret on their respective lines, and save.

Fetching and processing our data

If everything was done correctly, we should be ready to fetch our data. This process can take a LONG time due to twitter API limits, so once you've begun fetching it might be a good idea to getting your rendering computer set up. Run the following, replacing "acostoss" with your twitter username

./bin/twis fetch acostoss

Depending on how many people you follow and how many people they follow, this can take a while due to Twitter's rate limiting. Just let it do it's thing, peeknig in every once in a while to see if it has finished.

When it has finally completed, run the following, repalcing "acostoss" with your username.

./bin/twis process acostoss

This will create a file called output.gdf, which you' then want to copy to your rendering computer.

Renderer Setup

First off we need to install Gephi, the Open Graph Visualization Platform. Download and install it via the instructions on their site for your OS. before running Gephy, make sure you're also running Java 7. Gephi has issues with the latest Java, Java 8.

Open up the output.gdf file in Gephi, and follow this guide for a basic setup. You can customize by playing with the different filters and different values in the ForceAtlas2 Layout functions. I had good results by applying a K-Core filter and setting the paramaeter high enough until I had ~700 nodes to work with.

Save your file to PNG via File > Export, chosoe PNG as output filetype and set the resolution under the Options button. 8182x8192 works well.

Further Reading

Blog Posts

Gargron himself has posted a post-mortem/guide of his own, which goes into more detail on how the fetcher functions.

Graphs

This article is my 3rd oldest. It is 717 words long.