Gilad Lotan » processing

Seeing a Twitter #Hashtag Spread

gilad — Wed, 20 Jan 2010 20:51:05 +0000

#CheeringForTheYankeesIsLike is a hashtag created by @mattsly the morning of October 26th. He submitted the following snarky message – ‘Go Phillies. #CheeringForTheYankeesIsLike hoping investment bankers get really huge bonuses of at least 8 figures‘ – hoping to entertain his friends, and possibly get others to participate. Matt had 182 followers at the time, not sizeable by any means on Twitter. Little did he expect that some 9 hours later, 271 different users, most of whom have no connection to him whatsoever, would participate, posting around 500 messages in total.

How did this happen and what prompted this message to spread?

#CheeringForTheYankeesIsLike

About an hour after Matt sent out his first message, one of his followers, @lizzieohreally, wrote the following message ‘@jaketapper? @abcdude? …Hoping someone w/ more Twitter than I can help popularize #CheeringForTheYankeesIsLike (via @mattsly)‘. Lizzie clearly understood that in order to get many others to play, she would have to get someone with a large set of followers to participate. Lizzie had only around 500 followers at the time, so posted this message in an attempt to seek @jaketapper or @abcdude’s attention.

Sure thing, some twenty minutes later, @abcdude see’s the message and adds his own variation to the meme: ‘#cheeringfortheyankeesislike pulling for Regina George in “Mean Girls.”‘ He enjoys it so much that he promptly posts another message and attaches the hashtag. @abcdude is a new york based correspondent for ABC news. He dubs himself a RedSox fan and a cosmic power broker. Not as cosmic as Lizzie had hoped, but still, he has some 7,000 followers, which could certainly help give the meme some traction. We see a small spike after @abcdude’s participation, and by now, some 3 hours after Matt sent the original message, there have been 34 different messages posted with this unique hashtag.

But it wasn’t until @jaketapper joined in that the conversation really took off. The hashtag came to Jake’s attention after @DetourJazz, whom he follows, participated. Jake reacted by posting: ’RT @DetourJazz: #cheeringfortheyankeesislike rooting for “Craterface” in Grease to beat Danny (via @Laura_Martin)’. He then added a new message that he posted to his followers. Jake is a senior White House correspondent for ABC news with over 30,000 followers. Before he took part in this meme, new posts appeared at a frequency of one every 5 minutes. Immediately after he joined, we see a sharp rise in participation, with multiple messages from a variety of users every minute.

Seeing it Spread

1. Graphing the Network – Every user who participated in the meme is represented by a gray circle (Matt, whom first started the meme, is shown in yellow). Edges represents the person who most likely influenced the other to first participate.

2. Seeing the Flow – in this applet, a user is represented by their twitter icon. As the timeline moves forward, each profile lights up when they post a new message with the hashtag. Tthe moment that @jaketapper chose to participate is evident – there’s a clear, sudden spike in participation after his profile picture lights up.

3. Seeing the distance – the following applet highlights the total social distance that this hashtag traveled between users. Each user is represented by a circle, the more influence a user has, the larger their circle is drawn. Edges in this example represent the social ties – when there’s a follower/friend relationship between two users, a line is placed between their representation on the screen. The first column includes only Matt who first used the hashtag. The second row consists of only those people he directly influenced to participate (his followers). While there are a total of 9 columns, it is crystal clear that the most important phase happened in the second and third column, when a core cluster of users chose to participate, and a mini tipping point was reached.

Parsing the Data

#CheeringForTheYankeesIsLike lasted for a total of 9 hours that day, activated 271 different users and included around 500 messages in total. From looking at this meme, it is clear that on Twitter, there’s great advantage to having many followers if one intends to spread a message. It is also clear that having the right followers is key. If it were not for @lizzieohreally who knew to actively pass the message onwards to heavy Twitter users, the meme would never have spread out the way it did. In order to come to these conclusions it was necessary for me to look at social ties in addition to the semantics of the messages posted.

I used the Twitter API to discover the follower/friend relationships between all users who participated in this meme. This is extremely important data, especially when modeling the flow of participation and influence within this hashtag. For example, lets look at a simple case where user B follows user A. If user A first participates and is followed by user B participating, user A is rewarded some number of influence points – this is assuming user B saw the hashtag posted by user A, and decided to participate. Additionally, if a user is retweeted or ‘@’ messaged they are rewarded some number of influence points. Real life situations can easily become complicated, as user B might also be following user C, who participated in the meme as well. Now how do we know if user B was influenced by user A or user C? Hard to tell, but we can build an influence model that takes these situations into account, which is exactly what I did.

Translating the semantics and social ties from the dataset into a visual language that made sense was key to helping me understand this hashtag experiment. I am a big fan of visualization as a means to parse large datasets, however dealing with social, implicit data is tricky, and extremely challenging to represent visually. But when done right, these representations can shine a whole new light and hopefully help us better understand some of the dynamics at play.

Seeing the Elections Buzz

Fri, 29 Feb 2008 13:31:41 +0000

I’ve been working feverishly on a news visualization, trying to capture the buzz and conversations happening within the comment spaces around news articles online. Am very excited at the direction this is heading. I’ve posted my earlier screen-captures a couple of weeks ago. Today I have some images from another part of the visualization – wordLapse. This looks at the most frequently used words over time.

From an initial glance, ‘Buzz’ words are obvious: the first image represents comments from the first week of October, while the second, the last week of February.

More to come, very soon.

[tags] visualization, elections, processing, latimes, nlp [/tags]

Visualization of User-Generated Comments on US Elections

Thu, 24 Jan 2008 12:37:05 +0000

Here’s a glimpse of a work-in-progress. A little project I’m working on (check out the live version) that takes user-generated comments from the LA-times Campaign ‘08 page into a visualization powered by processing. It looks something like this:

The background red/blue colors are directly correlated to the text => the more written about the democrats, the bluer it gets, etc…

This is how the original comments page looks like on the LA times site:

This is a work in progress, so any comments, ideas or thoughts would be more than welcome.

[tags] visualization, processing, elections, US, comments, news, latimes [/tags]

Processing connection (PC) to Bluetooth (BlueSMIRF)

Mon, 23 Oct 2006 05:12:24 +0000

During the past week we’ve been working hard on getting the two to talk in a stable manner. There were many hurdles along the way, but we’ve managed to get a very stable connection between our processing app (running on a PC) and the BlueSMIRF chip from sparkfun, which we connected to a microchip PIC chip (we used the 18F252).

Step 1: Pairing the BlueSMIRF with the computer

In order to use the blueSMIRFS directly with the PC, they need to first plugged into power (ground, and connecting CTS and RTS). Then a connection needs to be established between the PC and the blueSMIRF (password is ‘default’). Later, it is possible to change the blueSMIRF’s name from the initial factory setting to whatever is wanted using microcode studio’s serial communicator.

Since my laptop doesn’t have a built in bluetooth device, I bought a Bluespoon USB dongle at Radioshack (a bit pricy, but is Bluetooth 2.0, and works very well – I found out that some USB Bluetooth dongles don’t work with the BlueSMIRF, so that’s something to consider). This step depends on the bluetooth software included on your computer, but basically at the end of this step, there should be an additional COM port in the hardware device manager (in the PC: right click ‘My Computer’->properties->hardware tab->device manager). When setting up the paired connection, it is usually possible to pick which COM port to use (when using the advanced setup mode). It is extremely important to remember the number of this port. This is the virtual communications port with which we can communicate directly to the specific BlueSMIRF chip from our computer.

Step 2: Setting up the BlueSMIRF

When setting up the configurations for the BlueSMIRF chip, we used the serial communicator provided by the Microcode Studio software (command F4 opens it). There, a direct connection to the BlueSMIRF can be made using the appropriate COM port (from the previous step). Here are some of the AT commands that we’ve used:

‘+++’ – to enter command mode

‘ATSN, ubiach1′ – change its name to ubiach1

‘ATMD’ – exit command mode

Here is a complete list of the blueradios AT command list, direct from the Faludi blog.

Step 3: Building the circuit

It is important to remember the the BlueSMIRF uses TTL logic while the PIC uses RS-232 for its’ serial communications. This will need to be taken into consideration when writing the PIC code (serin and serout commands between the two have to invert the logic – instead of using the 16864 option which we usually use for serial communications, we use 84 in order to implement the inverted logic connection with the BlueSMIRF: serout2 portc.6, 84, [inputData] ). Except for that, we used a 20Mhz clock, as well as a 3.3V regulator (5V can also be used). There are 3 status LEDs: The yellow blinks at startup, the green every iteration of the loop (when there’s no input data coming from the BlueSMIRF), and the red – toggled on and off using the processing code on the computer.

Step 4: The code

We implemented a simple call and response code (built on top of Tom Igoe’s PIC-processing code). There are some important things when writing the processing code, otherwise the communications won’t work. First of all, the appropriate COM port needs to be chosen from the generated list (the same COM port which was paired to the BlueSMIRF device). Sending data to the BlueSMIRF is done using the port.write() command. It is important to send a ‘\r’ when sending a byte to the BlueSMIRF (otherwise it crashes… at least on my computer). It is also important to take note of what you’re sending. The BlueSMIRF has a buffer, but the PIC doesn’t. So if it is more than one byte, a more complex method of communication between the PIC and the BlueSMIRF needs to be implemented (using CTS and RTS or interrupts). Since we only need to send and receive single bytes between the computer and the object that we’re building, it was enough for us to send single bytes (’char’ types). The next step would be to write a more complex bit of code, that would allow for proper transfer of any variable type between the two. Here is the sample code:

PIC Bluetooth Sample Code for the PIC.

Processing Sample Bluetooth Code for the Computer.

Next Step: audio

We want to figure out how to stream audio from the computer, through these BlueRadio chips (these are the bluetooth chips used on the BlueSMIRF) and into a physical object which we build. It is possible using more complex AT commands, in addition to other pins, which are not implemented using the BlueSMIRF. We might order this product from sparkfun and start playing around with that.