I’m extremely passionate about data analysis and design. [...]]]> ::Making Sense of the Ebbs and Flow of Social Data

Below are notes + slides of my presentation at the BRANDSconf. I’d like to acknowledge Hunter Whitney. Portions of this content were based on a discussion and an upcoming article he is writing on this topic (link coming shortly):

I’m extremely passionate about data analysis and design. My work focuses on the intersection of the two. I play with data, and figure out ways to make it more accessible to people. I’m here to talk about why the art of making sense of massive amounts of social data is critical not only for geeks like me, but any professional using Twitter. And my goal is to get YOU all excited about the opportunity that understanding data unveils for us.

Whether you’re a multi-national enterprise, a local deli or a mah-jong meetup, the proliferation of social network services like Twitter have created an expectation that you interact with your customers, users and followers. There’s an expectation to connect rather than broadcast. We’ve been hearing this over and over this morning – you are a brand. And as a brand you are expected to interact with your audience like a person would interact with others. You need to engage in conversations, provide and receive feedback, network, create hype, and do all this in a timely manner.

But how can we be expected to interact with an ever growing and diverse group of people when we can’t really “see” them?

Giving Shape to our Audience

Judith Donath of Harvard’s Berkman Center talks about human signaling and how that translates to digital spaces. I get a variety of signals from merely standing in front of you all – your age, what you’re wearing, how you’re feeling, whose smiling and whose already fallen asleep. Being here, with you, part of this event, I have context that helps me understand how best to interact with you all. I’ll happily switch to speaking Hebrew, but obviously that will not be helpful. Even the little bit that I know about you helps me make some useful assumptions – speak English, tune down the analytics/mathematics terms, tune up the user experience/brand jargon.

Social network spaces are fueled by social interactions. Think of people’s interactions online as digital breadcrumbs, trails of connections, likes, thoughts and opinions. By piecing together these crumbs we can start making sense of the people giving us attention on Social Network sites. We must use as much of the tools available to mine the data about our audience – location, time of day, language, interests. In order to interact with an audience we need to be able to sense it.

There are a variety of tools that give us this opportunity to mine content. This is only the first step. We need to put an emphasis on looking at the connections between people, and not only the content that is being published.

The Social Graph

Social Graph is a term that I’m certain you all will hear more and more as social network spaces become a fundamental component of our lives. A social graph is a dataset that represents people and their inter-connections within a group. Mark Zuckerberg is known for popularizing the term in his description of the value that Facebook Connect brings to websites. Facebook’s social graph is made up of you all who I’m sure have accounts, and all your connections. Additionally, that graph distinguishes between types of connections – whether colleagues, friends or family.

Twitter’s social graph is different. Its a directed, which means that connections have directions. The person who you follow does not necessarily follow you back. Twitter’s social graph is fascinating because it maps people’s interests; what people are willing to give their attention to. By understanding people’s interests over time as well as their interconnections, we have the ability to identify we can reveal valuable points such as (1) bridges: people who connect two distinct communities (2) influencers: those who can get their audience to participate (3) experts: people who specialize on a specific topic (4) hustlers: culture creators.

While it is fairly straightforward to aggregate large datasets, we are still challenged by making sense of graph based data. These constantly changing graph indexes are massive at scale and may require complex queries in realtime: whats the shortest path between person A and person B, whats the intersection between group C and D or whats the clustering coefficients amongst group E. Once calculated, these results reflect on the intricacies of people’s relationships, and shedding light on properties that directly affect their behavior: influence, trust, authority and personal preference.

Understanding information flows

In the social web, information spreads through people, networks of friends, fans and followers. Social network sites create compelling spaces where users feel comfortable to hang out, interact, consume, poke and publish. Social interactions lubricate the flow of information within these spaces, creating a plethora of dynamics. These spaces are filled with endless streams of content, encouraging users to participate, add to, consume from and redirect content. As information flows by, users grab content when it is most relevant, valuable, entertaining or insightful, and at times, choose to pass it on.

Because information flows through networks of people, attention has become a scarce commodity. This is truly a game changer. Media companies no longer control people’s attention, but are rather fighting for a smaller section of the pie. True power lies in understanding how information flows and its effect of where people choose to focus their attention. In order for messages to propagate through social networks, people along the way must be attentive to the pieces of information, see them at the right time, and pass them onwards.

Whether you’re interested in socializing or in selling a product, understanding people’s habits around information consumption and production is imperative to attaining people’s attention and building an audience. By leveraging the publicly available data around people’s practices, we can create services that shed a light on people’s habits and preferences. Additionally, by mining this data over time, we can infer their value in affecting information flows.

::demo:: seeing a Twitter Hashtag Spread

I’ve been following @jeffpulver for a while now and know that he’s quite generous in terms of attention. A great time to catch Jeff is in the morning (wherever he is), as he sends out a ‘good morning’ Tweet, there tend to be reciprocal pings and messages. I also know Jeff is interested in new developments in the Israeli startup scene. If I have any juicy piece of information on that topic, I’d make sure to post it, possibly with a /cc/ to Jeff, and ideally around his morning time. I have a mental model in my head, around Jeff’s practices in consuming and producing content.

We all do this, but can only capture so much in our heads. We need tools that scale and capture our networks as a whole and not just individuals. Remember, its not necessarily about the size of an audience or someone’s number of followers, but rather who they are and who they’re connected to.

That all sounds really great, but in effect, representing large graph datasets can easily get out of hand, however loved by geeks, usually becomes a tangled mass of lines and dots. We must remember that this data is beneficial only if people are able to make sense of it. We need to think about interfaces that will let us play with the data; slice and dice the parts that we deem relevant or interesting. In addition to an intuitive interface, we need controls that will help us dive into and observe patterns or connections that would have otherwise been hidden.

Closing

There are three points I want to make sure you all come out of this talk thinking about:

1) Mine Digital Breadcrumbs – use the exiting tools to get a sense for how our audience looks and its segmentation (I’ve made a oneforty kit here)

2) Social Graphs are Extremely Useful – yet complex to aggregate and mine.

3) understanding information flows is Powerful – especially as we’re shifting from broadcast mode to that of engagement

Social network analytics tools may fundamentally change the way we engage with our online audiences. We need to build better tools that do the above mentioned tasks. But I need people like you all to be vocal about your needs and frustrations. As we’re building out these technologies, we want to make sure they are tailored to real needs. We’re only at the start of the journey, and I’m super excited to be a part of it!

“The stream is winding its way throughout the Web and organizing it by nowness.”

This real-time stream has been building for a while. It began with RSS, but is now so much stronger and swifter, encompassing not just periodic news and musings [...]]]> Thought provoking article on Techcrunch about the shift from dedicated web pages to real-time streams.

“The stream is winding its way throughout the Web and organizing it by nowness.”

This real-time stream has been building for a while. It began with RSS, but is now so much stronger and swifter, encompassing not just periodic news and musings but constant communication, status updates, instantly shared thoughts, photos and videos.

The author presents a coherent stream metaphor: “A real time, flowing, dynamic stream of information – that we as users and participants can dip in and out of and whether we participate in them or simply observe we are a part of this flow.” He claims that the stream does not replace Web pages or search, for that matter, but it has the potential to completely transform them:

“Traffic occurs in bursts, depending on what people are paying attention to at that second across a variety of services. Someone might notice an obscure blog post on Twitter, where it starts spreading, then it moves to FriendFeed and Facebook and desktop stream readers such as Tweetdeck or Seesmic desktop and before you know it, a hundred thousand people are reading that article. The stream creates a different form of syndication which cannot be licensed and cannot be controlled.”

“The problem, more than ever before, becomes one of information overload. How do you keep from drowning in the deluge? Borthwick suggests letting go of the notion that you can ever master the stream, even just your own personal data stream of friend’s Tweets, updates, blog posts, Flickr photos, YouTube video finds and so on:

This isn’t an inbox we have to empty, or a page we have to get to the bottom of — its a flow of data that we can dip into at will but we can’t attempt to gain an all encompassing view of it.”

This is where I have to somewhat disagree. Yes, the average user must deal with information overload more than ever. Our emphasis must not be focused towards this notion of “giving up” or realizing that we can ‘never master the stream’. We need to build smart filtering mechanisms that help us navigate this overload and data-heavy information ether. We need to build systems that help us know when and where we *should* hop into the stream. We need to be able to set and identify levels of immediacy – I am not willing to miss out on any content from my closest friends, but will easily let other content slide by. Why can I not easily set preferences across the board and am constantly prompted to setup less-than ideal preferences within walled gardens?

“So jump into the stream and let it carry you away. Or you can stand timidly on the banks until everyone else around you has already taken the plunge.”

The information overload “problem” is that of smart filtering, and in a way, the article’s last sentence portrays one of the most promising filtering mechanisms – social & collaborative. By letting people you trust around you do some of the work, it becomes much easier for users to take the plunge and step into the stream.

So start classifying y’all. I want to be able to finally deal with all my feeds!

[tags]filtering,overload,web,rss[/tags]

The Art Center College [...]]]> For the past several months I’ve been working with the living profiles team at the Art Center College of Design. The project is part of the larger HealthDesign initiative, designing new open-source methods for keeping personal health records and sponsored by the Robert Wood Johnson Foundation. Here’s a framing of our project:

The Art Center College of Design in Pasadena is developing a PHR application to help adolescents with chronic illnesses transition from pediatric to the adult care system, in which these young patients will assume greater responsibility for their health and their personal health information. The project team is working on an aggregate set of tools that gathers and integrates discrete data. By tapping into exhibited teen behavior such as texting and emotional connectivity through music, the tools seamlessly incorporate into their every day world improving communication with their caregivers.

The project is divided into several stages. For the first, we will prototype and build a moodmeter, for which the ultimate goal is to find an effective and engaging method to capture user’s moods and feelings through their daily lives. The second part of this application will include visualization of the information. The biggest challenge, I think, is creating an engaging interaction, one that will make its users want to come back and use it frequently – to create a sticky application. Our user demographics will be teens, for whom applications become sticky *mostly* because of social circumstances: they hang out in myspace or facebook because their friends are there. They participate, write comments, rate, and more, because their friends see it. One’s mood is personal. Especially when dealing with teens with cronic illnesses (the focus of this project). They will not necessarily want to share this information with all their friends. We will need to find another way to create an engaging experience. A sticky app. Perhaps in the form of a game, or with constant reminders.

But before we start exploring that, I have a more basic, underlying question. What constitutes a mood? Is hungry a mood? tired? Emotions are more easily accepted as moods. But what about physical conditions?

I fished around for a bit and found some interesting ways people use existing applications to display their “moods”. People generally use IM consoles to give some form of context to their presence. From what they feel (”head hurts”, “bored la”) to where they are (”LA”, “Away”).

moodgets is an attempt to create a sticky application that allows its users to share their feelings with friends. The site helps you create a moodget (personalized widget including an emoticon and some text) and has buttons which allow you to post it to chosen sns-es (although it didn’t work with my myspace), or provides you with the HTML code to post on the web. Very easy to use. Problem: you need to go to their website in order to edit. Here’s what I did:

twitter:: what are you doing? a very open ended question. Used differently by groups of people. An extremely simple way to get content from your phone to the web. However, for capturing one’s mood, the feedback structure is not engaging enough.

facebook :: has a status update very similar to twitter’s ‘what are you doing’. With the addition of Facebook’s SMS shortcode, it is now possible to update your status via SMS (to FBOOK).
Through the facebook apps, though, I found the moods application. It gives you a list of moods and lets you pick which one to display on your page. It also, annoyingly, asks you to write “why you feel this way” so your friends could see it. AArgh.

So explicit… And so fake. What will it take to truly capture someone’s mood? Picking emoticons is not enough. I want the application to understand the implicit. I don’t want to pick my emotions out of a category. How is it at all possible? That’s the big question for me in the weeks to come.

]]> http://giladlotan.com/blog/2008/01/project-living-profiles-the-moodmeter/feed/ 2 http://giladlotan.com/blog/2008/01/on-transactive-memory-and-why-i-disslike-twitter/ http://giladlotan.com/blog/2008/01/on-transactive-memory-and-why-i-disslike-twitter/#comments Fri, 04 Jan 2008 05:31:57 +0000 http://giladlotan.com/blog/?p=213 Overload [def]:: to load to excess; overburden; fill to excess so that function is impaired – precisely how i feel about social networking sites recently. Plunged from their initial excitement to being yet another procrastination tool – one that minimizes the already little amount of sincere conversations I have with friends. Even though Twitter seems [...]]]> Overload [def]:: to load to excess; overburden; fill to excess so that function is impaired – precisely how i feel about social networking sites recently. Plunged from their initial excitement to being yet another procrastination tool – one that minimizes the already little amount of sincere conversations I have with friends. Even though Twitter seems like another perfectly good tool for me to stay in touch with friends, I cringe.

The question that has been on my mind recently is why I dislike Twitter so much and what it is that I’m looking for in current web technology. In his book, ‘The Tipping Point‘, Malcolm Gladwell writes about an intellectual and emotional capacity we have before we become overwhelmed. A limited ability to process so much raw information. He defines a sympathy group – people whose death would leave us truly devastated:

Why isn’t this group larger? Its a question of time. If you look at the names on your sympathy list, they are probably the people whom you devote the most attention to – phone, in person or thinking and worrying about. If twice as long – you spend less time with people.
To be someone’s close friend requires a minimum investment of time. Moreover, it takes emotional energy. Caring about someone deeply is exhausting. At a certain point, at somewhere between 10 and 15 people, we begin to overload. (source)

Most of the people in my sympathy group live thousands of miles away; many do not ‘hang out’ online, nor are they dedicated social network site users. I want more tools to help maintain intimate relationships with them. While it is much easier to “stay in touch”, I feel like the available tools draw me further away, and leave me with only the superficial conversations. I long for tools that support sincere interactions – other than phone conversations (which are still costly). Ways to connect truly and sincerely. Even though facebook has pretty much spanned all my networks, it is becoming too big too fast and way too cluttered. Too many friends mixed up all together in a conglomerated mess, with too much spam.

What I dislike most about the currently available services is the philosophy in which one acknowledges social interactions through an egocentric self – one that gives credit to gathering a crew of followers (Twitter’s terminology). I am truly annoyed by everyone’s acceptance of this terminology – ‘followers’ and the web’s support of a self-centered method to socialize; a way that does not work in the real world. Collecting a group of defined ‘followers’ and publishing ME-centric feeds does not work for close friends.

Ori Neidich defines a Twitter feed as ‘an IRC channel, just with a very very long lag time…‘. It is essentially a multi-cast SMS message, that is also placed online. The really great thing about it is the fact that Twitter pays for the short code – which makes it is so darn easy to get your data up online. Once there, it is easy to manipulate through RSS feeds. But what can it be used for?!

Fun, mostly.

Although I’ve been discovering that Twitter can actually help tremendously in creating applications for several projects I’m working on, for which I am looking for simple ways for users to input data from their mobile phones. The ideal situation would be a J2ME application running on their S60 phones along with a data-plan. But those conditions are quite hard to find (even I can’t afford a data-plan…), especially when dealing with mainstream, working class families, on which we’ll be testing the applications. That’s precisely why Twitter is ideal – all the user needs to do is have an account setup and validated. Then sending updates from the phone is an extremely easy task. When I was working on my thesis project, I saw Twitter as an inspirational service – one that allows anyone to place content online from anyplace (within the US) for free. They absorb the cost of having a short-code (god knows how they’re going to make money) – which allows for people to easily send in their text. This type of service would be extremely useful in developing and third world countries, where I’m envisioning a multitude of mobile-based services using local SMS messages.

On the other had, Flickr (which I love) also has a certain aspect of “following” someone’s photostream. You broadcast your photos for anyone to see, but the main difference is that Flickr has an added value: kickass organization and storage tools for its user’s digital images. I use flickr mostly for my own needs, and secondly for socializing.

I don’t want it to be so easy. But I want it free… Guess I’ll need to develop something

[tags] twitter, facebook, sympathy group, gladwell, flickr, social networks, intimacy [/tags]

]]> http://giladlotan.com/blog/2008/01/on-transactive-memory-and-why-i-disslike-twitter/feed/ 1