25 Billion Tweets

Twitter is doing a “year in review” on their blog and a stat in this post caught my attention.  That’s right, over 25 Billion tweets were sent last year.  That’s 68 million tweets each and every day.  As someone fascinated with data, I feel like a kid in a candy store… scratch that, a kid owning a chain of candy stores.  And the beauty of Twitter vs. something like Facebook is they allow and even encourage full and complete access to most every bit of that data (exception is private accounts), and the relationships contained in it – not only the tweets, but info about who made them, who replied to who, etc.

If you’re an individual, that’s not super relevant.  Your following the people you care about, you may have lists/searches setup to track things important to you, and has been said, Twitter is like a flowing river (a large and growing river apparently).  You sip a bit once in a while as the info flows by.

But as an organization/business, there’s more of an interest in what’s being said.  That’s what interests me, dealing with billions of tweets is impossible, and there seems two viable methods of dealing with it all.  One is search terms: something that’s very well known, easy to do on either Twitter itself or many other tools.  But there’s a technique I’ve been working on for some time that I want to share here.  I refer to it as “identifying influencers”. 

For a specific segment – agriculture, moms, Kansas folks, pick your interest – we want to identify those accounts that seem to have significant influence on Twitter.  Many people follow/retweet/converse with them.  Their “influential” in their community.  And than we want to track their activity.  The three groups listed above, agriculture, moms, Kansas folk are the three test cases I am currently working with.  Agriculture because it’s the industry I’m part of and love.  Mom’s because they are a key example of an important consumer demographic group.  Kansas because I wanted a test case using location and there’s a personal interest in the state I live.

But those three are just a start, over the last year as this technology has slowly came together, gardening, health, non-profits, vegans, pets, science are some other things we’ve looked at.  As you can see many of them have some connection to agriculture.  This technology is not specific to agriculture, but that’s where my passion and interest lies.

So how does it work?  Like most of these things, sharing full details would hurt my ability to get good data, but when Twitter came out with their list technology I knew it was a significant moment for this concept.  What I’m looking for is the list names that folks have put an account on.  It’s a “wisdom of the crowds” concept.  The more people that have added you to an ag list, or a mom list, the more influential you must be in that community.  That forms the basis of the algorithm.  There’s a LOT of other factors to consider, and I would never claim the results are perfect.  It involves a lot of data processing, but when finished I think it returns a solid, relevant list of targeted influencers.  Although this post isn’t about the data specifically, I’ll throw out a couple of teasers:  About 1500 ag lists were found, and that’s searching pretty intensely.  Over 4000 mom lists were found, without as intense of an effort to find every one.

From either of these two methods, we capture a large number of tweets.  The second part of what I’m doing I’ll refer to as “tracking conversations”.  It’s still pretty early in the thought process and development effort there so I don’t have a lot to talk about.  In general it’s about how we look at those tweets for key pieces of info that may be relevant.   A key difference vs. how a tool like TweetDeck or Twitter.com is that it’s not real time and we want to capture everything.  This is summary information after the fact, the next day or week for example, or possibly monitored several times an hour, but it’s designed to summarize whats been said, not show every tweet for review.

When the chores are done this winter (a slow/VERY cold process this weekend unfortunately), the basic farm work and prep for next year taken care of, this is what I’ll be working on this winter.  As always, I value feedback and input, so let me know with any questions/comments, etc.  Stay tuned for more…


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: