Social Media Data Aggregation

Posted by admin on February 5, 2012

I work for a company owned by MDC Partners. MDC owns a group of ad agencies. This week, at some corporate meetings, discussions came up about the value of social media data, and how you could possibly aggregate and monetize this data.

Being on the I.T. side, I understand some of the challenges of trying to capture this type of data, generated in Petabytes and not normalized. To start with, I would like to list some of the current options available on the commercial market.

GNIP

http://gnip.com/

Gnip (pronounced "guh nip") is the original in this arena. Gnip was founded in 2008 and is backed by several VC groups including The Foundry Group, SoftTech, and First Round Capital. Their goal is to gather social media data and present it to customers in a "consistent and reliable" architecture.

Gnip is the most advanced option if you have the money and just want to buy the data. They pull feeds from the following systems;

Clipmarks  *  Dailymotion  *  Delicious  *  Digg    Diigo  *  Facebook *  Flickr  *  Flixster  *  Fotolog  *  FriendFeed  *  Google Plus  *  Hulu  *  Identi.ca  *  iLike  *  IntenseDebate  *  MySpace  *  Newsgator    Photobucket  *  Plurk  *  SlideShare  *  SmugMug  *  StockTwits  *  StumbleUpon  *  Tumblr  *  Twitter  *  Vimeo  *  Wordpress  *  Xanga  *  Yahoo  *  YouTube

So, as you can see, it's a pretty exhaustive list of data. They generally offer three levels of data;

1. Username
2. Keyword
3. Firehose - partial levels, if available

Don't think this type of data is cheap; for example, some investigation suggests that pricing for Twitter to be something around $30,000/month for the "full firehose" which is 50% of all Twitter data. This is the highest level of data available for Twitter. There are also 10% firehose, keyword and username options that are less costly.

Datasift

There are only two companies who have firehose agreements with Twitter. GNIP is the original, and the second company is Datasift.

I'm not familiar with their offerings, or pricing. It doesn't look like they have firehose access, but rather brand, topic, or other segmented data.

FBI Seeks Social Media Aggregation Tool

The FBI has a RFI out right now looking for a social media aggregation tool;

Informatica CMO Chris Boorman on Master Data Management

Great article by the CMO of Informatica. He discusses some of the challenges of huge datasets, user identity and how to tie it all together.

http://mashable.com/2011/02/25/data-mining-social-marketing/

Closing Random Links

I read The Economist, and I remembered an article in there. So I got online and found the link to the artical titled "Untangling the social web"

Excerpt; Software: From retailing to counterterrorism, the ability to analyse social connections is proving increasingly useful

I also remembered when GNIP signed up Wordpress (the hosted Wordpress blogs), so here's a link to that article.