The Big Deal on Big Data (Part 1)

by

In today’s world, making effective decisions depends on having good information at your fingertips. But as our ability to collect and analyze vasts amount of this information has grown over the past decate, our capability to effectively use this information hasn’t sufficiently matured. It’s very likely that the big investments in the collection and storage of this data isn’t paying off in better decision making.

Yet, while we struggle with that gap today, the pace of data continues to accelerate. There are now more than 2 billion users of the Internet[i] accessing and generating vast amounts of data. According to Cisco’s most recent Visual Networking Index[ii], Internet traffic increased eightfold over the last five years and will increase another fourfold over the next five. They estimate that by 2015, annual Internet traffic will approach one zetabyte. That’s a staggering amount of data. The gap will continue to grow.

To put that into perspective, let’s start smaller with a petabyte. A petabyte is 1015 bytes or 1 million gigabytes – capable of storing about 350 million MP3 songs[iii]. Using Gracenotes[iv] as an estimate on the total number of songs available (around 97 million) and the release of about 50 albums (or 500 songs) per week, it would take almost a thousand more years to have a petabyte of professionally recorded music. And a zetabyte is one million petabytes!

Figure: How Big is Each Byte

NameNumber of BytesNumber of SongsAll of Wikipedia[v]
Megabyte

1,000,000

< 1< 1
Gigabyte

1,000,000,000

350< 1
Terabyte

1,000,000,000,000

350 thousandOne tenth
Petabyte

1,000,000,000,000,000

350 million100 copies
Exabyte

1,000,000,000,000,000,000

350 billion100,000 copies
Zetabyte

1,000,000,000,000,000,000,000

350 trillion100,000,000 copies

 

The reason why this is so problematic to on-line marketers is that it continues to underscore the one-to-one marketing “data treadmill” – no matter how much data you collect about a single customer or potential customer, there’s always more to collect as their “digital exhaust”[vi] continues to expand in size and scope. While some of today’s largest marketing databases range into the terabytes of data, future database will need to expand significantly. But the emphasis will continue to be applying “big judgment” to “big data”; collecting more data and throwing it at existing processes is just throwing gasoline at the fire.

 


[i] http://www.internetworldstats.com/stats.htm

[ii] http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/VNI_Hyperconnectivity_WP.html

[iii] Assuming about 2.8 megabytes per recorded song.

[iv] http://www.gracenote.com/

[v] http://en.wikipedia.org/wiki/Wikipedia:Database_download. Using 10 terabytes to make the math a big more straightforward. The size does not include images, just the text.

[vi] http://en.wikipedia.org/wiki/Digital_exhaust#cite_note-digital_exhaust_1-0

READ MORE

Fake Case Study: Jack of all trades vs. Master of One

Fake Case Study: Jack of all trades vs. Master of One

  Listen to any earnings call or executive presentation and you will likely hear the terms “top line” and “bottom line.” These are words used to describe a business’s performance. According to Investopedia, the words are defined as follows: Top line refers to the...

read more
Your Personality Is Showing

Your Personality Is Showing

There I was, minding my own business one evening, digging into my organization's SEO performance (as one does), when I came across something interesting. Search terms related to "MBTI" — or the Myers-Briggs Type Indicator, developed by Katherine Cook Briggs and Isabel...

read more
Lessons From a Change Manager Who Hates Change

Lessons From a Change Manager Who Hates Change

Hello. My name is Monique, and I’m a change manager who hates change.   After years of receiving “consulting therapy” from various mentors, I am now able to say these words out loud and proudly. But for a long time, it felt more like an admission of guilt. I mean, who...

read more
Creativity as a Cure

Creativity as a Cure

The topic of creative solutioning has been front and center these days as we talk more and more about organizational adaptability in the face of dynamic and uncertain times. For example, I recently read about a project that got me thinking about specific priorities...

read more
Thought Ensemble, a Pariveda Company — Why Now?

Thought Ensemble, a Pariveda Company — Why Now?

Big news over here as we close out the year - we have been acquired by Pariveda, a 750-person consulting firm in 12 markets across North America! We are now “Thought Ensemble, a Pariveda Company” and I’ll be serving as the Managing Vice President continuing to lead...

read more
Thought Ensemble Joins Pariveda Solutions!

Thought Ensemble Joins Pariveda Solutions!

Dallas, December 9, 2021 /PRNewswire/ -- Pariveda, a leader specializing in solving complex technology and business problems, announces the acquisition of Thought Ensemble. With the addition of Thought Ensemble, Pariveda now provides holistic business strategy,...

read more
Thoughts on Colorado’s Equal Pay for Equal Work Act

Thoughts on Colorado’s Equal Pay for Equal Work Act

It was about a year ago that we first started hearing about Colorado’s Equal Pay for Equal Work Act (SB19-085) and I knew it was going to be national news. We’d just gotten past the “Rocky Mountain High” jokes, and our lovely state was trying to break new ground...

read more
Disruption Is the New Normal

Disruption Is the New Normal

By nature, disruptors are not popular. “First they ignore you, then they laugh at you, then they fight you, then you win, then they copy you.” We have all heard some version of this quote, and we have all seen it play out in real life. We've seen it with building...

read more
What Would You Say You Do Here?

What Would You Say You Do Here?

“I deal with the … customers so the engineers don't have to! I have people skills!” That famous Office Space quote from Tom Smykowski cracks me up every single time. I know Toms. I’ve been Tom. Change the quote to say, “IT Team” instead of “engineers,” and there’s a...

read more