Discover Performance

HP Software's community for IT leaders // June 2014

Tapping the profit potential of dark data

Structured and semi-structured data lie dormant in your data warehouse—but it’s becoming increasingly easy to monetize them.

The bottom line

What: Your next analytics frontier is "dark data."
Why: A lot of actionable information is trapped in your enterprise—and it’s worth money.
How: Change your approach to your data warehouse.
More: Watch the webinar, "Monetizing your dark data" (reg. req’d).

Millions of dollars may lurk somewhere inside your organization’s data warehouses and mainframes. You’ve collected, processed, and stored it; you just haven’t monetized it. You might call it "dark data," and if you can transmute it, the return on investment could be enormous.

Dark data encompasses a variety of data types, but essentially it’s semi-structured and unstructured data that you can’t see or use effectively. If Big Data is the new oil, then dark data is shale gas. And just as drilling for gas-rich shale means diving deeper into the earth’s crust, the process of extracting dark data requires another level of identifying, offloading, and deploying if you’re going to hit pay dirt. 

"There’s a huge benefit from monetizing dark data," Steve Totman, director of strategy at Syncsort, said in a recent HP Vertica webinar. In fact, he said, there’s "billions of dollars’ worth" of benefits. 

Framing the challenge

The problem mining dark data poses is that it usually resides in your mainframe, and critical mainframe data is hidden to the business.

"Data warehouses aren’t the dynamic forces they should have been," Totman said. "Legacy technology has posed significant barriers to accessing your organization’s dark data. To take advantage of it at scale ... the infrastructure you’d need becomes unaffordable."

Utilizing mainframe data has also traditionally been a time-consuming endeavor. You’re forced to choose between speed of decisions and quality of insight—not to mention how expensive it is to process and store mainframe data. Your monthly bill is based on CPU utilization, and the typical annual cost for the "average" $10 billion organization is $15.7 million: reductions in usage translate into instant OpEx savings.

At the same time, semi-structured and unstructured information like social media data can be the key to maximizing your marketing capabilities. In the webinar, HP Vertica VP Chris Selland said that "dark data gives you a much deeper understanding of your customers" when you move it into the light.

To do so, you need three things from your data warehouse:

  • Massive scalability
  • Real-time speed
  • The ability to get to semi-structured and unstructured data

Selland discussed HP Vertica’s Flex Zone offering as a powerful approach to loading, managing, and exploring semi-structured data. After you’ve identified mainframe and heavy ELT workloads in your data warehouse, Flex Zone performs an exploratory analysis that produces analytic reports in real time. 

Revealing dark data allows you to monetize and streamline workflows and eliminate legacy waste. The savings are immediate. The earning potential could be massive. And you’ve been sitting on it all along.

For more on accessing—and monetizing—unstructured and semi-structured data, watch the free, on-demand webinar, "Monetizing your dark data" (reg. req’d), and visit for an article and video on Vertica 7’s Flex Zone.


IT leader assessment

This tool evaluates the correlation between IT attributes and business success and, based on how your answers compare with average scores, will advise you where to invest in IT.

It is based on data HP collected from 650 global companies about a range of IT characteristics (server capacities, approach to information management, security, BYOD, etc.) and how they correlate to revenue gain. This assessment will compare your answers to the average scores in that study.

There are 12 questions that will require an estimated 10 minutes of your time. You'll receive a summary of your rating upon completion.

Let's get started

Please select an answer.


Your answer:
Your score:
Average score:
Revenue leaders' score:


Please select an answer.



Your score:
Average score:
Revenue leaders' score:

Get detailed results:


Popular tags


Discover Performance Weekly

HP Software’s Paul Muller hosts a weekly video digging into the hottest IT issues. Check out the latest episodes.

Big Data as key to change management

Change is hard—and risky—in any IT organization. Learn how better analytics makes things smoother and more successful.

Enterprise 20/20

Marketing 20/20

Welcome to a new reality of split-second decisions and marketing by the numbers.

Mobility 20/20

Looking toward the era when everyone — and everything — is connected.

Introduction to Enterprise 20/20

What will a successful enterprise look like in the future?

CIO 20/20

Challenges and opportunities for the CIO of the future.

Dev Center 20/20

How will we organize development centers for the apps that will power our enterprises?

IT Operations 20/20

How can you achieve the data center of the future?

Employee 20/20

What the workforce of 2020 can expect from IT, and what IT can expect from the workforce.

Security 20/20

Preparing today for tomorrow’s threats.

Data Center 20/20

The innovation and revenue engine of the enterprise.

Read more

HP Software related

Most read articles

Discover Performance


Tweets @ HPITperformance