Taming the Big Data Tidal Wave: Finding Opportunities in Huge Data Streams with Advanced Analytics
Buy Rights Online Buy Rights

Rights Contact Login For More Details

More About This Title Taming the Big Data Tidal Wave: Finding Opportunities in Huge Data Streams with Advanced Analytics


You receive an e-mail. It contains an offer for a complete personal computer system. It seems like the retailer read your mind since you were exploring computers on their web site just a few hours prior….

As you drive to the store to buy the computer bundle, you get an offer for a discounted coffee from the coffee shop you are getting ready to drive past. It says that since you’re in the area, you can get 10% off if you stop by in the next 20 minutes….

As you drink your coffee, you receive an apology from the manufacturer of a product that you complained about yesterday on your Facebook page, as well as on the company’s web site….

Finally, once you get back home, you receive notice of a special armor upgrade available for purchase in your favorite online video game.  It is just what is needed to get past some spots you’ve been struggling with….

Sound crazy? Are these things that can only happen in the distant future? No. All of these scenarios are possible today! Big data. Advanced analytics. Big data analytics. It seems you can’t escape such terms today. Everywhere you turn people are discussing, writing about, and promoting big data and advanced analytics. Well, you can now add this book to the discussion.

What is real and what is hype? Such attention can lead one to the suspicion that perhaps the analysis of big data is something that is more hype than substance. While there has been a lot of hype over the past few years, the reality is that we are in a transformative era in terms of analytic capabilities and the leveraging of massive amounts of data. If you take the time to cut through the sometimes-over-zealous hype present in the media, you’ll find something very real and very powerful underneath it. With big data, the hype is driven by genuine excitement and anticipation of the business and consumer benefits that analyzing it will yield over time.

Big data is the next wave of new data sources that will drive the next wave of analytic innovation in business, government, and academia. These innovations have the potential to radically change how organizations view their business. The analysis that big data enables will lead to decisions that are more informed and, in some cases, different from what they are today. It will yield insights that many can only dream about today. As you’ll see, there are many consistencies with the requirements to tame big data and what has always been needed to tame new data sources. However, the additional scale of big data necessitates utilizing the newest tools, technologies, methods, and processes. The old way of approaching analysis just won’t work. It is time to evolve the world of advanced analytics to the next level. That’s what this book is about.

Taming the Big Data Tidal Wave isn’t just the title of this book, but rather an activity that will determine which businesses win and which lose in the next decade. By preparing and taking the initiative, organizations can ride the big data tidal wave to success rather than being pummeled underneath the crushing surf. What do you need to know and how do you prepare in order to start taming big data and generating exciting new analytics from it? Sit back, get comfortable, and prepare to find out!


BILL FRANKS is the Chief Analytics Officer for Teradata, where he provides insight on trends in the analytics and big data space and helps organizations implement their analytics effectively. In addition, Bill is a faculty member of the International Institute for Analytics and is an active speaker and blogger. His consulting work has spanned many industries for companies ranging from Fortune 100 companies to small non-profits.


Foreword xiii

Preface xvii

Acknowledgments xxv


Chapter 1 What Is Big Data and Why Does It Matter?  3

What Is Big Data? 4

Is the “Big” Part or the “Data” Part More Important? 5

How Is Big Data Different? 7

How Is Big Data More of the Same? 9

Risks of Big Data 10

Why You Need to Tame Big Data 12

The Structure of Big Data 14

Exploring Big Data 16

Most Big Data Doesn’t Matter 17

Filtering Big Data Effectively 20

Mixing Big Data with Traditional Data 21

The Need for Standards 22

Today’s Big Data Is Not Tomorrow’s Big Data 24

Wrap-Up 26

Notes 27

Chapter 2 Web Data: The Original Big Data  29

Web Data Overview 30

What Web Data Reveals 36

Web Data in Action 42

Wrap-Up 50

Note 51

Chapter 3 A Cross-Section of Big Data Sources and the Value They Hold  53

Auto Insurance: The Value of Telematics Data 54

Multiple Industries: The Value of Text Data 57

Multiple Industries: The Value of Time and Location Data 60

Retail and Manufacturing: The Value of Radio Frequency Identification Data 64

Utilities: The Value of Smart-Grid Data 68

Gaming: The Value of Casino Chip Tracking Data 71

Industrial Engines and Equipment: The Value of Sensor Data 73

Video Games: The Value of Telemetry Data 76

Telecommunications and Other Industries: The Value of Social Network Data 78

Wrap-Up 82


Chapter 4 The Evolution of Analytic Scalability  87

A History of Scalability 88

The Convergence of the Analytic and Data Environments 90

Massively Parallel Processing Systems 93

Cloud Computing 102

Grid Computing 109

MapReduce 110

It Isn’t an Either/Or Choice! 117

Wrap-Up 118

Notes 119

Chapter 5 The Evolution of Analytic Processes  121

The Analytic Sandbox 122

What Is an Analytic Data Set? 133

Enterprise Analytic Data Sets 137

Embedded Scoring 145

Wrap-Up 151

Chapter 6 The Evolution of Analytic Tools and Methods 153

The Evolution of Analytic Methods 154

The Evolution of Analytic Tools 163

Wrap-Up 175

Notes 176


Chapter 7 What Makes a Great Analysis?  179

Analysis versus Reporting 179

Analysis: Make It G.R.E.A.T.! 184

Core Analytics versus Advanced Analytics 186

Listen to Your Analysis 188

Framing the Problem Correctly 189

Statistical Signifi cance versus Business Importance 191

Samples versus Populations 195

Making Inferences versus Computing Statistics 198

Wrap-Up 200

Chapter 8 What Makes a Great Analytic Professional?  201

Who Is the Analytic Professional? 202

The Common Misconceptions about Analytic Professionals 203

Every Great Analytic Professional Is an Exception 204

The Often Underrated Traits of a Great Analytic Professional 208

Is Analytics Certifi cation Needed, or Is It Noise? 222

Wrap-Up 224

Chapter 9 What Makes a Great Analytics Team?  227

All Industries Are Not Created Equal 228

Just Get Started! 230

There’s a Talent Crunch out There 231

Team Structures 232

Keeping a Great Team’s Skills Up 237

Who Should Be Doing Advanced Analytics? 241

Why Can’t IT and Analytic Professionals Get Along? 245

Wrap-Up 247

Notes 248


Chapter 10 Enabling Analytic Innovation  251

Businesses Need More Innovation 252

Traditional Approaches Hamper Innovation 253

Defi ning Analytic Innovation 255

Iterative Approaches to Analytic Innovation 256

Consider a Change in Perspective 257

Are You Ready for an Analytic Innovation Center? 259

Wrap-Up 269

Note 270

Chapter 11 Creating a Culture of Innovation and Discovery 271

Setting the Stage 272

Overview of the Key Principles 274

Wrap-Up 290

Notes 291

Conclusion: Think Bigger! 293

About the Author 295

Index 297