Title Details

DOUGLAS W. HUBBARD is the inventor of Applied Information Economics (AIE), a measurement methodology that has been used in IT portfolios, entertainment media, military logistics, R&D portfolios, and many more areas where big decisions are based on factors that seem difficult or impossible to measure. He is an internationally recognized expert in metrics, decision analysis, and risk management, and is a popular speaker at numerous conferences. He has written articles for InformationWeek, CIO Enterprise, Architecture Boston, Analytics and OR/MS Today and is also the author of The Failure of Risk Management: Why It's Broken and How to Fix It.

Preface xi

Acknowledgments xv

SECTION IMEASUREMENT: THE SOLUTION EXISTS 1

CHAPTER 1Intangibles and the Challenge 3

Yes, I Mean Anything 5

The Proposal 6

CHAPTER 2An Intuitive Measurement Habit: Eratosthenes, Enrico, and Emily 9

How an Ancient Greek Measured the Size of Earth 10

Estimating: Be Like Fermi 11

Experiments: Not Just for Adults 13

Notes on What to Learn from Eratosthenes, Enrico, and Emily 18

CHAPTER 3The Illusion of Intangibles: Why Immeasurables Aren't 21

The Concept of Measurement 22

The Object of Measurement 26

The Methods of Measurement 28

Economic Objections to Measurement 35

The Broader Objection to the Usefulness of "Statistics" 37

Ethical Objections to Measurement 39

Toward a Universal Approach to Measurement 41

SECTION II BEFORE YOU MEASURE 45

CHAPTER 4 Clarifying the Measurement Problem 47

Getting the Language Right: What "Uncertainty" and "Risk" Really Mean 49

Examples of Clarification: Lessons for Business from, of All Places, Government 51

CHAPTER 5Calibrated Estimates: How Much Do You Know Now? 57

Calibration Exercise 59

Further Improvements on Calibration 64

Conceptual Obstacles to Calibration 65

The Effects of Calibration 71

CHAPTER 6 Measuring Risk through Modeling 79

How Not to Measure Risk 79

Real Risk Analysis: The Monte Carlo 81

An Example of the Monte Carlo Method and Risk 82

Tools and Other Resources for Monte Carlo Simulations 91

The Risk Paradox and the Need for Better Risk Analysis 93

CHAPTER 7Measuring the Value of Information 99

The Chance of Being Wrong and the Cost of Being Wrong: Expected Opportunity Loss 100

The Value of Information for Ranges 103

The Imperfect World: The Value of Partial Uncertainty Reduction 107

The Epiphany Equation: How the Value of Information Changes Everything 110

Summarizing Uncertainty, Risk, and Information Value: The First Measurements 114

SECTION III MEASUREMENT METHODS 117

CHAPTER 8 The Transition: From What to Measure to How to Measure 119

Tools of Observation: Introduction to the Instrument of Measurement 120

Decomposition 124

Secondary Research: Assuming You Weren't the First to Measure It 127

The Basic Methods of Observation: If One Doesn't Work, Try the Next 128

Measure Just Enough 131

Consider the Error 132

Choose and Design the Instrument 136

CHAPTER 9 Sampling Reality: How Observing Some Things Tells Us about All Things 139

Building an Intuition for Random Sampling: The Jelly Bean Example 141

A Little about Little Samples: A Beer Brewer's Approach 142

Statistical Significance: A Matter of Degree 145

When Outliers Matter Most 148

The Easiest Sample Statistics Ever 150

A Biased Sample of Sampling Methods 153

Measure to the Threshold 162

Experiment 165

Seeing Relationships in the Data: An Introduction to Regression Modeling 169

One Thing We Haven't Discussed—and Why 174

CHAPTER 10 Bayes: Adding to What You Know Now 177

Simple Bayesian Statistics 178

Using Your Natural Bayesian Instinct 181

Heterogeneous Benchmarking: A "Brand Damage" Application 187

Bayesian Inversion for Ranges: An Overview 190

Bayesian Inversion for Ranges: The Details 193

The Lessons of Bayes 196

SECTION IV BEYOND THE BASICS 201

CHAPTER 11 Preference and Attitudes: The Softer Side of Measurement 203

Observing Opinions, Values, and the Pursuit of Happiness 203

A Willingness to Pay: Measuring Value via Trade-offs 207

Putting It All on the Line: Quantifying Risk Tolerance 211

Quantifying Subjective Trade-offs: Dealing with Multiple Conflicting Preferences 214

Keeping the Big Picture in Mind: Profit Maximization versus Purely Subjective Trade-offs 218

CHAPTER 12 The Ultimate Measurement Instrument: Human Judges 221

Homo absurdus: The Weird Reasons behind Our Decisions 222

Getting Organized: A Performance Evaluation Example 227

Surprisingly Simple Linear Models 228

How to Standardize Any Evaluation: Rasch Models 230

Removing Human Inconsistency: The Lens Model 234

Panacea or Placebo?: Questionable Methods of Measurement 238

Comparing the Methods 246

CHAPTER 13 New Measurement Instruments for Management 251

The Twenty-First-Century Tracker: Keeping Tabs with Technology 251

Measuring the World: The Internet as an Instrument 254

Prediction Markets: A Dynamic Aggregation of Opinions 257

CHAPTER 14 A Universal Measurement Method: Applied Information Economics 265

Bringing the Pieces Together 266

Case: The Value of the System that Monitors Your Drinking Water 270

Case: Forecasting Fuel for the Marine Corps 275

Ideas for Getting Started: A Few Final Examples 281

Summarizing the Philosophy 287

APPENDIX Calibration Tests (and Their Answers) 289

Index 299

"How to Measure Anything was already my favorite book (just ahead of Hubbard's second book, The Failure of Risk Management) and one I actively promote to my students and colleagues. But the Second Edition, improving on the already exquisite first edition, is and achievement of its own. As a physicist and economist, I applied these techniques in several fields for several years. For the first time, somebody wrote together all these concerns on one canvas that is at the same time accessible to a broad audience and applicable by specialists. This book is a must for students and experts in the field of analysis (in general) and decision-making."
—Dr. Johan Braet, University of Antwerp. Faculty of Applied Economics, Risk Management and Innovation

"Now, performance measures can be defined for even the most difficult problems. Doug Hubbard's book is a marvelous tutorial on how to define sound metrics to justify and manage complex programs. It is a must read for anyone concerned about mitigating the risks involved with Capital Planning, Investment Decisions and Program Management."
—Jim Flyzik, former Government CIO, White House Technology Advisor and CIO Magazine Hall of Fame Inductee

Praise from How to Measure Anything, First Edition

"I love this book. Douglas Hubbard helps us create a path to know the answer to almost any question, in business, in science or in life...Hubbard helps us by showing us that when we seek metrics to solve problems, we are really trying to know something 'better than we know it now,' to put something into context, to find insight to help us get our jobs done, to be more successful, to discover things, or to build things. How to Measure Anything provides just the tools most of us need to measure anything better, to gain that insight, to make progress, and to succeed."
—Peter Tippett, Ph.D., M.D., Chief Technology Officer at CyberTrust and inventor of the first antivirus software

"Interestingly written and full of case studies and rich examples, Hubbard's book is a valuable resource for those who routinely make decisions involving uncertainty. This book is readable and quite entertaining, and even those who consider themselves averse to statistics may find it highly approachable."
—Strategic Finance

"Hubbard has made a career of finding ways to measure things that other folks thought were immeasurable. Quality? The value of telecommuting? The risk of IT project failure? the benefits of greater IT security? Public image? He says it can be done—and without breaking the bank.... If you'd like to fare better in the project-approval wars, take a look at this book."
—ComputerWorld, August 2007

"I use this book as a primary reference for my measurement class at MIT. The students love it because it provides practical advice that can be applied to a variety of scenarios; from aerospace & defense, healthcare, politics, etc."
—Ricardo Valerdi, PhD, Lecturer, MIT

"This book is remarkable in its range of measurement applications and its clarity of style. A must-read for every professional who has ever exclaimed, 'Sure, that concept is important, but can we measure it?'
—Dr. Jack Stenner, Cofounder and CEO of MetaMetrics, Inc.