Rights Contact Login For More Details
- Wiley
More About This Title Value for Money: How to Show the Value for Money for All Types of Projects and Programs in Governments, Non-Governmental Organizations, Nonprofits, an
English
Written by two of the world’s most well-known ROI (Return on Investment) gurus, this guide is indispensable for anyone involved in showing the value of money for projects and programs in governments, non-governmental organizations, nonprofits, and businesses. These range from human capital programs to marketing initiatives, technology implementations, systems integrations, quality and lean processes, public health initiatives, procurement procedures, public relations events, risk management policies, economic development programs, corporate social responsibility projects, public policy programs, branding activities, innovation programs, customer satisfaction projects, and everything in between.
In a step-by-step process, the book shows how to measure the success of projects and programs, including measuring impact and ROI (Return on Investment). This book also shows how to forecast the value of the project in advance and how to collect data during and after project implementation. It addresses improvements throughout the process so that the project delivers optimum value. In addition to businesses, this book is appropriate for governments, NGOs, nonprofits, universities and healthcare organizations.
As a reference for those who are seeking ways to assign value to what they have measured, the book will clarify and resolve much of the mystery surrounding the conversion of data to monetary values. Building on a tremendous amount of experience, application, practice, and research, the book will be based on the work of many individuals and organizations, particularly those who have been reaching the ultimate levels of accountability using the ROI Methodology. Developed in an easy-to-read format and fortified with examples, tips, and checklists, this will be an indispensable guide for those who seek to understand accountability issues.English
Patricia Pulliam Phillips, PhD is an internationally recognized author, consultant, and President and CEO of ROI Institute, Inc. Phillips provides consulting services to organizations worldwide. She helps organizations build capacity in the ROI Methodology by facilitating the ROI certification process and teaching the ROI Methodology through workshops and graduate level courses. She has been an author or editor of more than a hundred books and numerous journal articles.
Jack J. Phillips, PhD is Chairman of ROI Institute and a world-renowned expert on measurement and evaluation. Former bank president Phillips provides consulting services for Fortune 500 companies and workshops for major conference providers worldwide. Phillips is also the author or editor of more than a hundred books and hundreds of articles. His work has been featured in the Wall Street Journal, Bloomberg Businessweek, Fortune, and on CNN.
Gina Paone joined the International Monetary Fund in February 2011 after nearly eleven years with the United Nations World Food Programme based in Rome, Italy and 10 years in the private sector in a marketing capacity based in Toronto, Canada. She is currently the Division Chief, Talent Acquisition and Operations. Ms. Paone holds a bachelor's degree in Business Administration from the University of Western Ontario in London, Ontario Canada and a Masters in Organizational Development from Pepperdine University in Malibu, California USA.
Cyndi Huff Gaudet, PhD, is Chair of the Dept. of Human Capital Development at The University of Southern Mississippi, a scholar-practitioner managing projects integrating Phillips ROI Methodology, and a recognized national expert helping organizations implement a systematic approach for developing human capital. Cyndi holds a Ph.D. in Human Resource Education and Workforce Development from LSU and ROI Certification.
English
Praise for Value for Money v
Foreword xxv
Preface xxvii
Value is Changing… xxvii
Need for a New Approach xxviii
The ROI Methodology: The Enhanced Logic Model xxix
We Can’t Measure Our Way to Success xxx
Flow of the Book xxxi
Acknowledgements xxxv
Authors xxxviii
1 The Value Evolution 1
The Value Shift 4
Types of Values 5
The Importance of Monetary Values 6
The “Show Me” Generation 6
The New Definition of Value 7
Why Now? 8
Program Failures 8
Budget Constraints 9
Project and Program Costs 9
Donors are More Demanding 10
Measurement at the Impact Level Is No Longer Optional 11
Process Improvements Use 12
Managers’ New Business Focus 12
Case Study 12
Globalization 13
The Growth of Project Management 14
Evidence-Based or Fact-Based Management 14
Mistrust of Institutions 15
Benchmarking Limitations 15
The Executive Appetite for Monetary Value 16
Challenges Along the Way 16
Preparation and Skills 17
Fear of ROI 17
Time Commitment 17
Power and Politics 18
Misleading Hype 18
Sustainability 19
Final Thoughts 19
2 Six Ways to Show Value for Money 21
Six Ways to Show Value for Money 25
#1 - Actual ROI Calculation 27
#2 - Focus on Impacts and Intangibles 28
#3 - Meeting Expectations 29
#4 - The Payoff of Improved Behaviors 29
#5 - Cost-Effective Approach 30
#6 - Cost of Not Implementing the Program 31
Conclusion 32
Barriers to Showing Value for Money 32
We Don’t Make Widgets 32
Because there are No Profits, ROI is Not Possible 33
ROI is Just Not Appropriate for the Public Sector 34
We Don’t Have the Resources 35
This is too Complicated 36
No One is Asking for this 37
Final Thoughts 38
3 Needed: An Enhanced Logic Model 39
A Review of Models 42
Definition 42
Logic Model Elements 44
Other Models 45
Concerns about Current Models 47
Lack of Focus on Benefit-Cost Analysis 47
Lack of Emphasis on Attribution 49
Confusion on Outcomes and Impact 49
Little Focus on the Chain of Value of a
Particular Program or Project 50
Unclear Timing of Data Collection 51
Needs Assessments versus Activities 52
Lack of Focus on Results 52
Inability to “Show the Money” 54
Not CEO- and CFO-Friendly 56
Managing Resources for Evaluation 57
Lack of Focus on Process Improvement 58
The Inability to Influence Investment 59
How Does Your Current Model Stack Up? 60
Focus of Use 61
Standards 61
Types of Data 61
Dynamic Adjustments 62
Connectivity 62
Approach 63
Conservative Nature 63
Simplicity 63
Theoretical Foundation 63
Acceptance 64
Requirements for the Value for Money: A Measurement Process 64
ROI Methodology 65
Terminology: Projects, Solutions, Participants 66
Final Thoughts 67
4 Introducing the ROI Methodology 69
Types of Data 70
Input 70
Reaction and Planned Action 72
Learning 73
Application and Implementation 73
Impact 74
Return on Investment 74
The Initial Analysis 75
Using Design Thinking to Deliver and Measure Results 77
The ROI Process Model 78
Plan the Evaluation 78
#1 - Start with Why: Align Programs with the
Business 78
#2 - Make it Feasible: Select the Right Solution 81
#3 - Expect Success: Design for Results 81
Collect Data 82
#4 - Make it Matter: Design for Input,
Reaction, and Learning 82
#5 - Make it Stick: Design for Application
and Impact 83
Analyze Data 83
#6 - Make it Credible: Isolate the Effects of
the Program 83
#7 - Make it Credible: Convert Data to
Monetary Value 84
#8 - Make it Credible: Identify Intangible Benefits 84
#9 - Make it Credible: Capture Costs of Projects 85
#10 - Make it Credible: Calculate the Return
on Investment 85
Optimize Results 86
#11 - Tell the Story: Communicate Results to
Key Stakeholders 86
#12 - Optimize the Results: Use Black Box
Thinking to Increase Funding 86
Operating Standards and Philosophy 87
Implementing and Sustaining the Process 87
Benefits of this Approach 88
Aligning with Business 89
Validating the Value Proposition 89
Improving Processes 89
Enhancing the Image and Building Respect 90
Improving Support 90
Justifying or Enhancing Budgets 90
Building Productive Partnerships 91
Final Thoughts 91
5 Start with Why: Align Programs with the Business 93
Impact Measures are Critical 96
The Challenge 97
Begin with the End in Mind 97
It’s a Change 97
It Requires Discipline 98
Avoid Paralysis by Analysis 98
The Alignment Model 98
Payoff Needs 99
Key Questions to Ask 102
Obvious versus Not-So-Obvious Payoffs 102
Reasons for New Programs 104
The Costs of the Problem 105
Case Study 107
The Value of Opportunity 107
To Forecast or Not to Forecast 108
Case Study 108
Business Needs 109
Determining the Opportunity 109
Identifying the Business Measure—Hard Data 110
Output 110
Quality 110
Cost 110
Time 112
Case Study 112
Defining the Business Need—Soft Data 113
Leadership 114
Work Climate/Satisfaction 114
Client Service 114
Employee Development/Advancement 114
Initiative/Innovation 114
Image/Reputation 115
Using Tangible versus Intangible—A Better Approach 115
Finding Sources of Impact Data 115
Identifying All the Measures 116
What Happens if You do Nothing? 117
Case Study 117
Final Thoughts 118
6 Make It Feasible: Select the Right Solution 119
Performance Needs 122
The Performance Dialogue 122
Examine the Data and Records 123
Initiate the Discussion 123
Case Study 123
Use Benchmarking from Similar Solutions 124
Use Evaluation as the Hook 124
Involve Others 125
Discuss Disasters in Other Places 125
Use Analysis Techniques 125
Keep it Sensible 126
Case Study 127
Learning Needs 131
Subject-Matter Experts (SMEs) 131
Job and Task Analysis 131
Observations 132
Demonstrations 132
Tests 132
Management Assessment 133
Case Study 133
Preference Needs 134
Key Issues 134
Case Study 135
Matching Solutions to Needs 135
Some Solutions are Obvious 135
Solutions Can Come in Different Sizes 136
Some Solutions Take a Long Time 136
Solutions Should be Tackled with the Highest Priority
Items First 136
The Matrix Diagram 136
Selecting Solutions for Maximum Payoff 138
Short-Term versus Long-Term Costs 138
Consider Forecasting ROI 138
Time Needed for Implementation 138
Avoid Mismatches 139
Verifying the Match 139
Tackling Multiple Solutions 140
Final Thoughts 140
7 Expect Success: Design for Results 141
The Power of Expectations 144
Case Study 144
Keep it Sensible 146
Defining the Success of Programs 147
Designing for Results at Each Level 149
Level 0, Input 149
Level 1, Reaction 150
Level 2, Learning 150
Level 3, Application 150
Level 4, Impact 150
Case Study 151
Level 5, ROI 151
Use Empathy 151
Developing Objectives at Multiple Levels 152
Case Study 152
Reaction Objectives 153
Learning Objectives 154
Application Objectives 155
Impact Objectives 157
Return on Investment (ROI) Objectives 158
Case Study 158
The Power of Objectives 160
Application/Impact Objectives Drive Programs 160
Application/Impact Objectives Enhance
Design and Development 160
Application/Impact Objectives Improve
Facilitation 160
Application/Impact Objectives Help Participants
Understand What Is Expected 161
Impact Objectives Excite Sponsors and Donors 161
Application/Impact Objectives Simplify Evaluation 161
All Levels of Objectives Inform the Stakeholders 162
Defining Roles and Responsibilities 162
Analysts 163
Designer 163
Developer 163
Program Owner 164
Facilitator 164
Participants 164
Sponsor or Donor 165
Managers of Participants/Significant Others 165
Evaluator 165
Other Stakeholders 165
Planning the Evaluation 166
Evaluation Purpose 166
Feasibility of Outcome Evaluations 167
Data Collection Plan 167
ROI Analysis Plan 167
Project Plan 171
Final Thoughts 172
8 Make It Matter: Design for Input, Reaction, and Learning 173
Communicating with Results in Mind 174
Announcements 175
Brochures 175
Case Study 175
Correspondence 176
Workbooks and Participant Guides 176
Changing the Role of Participants 177
The Necessity 177
Defining Roles 177
Documenting the Roles 178
Creating Expectations 178
Identifying Impact Measures Before Participating in
the Program 179
Case Study 179
Involving Managers or Significant Others 181
Think ROI 181
ROI Review 182
Case Study 182
Actions to Take 183
Design Input for Results 183
Target Audience 183
Case Study 184
The Need 184
Case Study 185
Timing and Duration 185
Motivation 186
Readiness 186
Conclusion 187
Design Reaction for Results 187
Topics to Measure 188
Case Study 189
Measuring Reaction 191
Case Study 192
Using Reaction Data 192
Forecasting ROI at this Level 192
Design Learning for Results 193
The Learning Style 194
Sequencing and Time 194
Activities 194
Data Collection for Input, Reaction, and Learning 194
Questionnaires and Surveys 194
Measuring with Tests 196
Measuring with Simulation 196
Timing of Data Collection 197
Early, Detailed Feedback 198
Pre-Program 198
Collecting at Periodic Intervals 198
For Long Programs with Multiple Parts 199
Final Thoughts 199
9 Make It Stick: Design for Application and Impact 201
Data Collection for Application and Impact 204
Questionnaires and Surveys 204
Improving the Response Rate for
Questionnaires and Surveys 204
Interviews 209
Focus Groups 210
Observations 210
Action Plans 211
Set Goals and Targets 212
Define the Unit of Measure 212
Place a Monetary Value on Each Improvement 214
Implement the Action Plan 214
Isolate the Effects of the Program 214
Provide a Confidence Level for Estimates 215
Collect Action Plans 215
Summarize the Data and Calculate the ROI 215
Performance Contract 217
Monitoring Business Performance Data 219
Existing Measures 219
Case Study 220
Developing New Measures 221
Selecting the Appropriate Method for Each Level 222
Type of Data 222
Participants’ Time for Data Input 222
Manager (or Significant Other) Time for Data Input 223
Cost of Method 223
Disruption of Normal Work Activities 223
Accuracy of Method 223
Utility of an Additional Method 224
Cultural Bias for Data Collection Method 224
Timing of Data Collection 224
Collecting Application Data 224
Collecting Impact Data 225
Built-In Application Tools 226
Improvement Plans and Guides 227
Application Tools/Templates 227
Job Aids 227
Case Study 228
Involving the Participants’ Manager or Significant Other 230
The Most Influential Group 230
Pre-Program Activities 231
During the Program Activities 232
Post-Program Activities 232
Reinforcement Tools 232
Case Study 233
Final Thoughts 235
10 Make It Credible: Isolate the Effects of the Program 237
The Importance of Pinpointing the Contribution 240
Reality 240
Myths 240
Preliminary Issues 242
Review Chain of Impact 242
Identify Other Factors 243
Quantitative and Research Isolation Methods 244
Case Study 244
Experimental Design 246
Trend Line Analysis 249
Mathematical Modeling 252
Calculating the Impact of Other Factors 253
Qualitative Isolation Methods 254
Participants’ Estimate of Impact 255
Steps to Measure Attribution Using Estimates 258
Manager’s Estimate of Impact 259
Customer Estimates of Program Impact 260
Internal or External Expert Estimates 260
Estimate Credibility: The Wisdom of Crowds 261
Case Study 261
Select the Method 263
Final Thoughts 264
11 Make It Credible: Convert Data to Monetary Value 265
The Importance of Monetary Value 267
Value Equals Money 267
Money Makes Impact More Impressive 268
Converting to Monetary Values is Similar to Budgeting 268
Monetary Value is Vital to Organizational Operations 269
Monetary Values Are Necessary to Understand
Problems and Cost Data 269
Key Steps in Converting Data to Money 270
Standard Monetary Values 272
Converting Output Data to Money 273
Case Study 274
Calculating the Cost of Inadequate Quality 276
Case Study 278
Converting Employee Time Savings Using Compensation 279
Case Study 280
Finding Standard Values 281
When Standard Values are Not Available 281
Using Historical Costs from Records 282
Time 282
Availability 282
Access 282
Accuracy 283
Using Input from Experts 284
Using Values from External Databases 285
Linking with Other Measures 286
Using Estimates from Participants 288
Using Estimates from the Management Team 289
Using Program Staff Estimates 289
Selecting the Technique 290
Choose a Technique Appropriate for the Type of Data 290
Move from Most Accurate to Least Accurate 290
Consider Source Availability 291
Use the Source with the Broadest Perspective on the Issue 291
Use Multiple Techniques When Feasible 291
Apply the Credibility Test 292
Consider the Short-Term/Long-Term Issue 292
Consider an Adjustment for the Time Value of Money 294
Final Thoughts 295
12 Make It Credible: Identify the Intangibles 297
Why Intangibles are Important 299
Intangibles are the Invisible Advantage 300
We are in the Intangible Economy 301
More Intangibles are Converted to Tangibles 302
Case Study 302
Intangibles Drive Programs and Investments 303
Measuring and Analyzing Intangibles 304
Measuring the Intangibles 304
Converting to Money 307
Case Study 307
Case Study 310
Identifying and Collecting Intangibles 311
Analyzing Intangibles 312
Final Thoughts 313
13 Make It Credible: Capture Costs of the Program and Calculate ROI 315
The Importance of Costs and ROI 319
Fundamental Cost Issues 320
Fully Loaded Costs 320
Case Study 321
Costs Reported without Benefits 322
Develop and Use Cost Guidelines 323
Sources of Costs 323
Prorated versus Direct Costs 324
Employee Benefits Factor 325
Specific Costs to Include 325
Initial Analysis and Assessment 326
Development of Program Solutions 326
Acquisition Costs 326
Implementation Costs 326
Case Study 327
Maintenance and Monitoring 327
Support and Overhead 328
Evaluation and Reporting 328
Cost Tabulation in Action 328
Problem and Solution 328
Program Description 329
Selection Criteria 330
Program Administration 331
The Drivers of Evaluation 331
Program Costs 332
The ROI Calculation 333
Benefit/Cost Ratio 334
ROI Formula 335
Case Study 336
Monetary Benefits 337
Misuse of ROI 338
Social Return on Investment 339
ROI Objectives 340
Whose ROI? 340
Other ROI Measures 342
Payback Period (Breakeven Analysis) 342
Discounted Cash Flow 342
Internal Rate of Return 343
Final Thoughts 343
14 Tell the Story: Communicate Results to Key Stakeholders 345
The Presentation 346
Reaction and Learning 347
Application 348
Business Impact 348
ROI 350
Intangibles 351
Conclusion and Recommendations 351
Reflection 352
The Importance of Communicating Results 352
Communication is Necessary to Make Improvements 352
Communication is Necessary to Explain the Contribution 353
Communication is a Politically Sensitive Issue 353
Different Audiences Need Different Information 353
Case Study 354
Principles of Communicating Results 355
Communication Must be Timely 355
Communications Should be Targeted to
Specific Audiences 355
Media Should be Carefully Selected 356
Communication Should be Unbiased and Modest in Tone 356
Communication Must Be Consistent 356
Make the Message Clear 356
Testimonials Must Come from Respected Individuals 357
The Audience’s Bias of the Program Will Influence the
Communication Strategy 357
The Process for Communicating Results 357
Step 1: Analyze Reason for Communication 359
Step 2: Plan for Communication 359
Step 3: Select Audience 360
Step 4: Develop Reports 363
Step 5: Select Media 363
Meetings 363
Interim and Progress Reports 366
Routine Communication Tools 366
Case Study 367
Email and Electronic Media 369
Program Brochures and Pamphlets 369
Case Studies 370
Step 6: Present Information 370
Routine Feedback on Program Progress 371
Storytelling 373
Presentation of Results to Senior Management 374
Case Study 376
Step 7: Analyze Reaction 378
Final Thoughts 379
15 Optimize Results: Use Black Box Thinking to Increase Funding 381
Process Improvement is the Key: Black Box Thinking 383
The Aviation Industry 384
The Healthcare Industry 384
Failures in Programs 386
Making Adjustments in Programs 387
The Fear of Results 387
You Can Always Make it Better 388
When Do You Discontinue the Program? 388
The Timing of Changes 390
Level 0, Input 390
Level 1, Reaction Measures 390
Level 2, Learning Measures 391
Level 3, Application Measures 391
Level 4, Impact Measures 392
Making the Adjustments 392
Increasing ROI 393
Addressing Costs 393
Addressing the Monetary Benefits 395
Timing of Assessments 395
Influencing Allocation 396
Investment versus Cost 396
Competition for Funding 398
Anxiety and Downturns Translate into Cost Reduction 399
Final Thoughts 400
16 Forecast the ROI 401
The Importance of Forecasting ROI 407
Expensive Programs 407
High Risks and Uncertainty 407
Case Study 408
Post-Program Comparison 409
Compliance 410
The Trade-Offs of Forecasting 411
Pre-Program ROI Forecasting 413
Basic Model 413
Basic Steps to Forecast ROI 414
Sources of Expert Input 417
Securing Input 418
Conversion to Money 419
Estimate Program Costs 419
Case Study 420
Forecasting with a Pilot Program 422
ROI Forecasting with Reaction Data 423
Case Study 424
Use of the Data 426
Forecasting Guidelines 427
Final Thoughts 429
17 Make It Work: Sustaining the Change to a Results-Based Process 431
Overcoming Resistance 433
Assess the Climate 434
Develop Roles and Responsibilities 434
Identifying a Champion 434
Developing the Champion 435
Establishing a Task Force 435
Assigning Responsibilities 436
Establish Goals and Plans 436
Setting Evaluation Targets 437
Developing a Plan for Implementation 438
Revise Guidelines and Procedures 438
Prepare the Team 440
Involving the Team 440
Teaching the Team 440
Initiate ROI Studies 441
Prepare the Management Team 442
Remove Obstacles 443
Dispelling Myths 443
Delivering Bad News 444
Using the Data 444
Monitor Progress 445
Final Thoughts 446
References 447
Appendix A 459
Appendix B 467
Appendix C 471
Index 473
