Title Details

Matthew Leitch (Epsom, UK) is an author on a mission to make risk control easier, more natural, and much more valuable. His insightful, readable books are at the leading edge of thinking and practice in internal control and risk management. He frequently carries out original research on topical questions, such as how our use of words affects the way we think about uncertainty, and what expertise auditors need. He is a qualified chartered accountant and holds a BSc in psychology from University College London. He is author of Intelligent Internal Control and Risk Management, and runs the website, www.internalcontrolsdesign.co.uk. He speaks at numerous risk and audit conferences for organizations including the IIA and IIR.

Start here 1

Good choice! 1

This book 2

How this book works 3

The myth of mathematical clarity 5

The myths of quantification 7

The auditor’s mission 8

Auditing simple risk assessments 11

1 Probabilities 12

2 Probabilistic forecaster 13

3 Calibration (also known as reliability) 13

4 Resolution 14

5 Proper score function 15

6 Audit point: Judging probabilities 17

7 Probability interpretations 17

8 Degree of belief 18

9 Situation (also known as an experiment) 19

10 Long run relative frequency 20

11 Degree of belief about long run relative frequency 21

12 Degree of belief about an outcome 22

13 Audit point: Mismatched interpretations of probability 24

14 Audit point: Ignoring uncertainty about probabilities 25

15 Audit point: Not using data to illuminate probabilities 25

16 Outcome space (also known as sample space, or possibility space) 26

17 Audit point: Unspecified situations 27

18 Outcomes represented without numbers 28

19 Outcomes represented with numbers 29

20 Random variable 29

21 Event 30

22 Audit point: Events with unspecified boundaries 31

23 Audit point: Missing ranges 32

24 Audit point: Top 10 risk reporting 32

25 Probability of an outcome 33

26 Probability of an event 34

27 Probability measure (also known as probability distribution, probability function, or even probability distribution function) 34

28 Conditional probabilities 36

29 Discrete random variables 37

30 Continuous random variables 38

31 Mixed random variables (also known as mixed discrete-continuous random variables) 39

32 Audit point: Ignoring mixed random variables 40

33 Cumulative probability distribution function 41

34 Audit point: Ignoring impact spread 43

35 Audit point: Confusing money and utility 44

36 Probability mass function 44

37 Probability density function 45

38 Sharpness 47

39 Risk 49

40 Mean value of a probability distribution (also known as the expected value) 50

41 Audit point: Excessive focus on expected values 51

42 Audit point: Misunderstanding ‘expected’ 51

43 Audit point: Avoiding impossible provisions 52

44 Audit point: Probability impact matrix numbers 53

45 Variance 54

46 Standard deviation 55

47 Semi-variance 55

48 Downside probability 55

49 Lower partial moment 56

50 Value at risk (VaR) 56

51 Audit point: Probability times impact 58

Some types of probability distribution 61

52 Discrete uniform distribution 62

53 Zipf distribution 62

54 Audit point: Benford’s law 64

55 Non-parametric distributions 65

56 Analytical expression 65

57 Closed form (also known as a closed formula or explicit formula) 66

58 Categorical distribution 67

59 Bernoulli distribution 67

60 Binomial distribution 68

61 Poisson distribution 69

62 Multinomial distribution 70

63 Continuous uniform distribution 70

64 Pareto distribution and power law distribution 71

65 Triangular distribution 73

66 Normal distribution (also known as the Gaussian distribution) 74

67 Audit point: Normality tests 77

68 Non-parametric continuous distributions 78

69 Audit point: Multi-modal distributions 78

70 Lognormal distribution 79

71 Audit point: Thin tails 80

72 Joint distribution 80

73 Joint normal distribution 81

74 Beta distribution 82

Auditing the design of business prediction models 83

75 Process (also known as a system) 84

76 Population 84

77 Mathematical model 85

78 Audit point: Mixing models and registers 86

79 Probabilistic models (also known as stochastic models or statistical models) 86

80 Model structure 88

81 Audit point: Lost assumptions 89

82 Prediction formulae 89

83 Simulations 90

84 Optimization 90

85 Model inputs 90

86 Prediction formula structure 91

87 Numerical equation solving 93

88 Prediction algorithm 94

89 Prediction errors 94

90 Model uncertainty 94

91 Audit point: Ignoring model uncertainty 95

92 Measurement uncertainty 96

93 Audit point: Ignoring measurement uncertainty 96

94 Audit point: Best guess forecasts 97

95 Prediction intervals 97

96 Propagating uncertainty 98

97 Audit point: The flaw of averages 99

98 Random 100

99 Theoretically random 101

100 Real life random 102

101 Audit point: Fooled by randomness (1) 102

102 Audit point: Fooled by randomness (2) 104

103 Pseudo random number generation 104

104 Monte Carlo simulation 105

105 Audit point: Ignoring real options 109

106 Tornado diagram 109

107 Audit point: Guessing impact 111

108 Conditional dependence and independence 112

109 Correlation (also known as linear correlation) 113

110 Copulas 113

111 Resampling 114

112 Causal modelling 114

113 Latin hypercube 114

114 Regression 115

115 Dynamic models 116

116 Moving average 116

Auditing model fitting and validation 117

117 Exhaustive, mutually exclusive hypotheses 118

118 Probabilities applied to alternative hypotheses 119

119 Combining evidence 120

120 Prior probabilities 120

121 Posterior probabilities 120

122 Bayes’s theorem 121

123 Model fitting 123

124 Hyperparameters 126

125 Conjugate distributions 126

126 Bayesian model averaging 128

127 Audit point: Best versus true explanation 128

128 Hypothesis testing 129

129 Audit point: Hypothesis testing in business 130

130 Maximum a posteriori estimation (MAP) 131

131 Mean a posteriori estimation 131

132 Median a posteriori estimation 132

133 Maximum likelihood estimation (MLE) 132

134 Audit point: Best estimates of parameters 135

135 Estimators 135

136 Sampling distribution 138

137 Least squares fitting 138

138 Robust estimators 140

139 Over-fitting 140

140 Data mining 141

141 Audit point: Searching for ‘significance’ 142

142 Exploratory data analysis 143

143 Confirmatory data analysis 143

144 Interpolation and extrapolation 143

145 Audit Point: Silly extrapolation 144

146 Cross validation 145

147 R2 (the coefficient of determination) 145

148 Audit point: Happy history 147

149 Audit point: Spurious regression results 147

150 Information graphics 148

151 Audit point: Definition of measurements 148

152 Causation 149

Auditing and samples 151

153 Sample 152

154 Audit point: Mixed populations 152

155 Accessible population 152

156 Sampling frame 153

157 Sampling method 153

158 Probability sample (also known as a random sample) 154

159 Equal probability sampling (also known as simple random sampling) 155

160 Stratified sampling 155

161 Systematic sampling 156

162 Probability proportional to size sampling 156

163 Cluster sampling 156

164 Sequential sampling 157

165 Audit point: Prejudging sample sizes 158

166 Dropouts 159

167 Audit point: Small populations 160

Auditing in the world of high finance 163

168 Extreme values 164

169 Stress testing 165

170 Portfolio models 166

171 Historical simulation 168

172 Heteroskedasticity 169

173 RiskMetrics variance model 169

174 Parametric portfolio model 170

175 Back-testing 170

176 Audit point: Risk and reward 171

177 Portfolio effect 172

178 Hedge 172

179 Black–Scholes 173

180 The Greeks 175

181 Loss distributions 176

182 Audit point: Operational loss data 178

183 Generalized linear models 179

Congratulations 181

Useful websites 183

Index 185