Statistics for HCI. Making Sense of Quantitative Data

✍ Scribed by Alan Dix

• Publisher: Morgan & Claypool
• Year: 2020
• Tongue: English
• Leaves: 160
• Series: Synthesis Lectures on Human-Centered Informatics
• Category: Library

✦ Table of Contents

Preface
Introduction
Why are probability & statistics so hard
In two minds
Maths and more
Do you need stats at all?
From the real world to measurement & back again
The real' world There and back again Noise and randomness Why are you doing it Empirical research Software development Parallels What's next --- Randomness & Distributions Unexpected Wildness of Random Experiments in randomness Rainfall in Gheisra Two-horse races Lessons Quick (and dirty!) tip Case 1 – small proportions Case 2 – large majority Case 3 – middling Why does this work? More important than the math … Probability can be hard – from goats to DNA The Monty Hall Problem Tip: make the numbers extreme DNA evidence Properties of Randomness Bias and variability Bias Bias vs. variability Independence and non-independence Independence of measurements Independence of factor effects Independence of sample composition Play! Virtual two-horse races More (virtual) coin tossing Fair and biased coins No longer independent Characterising the Random through Probability Distributions Types of probability distribution Continuous or discrete? Finite or unbounded UK income distribution – a long tail One tail or two? Normal or not? Approximations The central limit theorem – (nearly) everything is Normal Non-Normal – what can go wrong? Power law distributions Parametric and Nonparametric --- If not p then what Probing the Unknown Recall … the job of statistics Conditional probability Likelihood Statistical reasoning Types of statistics Traditional Statistics Hypothesis testing The significance level – 5 percent and all that But what does it mean? Non-significant results In summary—significance Confidence intervals The interval Important as well as significant? Don't forget … Bayesian Methods Detecting the Martian invasion Quantifying prior belief Bayes for intelligent interfaces Bayes as a statistical method How do you get the prior? Handling multiple evidence Internecine warfare Common Issues Cherry picking Multiple tests Multiple statistics Outliers Post-hoc hypothesis The file drawer effect Inter-related factors Non-independently controllable factors Correlated features Everything is random The same or worse Everything is unlikely Numeric data More complexor worse'
Post-hoc corrections
Simulation and empirical methods
What you can say—phenomena and statisticians
Differences & Distinctions
Philosophical differences
What do we know about the world?
Not so different
So which is it?
The statistical crisis
Alternative statistics
On balance (my advice)
For both
Endnote
--- Design & Interpretation
Too few Participants
If there is something there, make sure you find it
The noise–effect–number triangle
General strategies
Subjects
More subjects or trials (increase number)
Within-subjects/within-groups studies (reduce noise)
Matched users (reduce noise)
Targeted user group (increase effect)
Tasks
Distractor tasks (increase effect)
Targeted tasks (increase effect)
Demonic interventions! (increase effect)
Restricted tasks (reduce noise)
Making Sense of Results
Look at the data
Fitts' Law—jumping to the numbers
But I did a regression …
Visualise carefully
Choice of baseline
Choice of basepoint
What have you really shown?
Think about the conditions
Individual or the population
System vs. properties
What went wrong?
Diversity: individual and task
Don't just look at the average
Tasks too
Mechanism
Quantitative and statistical meet qualitative and theoretical
Generalisation
Example: mobile font size
Building for the future
Repeatability and replication
Meta-analysis and open scholarship
Future of Statistics in HCI
Positive changes
Worrying trends
Big data and machine learning
Last words
Biblio
Index