GambleAware has today published commissioned research, authored by Professor Patrick Sturgis and Professor Jouni Kuha of the London School of Economics, which investigates how methodological differences between surveys affect the accuracy of estimates of gambling harms. The research was commissioned following a 2019 YouGov study which found substantially higher rates of gambling harms across Great Britain than had previously been reported by the 2016 and 2018 Health Surveys for England.

The research was commissioned to identify the best way to determine gambling participation and prevalence of gambling harms in Great Britain and to develop a better understanding of how methodological factors might account for the differences between the results of the YouGov study and the Health Survey for England’s results. The surveys reviewed in the report produced widely varying estimates of ‘problem gambling’[1] in Great Britain, indicated by a PGSI score[2] of 8+, ranging from 0.7% to 2.4% of adults.

The research reviewed eight different surveys into gambling participation and prevalence of gambling harms to identify differences in results and what causes them. The key finding is that surveys using predominantly, or exclusively online self-completion responses produce consistently higher estimates of gambling harm compared to surveys which use paper self-completion techniques as part of a face-to-face interview.

The primary cause of this discrepancy was found to be selection bias in online surveys.  Selection bias in this instance refers to the fact that online surveys skew towards people who are comfortable using online technologies and who use the internet regularly. These people are also more likely to be online and frequent gamblers, meaning online surveys tend to over-estimate gambling harm.

Given these findings, the researchers shared the following recommendations for future prevalence surveys:

1. Given the high and rising cost of in person surveys, measurement of gambling prevalence and harm should move to online surveying.
2. The move to online interviewing should be combined with a programme of methodological testing and development to mitigate selection bias.
3. In person surveying should not be ceased completely; probability sampling and face-to-face interviewing should be used to provide periodic benchmarks.

GambleAware commissioned this study to better understand the true demand for treatment and support for gambling harms across Great Britain and will use the findings of this study to inform and direct the future Annual Great Britain Treatment and Support surveys. Data from the surveys will continue to be used to update GambleAware’s interactive maps, which show in visual format the prevalence of gambling participation and harms at local authority and ward level across Great Britain.

Professor Patrick Sturgis, Department of Methodology at the London School of Economics, said: “Our research has found that online surveys tend to systematically overestimate the prevalence of gambling harm compared to face-to-face interview surveys. However, given the very high and rising cost of in person surveying, and the limits this places on sample size and the frequency of surveys, we recommend a shift to predominantly online data collection in future, supplemented by periodic in person benchmarks.”

Alison Clare, Research, Information and Knowledge Director at GambleAware, said:  “We want our prevention, treatment, and support commissioning to be informed by the best available evidence, and having survey data we can be confident in, within the constraints of data collection in an increasingly online world, is key. GambleAware’s annual GB Treatment & Support survey is an important tool in building a picture of the stated demand for gambling harms support and treatment, and of the services, capacity and capability needed across Great Britain to meet that demand.