mscroggs.co.uk
mscroggs.co.uk
Click here to win prizes by solving the mscroggs.co.uk puzzle Advent calendar.
Click here to win prizes by solving the mscroggs.co.uk puzzle Advent calendar.

subscribe

Blog

The importance of estimation error

 2017-02-25 
Recently, I've noticed a few great examples of misleading uses of numbers in news articles.
On 15 Feb, BBC News published a breaking news article with the headline "UK unemployment falls by 7,000 to 1.6m". This fall of 7,000 sounds big; but when compared to the total of 1.6m, it is insignificant. The change could more accurately be described as a fall from 1.6m to 1.6m.
But there is a greater problem with this figure. In the original Office of National Statistics (ONS) report, the fall of 7,000 was accompanied by a 95% confidence interval of ±80,000. When calculating figures about large populations (such as unemployment levels), it is impossible to ask every person in the UK whether they are employed or not. Instead, data is gathered from a sample and this is used to estimate the total number. The 95% confidence interval gives an idea of the accuracy of this estimation: 95% of the time, the true number will lie of the confidence interval. Therefore, we can think of the 95% confidence interval as being a range in which the figure lies (although this is not true, it is a helpful way to think about it).
Compared to the size of its confidence interval (±80,000), the fall of 7,000 is almost indistinguishable from zero. This means that it cannot be said with any confidence whether the unemployment level rose or fell. This is demonstrated in the following diagram.
A fall of 7,000 ± 80,000. The orange line shows no change.
To be fair to the BBC, the headline of the article changed to "UK wage growth outpaces inflation" once the article was upgraded from breaking news to a complete article, and a mention of the lack of confidence in the change was added.
On 23 Feb, I noticed another BBC News with misleading figures: Net migration to UK falls by 49,000. This 49,000 is the difference between 322,000 (net migration for the year ending 2015) and 273,000 (net migration for the year ending 2016). However both these figures are estimates: in the original ONS report, they were placed in 95% confidence intervals of ±37,000 and ±41,000 respectively. As can be seen in the diagram below, there is a significant portion where these intervals overlap, so it cannot be said with any confidence whether or not net immigration actually fell.
Net migration in 2014-15 and 2015-16.
Perhaps the blame for this questionable figure lies with the ONS, as it appeared prominently in their report while the discussion of its accuracy was fairly well hidden. Although I can't shift all blame from the journalists: they should really be investigating the quality of these figures, however well advertised their accuracy is.
Both articles criticised here appeared on BBC News. This is not due to the BBC being especially bad with figures, but simply due to the fact that I spend more time reading news on the BBC than in other places, so noticed these figures there. I quick Google search reveals that the unemployment figure was also reported, with little to no discussion of accuracy, by The Guardian, the Financial Times, and Sky News.

Similar posts

World Cup stickers 2018, pt. 3
World Cup stickers 2018, pt. 2
A bad Puzzle for Today
A 20,000-to-1 baby?

Comments

Comments in green were written by me. Comments in blue were not written by me.
I've seen archaeologists claiming proof that event A happened before event B because the radiocarbon date of A was 50 years before B. Except the standard error on both dates was 100 years. They even showed the error bars in their own graphics, but seemed to not understand what it meant.

My favorite species of ignoring the measurement error is the metric conversion taken to way too many decimal places. The hike was 50 miles (80.467 kilometers) long.
Perry Ramsey
                 Reply
 Add a Comment 


I will only use your email address to reply to your comment (if a reply is needed).

Allowed HTML tags: <br> <a> <small> <b> <i> <s> <sup> <sub> <u> <spoiler> <ul> <ol> <li>
To prove you are not a spam bot, please type "u" then "n" then "c" then "o" then "u" then "n" then "t" then "a" then "b" then "l" then "e" in the box below (case sensitive):

Archive

Show me a random blog post
 2020 

Nov 2020

Christmas (2020) is coming!

Jul 2020

Happy e√3-π-ϕ-1 Approximation Day!

May 2020

A surprising fact about quadrilaterals
Interesting tautologies

Mar 2020

Log-scaled axes

Feb 2020

PhD thesis, chapter ∞
PhD thesis, chapter 5
PhD thesis, chapter 4
PhD thesis, chapter 3
Inverting a matrix
PhD thesis, chapter 2

Jan 2020

PhD thesis, chapter 1
Gaussian elimination
Matrix multiplication
Christmas (2019) is over
 2019 
▼ show ▼
 2018 
▼ show ▼
 2017 
▼ show ▼
 2016 
▼ show ▼
 2015 
▼ show ▼
 2014 
▼ show ▼
 2013 
▼ show ▼
 2012 
▼ show ▼

Tags

graphs pythagoras pi approximation day programming python flexagons wave scattering propositional calculus radio 4 asteroids oeis interpolation weak imposition royal baby mathsjam signorini conditions graph theory bubble bobble weather station map projections football puzzles games noughts and crosses tennis sport data visualisation london manchester approximation matrices latex platonic solids statistics video games folding paper triangles phd preconditioning tmip hannah fry golden ratio bodmas arithmetic javascript game of life convergence chebyshev menace inverse matrices draughts christmas simultaneous equations frobel misleading statistics sorting matrix of cofactors data bempp quadrilaterals light rugby the aperiodical sobolev spaces php news trigonometry dataset determinants plastic ratio binary chess pizza cutting numerical analysis european cup squares probability error bars raspberry pi martin gardner geometry dragon curves electromagnetic field sound royal institution manchester science festival finite element method hats rhombicuboctahedron mathslogicbot folding tube maps accuracy palindromes inline code boundary element methods matt parker coins gaussian elimination estimation captain scarlet golden spiral mathsteroids pac-man curvature hexapawn go computational complexity logs world cup talking maths in public stickers speed cross stitch chalkdust magazine matrix of minors fractals realhats countdown wool craft a gamut of games harriss spiral braiding pi cambridge london underground gerry anderson national lottery big internet math-off books advent calendar reuleaux polygons game show probability geogebra machine learning logic nine men's morris ucl exponential growth ternary christmas card matrix multiplication reddit dates final fantasy people maths twitter polynomials

Archive

Show me a random blog post
▼ show ▼
© Matthew Scroggs 2012–2020