For the first time, Opensignal has assessed how mobile users experience real-time multiplayer mobile gaming when using different operators’ networks in the U.K. EE won the inaugural Games Experience award, beating Vodafone by 1.3 points.
However, none of the U.K.’s national operators placed in the two top categories (Excellent and Good), indicating that there is considerable room for improvement on this measure of the mobile experience.
As was the case back in April, EE has won all but one of the national awards — the exception again being Voice App Experience, which has now been won by Vodafone for three reports in a row.
Perhaps the biggest announcement in the U.K. mobile market since the last report, is the news that Ofcom is planning a spectrum auction in January 2021 to assign 80 MHz of spectrum in the 700 MHz band and 120 MHz of spectrum in the 3.6-3.8 GHz band.
Opensignal examined the mobile network experience of the four main mobile network operators in the U.K. – EE, O2, Three and Vodafone, over a period of 90 days beginning June 1, 2020, to see how they fared, and further delved deeper into 17 cities, comparing the results of these four operators.
They used 5G measurements in addition to those from previous generations of mobile network technology when determining the overall scores for each metric.
Key findings
EE wins first ever Games Experience award in the U.K.
The Games Experience metric is a measure of how mobile users experience real-time multiplayer mobile gaming on an operator’s network. Measured on a scale of 0-100, it analyses how users’ multiplayer mobile gaming experience is affected by mobile network conditions including latency, packet loss and jitter to determine the impact on gameplay.
EE is the first U.K. operator to win the Games Experience award and it has done so with a score of 72.3 — 1.3 points ahead of second-placed Vodafone.
Vodafone wins Voice App Experience award for third time in a row
While EE’s dominance over the rest of the awards table has remained unchanged, so has Vodafone’s grip on the Voice App Experience award, as the operator was able to win it for the third consecutive report. Its score of 78.2 points beat second-placed EE’s by 0.3 points, marginally down from the 0.4 points seen in our last report.
EE holds top spot for Video Experience, O2 pushes past Vodafone to come second
EE has now held the Video Experience award for four reports in a row, this time achieving a Video Experience score of 74.2 out of 100. In addition, O2 has taken second place from Vodafone.
While EE’s lead over O2 is 1.8 points higher than its lead over Vodafone for this measure of the mobile experience last time round, not everything has gone EE’s way. The operator has dropped down a category — from Excellent (75 or above) to Very Good (65-75), due to a 1.4 point drop in its score.
As a consequence, EE is now on an even footing with its rivals, all of whom also have placed in the Very Good category.
Download Speed Experience gap between EE and Vodafone widened
EE is once again the victor when it comes to the average overall download speeds observed by users in the U.K. and its lead over second-placed Vodafone has risen from the 11.6 Mbps (50.3%) that was seen in the previous report to 14.6 Mbps (71.4%).
However, the main cause of this increase was a 2.6 Mbps (11.4%) drop in the average overall download speeds observed by our Vodafone users rather than the 0.4 Mbps increase seen by their EE counterparts, which took its score to an impressive 35 Mbps.
Vodafone closes in on EE on Upload Speed Experience
While EE has won the Upload Speed Experience award for the fourth time in a row, its lead has slipped substantially. The average overall upload speeds observed by EE users fell by 0.9 Mbps (9.8%), while their counterparts on Vodafone saw theirs fall by a more modest 0.5 Mbps (6.1%).
These changes are enough to cut EE’s lead by 30%, reducing it to just over 1 Mbps for this measure of the mobile network experience.
National Analysis
Video Experience
EE once again won the Video Experience award while O2 has taken second place from Vodafone. EE has dropped down a category – from Excellent (75 or above) to Very Good (65-75), due to a 1.4 point drop in its score, while the other operators’ ratings are unchanged. This means that users on all four national operators observed a Very Good Video Experience.
This rating indicates that users enjoyed generally fast loading times and only occasional stalling but the experience might be somewhat inconsistent across users and/or video providers/resolutions.
In addition to being a big factor in dropping Vodafone down to third place from second — and enabling O2 to take second place — the 4.4 point drop in the Video Experience observed by Vodafone users was chiefly responsible for the gap between Vodafone’s and 3’s Video Experience scores shrinking to just 1.4 points, down from the 4.2 seen in the previous report.
Games Experience
EE is the winner of the inaugural Games Experience award in the U.K. — its score of 72.3 was 1.3 points ahead of second-placed Vodafone. However, EE, Vodafone and O2 all placed in the Fair (65-75) category for this measure of the mobile experience, while 3 ended up with a Poor (40-65) rating.
Those Opensignal users who had a Fair Games Experience found it to be “average” and in most cases the game was responsive to the actions of the player with the majority of players noticing a delay between their actions and the outcomes in the game.
A Poor rating indicates that most users found this level of experience to be unacceptable and many of them felt a lack of controllability in the Games Experience.
Measured on a scale of 0-100, Opensignal’s Games Experience metric quantifies the experience when playing real-time multiplayer mobile games on mobile devices connected to servers located around the world. The approach is built on several years of research quantifying the relationship between technical network parameters and the gaming experience as reported by real mobile users.
These parameters include latency (round trip time), jitter (variability of latency) and packet loss (the proportion of data packets that never reach their destination). Additionally, it considers multiple genres of multiplayer mobile games to measure the average sensitivity to network conditions.
The games tested include some of the most popular real-time multiplayer mobile games (such as Fortnite, Pro Evolution Soccer and Arena of Valor) played around the world.
Voice App Experience
Unlike the other measures of the mobile experience in which EE continues to reign supreme, Vodafone continues to hold onto the Voice App Experience award. The operator has managed to win it on no less than three consecutive occasions and this time has won with a score of 78.2 points out of 100, while its margin of victory over second-placed EE has shrunk slightly – dropping to 0.3 points.
In stark contrast to the previous report in which users across all four national operators saw their Voice App Experience improve — including a gain of 5 points by EE — this time round all four operators saw their scores decline by between 2.9 and 3.6 points.
O2 users saw the largest drop both in absolute and percentage terms, 3 the least. The declines were sufficient to push EE, O2 and Vodafone down a category — from Good to Acceptable — which puts them in line with 3.
An Acceptable Voice Experience rating means that some users experienced perceptible call quality impairments in the form of clicking sounds or distortion of short duration, and/or the volume may not have been sufficiently loud. However, listeners were generally able to comprehend without repetition.
Opensignal’s Voice App Experience measures the quality of experience for over-the-top (OTT) voice services — mobile voice apps such as WhatsApp, Skype, Facebook Messenger, etc. — using a model derived from the International Telecommunication Union (ITU)-based approach for quantifying overall voice call quality and a series of calibrated technical parameters. This model characterizes the exact relationship between the technical measurements and perceived call quality.
Download Speed Experience
Users have once again observed the highest average download speeds on EE’s network, with the operator racking up an impressive Download Speed Experience score of 35 Mbps, up 0.4 Mbps from the previous report. EE’s lead in this regard has widened from 11.6 Mbps to 14.6 Mbp, a rise of 3 Mbps.
This means that EE’s score is 71.4% higher than second-placed Vodafone. The increase in EE’s lead was primarily due to a 2.6 Mbps drop in the average speeds reported by Vodafone users. This decline has another consequence – the gap between Vodafone and third-placed 3 has narrowed to just 1.4 Mbps, down from 4 Mbps.
Upload Speed Experience
In contrast to Download Speed Experience, EE’s lead over second-placed Vodafone, has narrowed somewhat, dropping from 1.5 Mbps to 1.1 Mbps, a decline of 0.5 Mbps. The average upload speeds observed by users are down on those reported back in April across all four operators, with EE users experiencing the largest decline — 0.9 Mbps.
Vodafone’s Upload Speed Experience score dropped by 0.5 Mbps, while both 3 and O2’s scores dropped by 0.1 Mbps. As a result, 3’s score is 1.1 Mbps behind that of Vodafone’s, down from the 1.5 Mbps that users observed back in April.
4G Availability
EE is once again the winner of the 4G Availability award, this time with a score of 94.5%. As with Upload Speed Experience, all four operators’ scores for 4G availability have fallen.
As a result of this, the 90% mark remains elusive for O2 and Vodafone and 3’s score has slipped below 80%. EE’s lead over second-placed O2 has widened slightly, rising from the 5.6 percentage point difference reported back in April to 6.1 percentage points.
Meanwhile, the gap between O2 and Vodafone has narrowed to just 0.7 percentage points.
4G Coverage Experience
Users have observed incremental improvements in their 4G Coverage Experience on all four operators’ networks – 0.2 points on O2, 3 and Vodafone; along with 0.1 points on EE. EE is now only 0.3 points off a perfect score and has held onto the 4G Coverage Experience award.
However, now only 0.2 points separate EE’s score from that of second-placed O2, down from the gap of 0.3 points seen in our previous report.
Opensignal’s 4G Coverage Experience metric is a measure of how mobile subscribers experience 4G coverage on an operator’s network. Measured on a scale of 0-10, it analyzes the locations where customers of a network operator received a 4G signal relative to the locations visited by users of all network operators.
Regional Analysis
EE is the only operator in the U.K. to win any of the city awards outright, across all six of our measures of the mobile experience assessed at the city-level. The operator racked up 64 wins out of a possible 102 and tied for the remainder. The other operators at best jointly won some Opensignal city awards.
In 16 out of the 17 cities investigated, EE users observed an Excellent Video Experience, while the same was only true for O2 and Vodafone users in Sheffield and Slough, respectively. In most of the cities studied, 3, O2 and Vodafone users enjoyed a Very Good Video Experience, though 3 users in Bristol and Hull had to settle for a Good Video Experience instead.
Turning to Games Experience, only the score reported by users on EE’s network in Slough — 85.1 — placed in the Excellent category. EE users had a Very Good Experience in 15 out of 17 cities, while their Vodafone counterparts did so in 12 and there were only two cities — Nottingham and Slough — where O2 users did so. While there were five cities in which 3 users reported a Fair Games Experience, there were 12 in which they observed a Poor one.
Looking at Voice App Experience, EE won outright in two cities — Birmingham and London. It drew in 10 more with Vodafone and there were three-way ties between these two operators and O2 in the remaining five cities. EE users observed a Good Voice App Experience in 12 cities, but Vodafone users weren’t far behind with 11. In the vast majority of the cities analysed 3 and O2 users had an Acceptable experience, though users on 3 in Hull experienced a Poor one.
EE users observed average download speeds above 40 Mbps in 14 cities, with Liverpool and Birkenhead just 0.8 points shy of the 40 mark. In addition, EE users in Belfast’s average speeds were 0.3 Mbps below 50 Mbps. Vodafone did not obtain a Download Speed Experience score of 40 Mbps or higher in any of the 17 cities, although it came closest in Slough where its score was only 0.8 Mbps shy of the mark. Users on 3’s network experienced speeds below 20 Mbps in four cities: Hull, London, Bristol and Belfast, while the same was true for O2 users in seven cities — including the score of 10.7 Mbps seen by them in Reading.
On Upload Speed Experience, EE users observed speeds above 10 Mbps in 13 cities — with those in Reading just 0.1 Mbps shy of the 12 Mbps mark — and the same was true for Vodafone users in Glasgow and Slough. O2 and 3 users observed average upload speeds below 7 Mbps in 15 and three cities, respectively.
While EE did not hit the 95% mark for 4G availability nationally, it exceeded it in all 17 cities and the operator won in all but one city for this measure of the mobile experience — it tied with Vodafone in Slough. Speaking of Vodafone, the operator scored above 95% in Nottingham and below 90% in Cardiff. Users on Vodafone and O2 spent more than 90% of their time on 4G in 16 and 15 cities, respectively, while users on 3’s network only observed a score above 90% in Slough.
Moving onto the U.K.’s national capitals, in Belfast EE won on Games Experience, Download Speed Experience, Upload Speed Experience and 4G availability. There was a three-way tie between EE, O2 and Vodafone on Video Experience and a draw between EE and Vodafone on Voice App Experience.
EE had slightly less success in Cardiff, snapping up awards for Video Experience, Download Speed Experience and 4G availability. Vodafone and EE were joint winners for Games Experience, Voice App Experience and Upload Speed Experience.
In Edinburgh, EE won awards for four out of six metrics, tied with O2 on Games Experience and users there observed a three-way tie on Voice App Experience between EE, O2 and Vodafone. The average download speed observed by EE users was more than 20 Mbps faster than that seen by those on the operator with the second highest score — O2. Finally, in London, EE won outright across all six measures of the mobile experience
