Synthetic Benchmark & Real-world Download Speed Tests
Our Analysis of 4G LTE Networks in Singapore Continues ...
By now you might have noticed that we've been conducting a series of LTE networks assessment articles such as our surprise test on the three telcos across six shopping malls in the heart of the popular Orchard Road belt, the northern and southern territories and today, we continue our investigation at the eastern sector of our island.
Speed Test Parameters
To conduct an independent analysis, we got hold of three 4G SIM cards and all three telcos are completely unaware of these tests. This is to ensure we're able to report findings that are as close as possible to what you would experience when using your respective telco's 4G LTE service.
In our previous rounds of 4G LTE network tests last year, we conducted them using the Sony Xperia Z. This time, we used the Samsung Galaxy Note 3. Using three of these phones, we dedicated each phone to test each of the three telco's 4G LTE data service. We then subjected the three telcos' 4G LTE networks to the following tests in each location:
- Pinging a local server (Singapore, NewMedia Express) via the Speedtest.net app to check its download and upload speeds
- Downloading a 18.8MB PDF document from Dropbox
- Downloading a 20MB MP4 media file from Gmail
The above-mentioned tests were conducted at five different locations in the East:
- White Sands Shopping Centre (Pasir Ris)
- Tampines Mall (Tampines)
- Changi Airport Terminal 1
- Changi Airport Terminal 2
- Changi Airport Terminal 3
Before you read on, here are some important points you need to keep in mind while scanning through our performance findings:
- Prevailing network conditions will and always be affecting the speeds you will get on your device. For example, the data traffic and number of users in a certain location can play major roles in shaping the user experience.
- The tests were conducted at different locations at different timings, therefore actual performance may differ.
- All three telcos are constantly upgrading their network infrastructures based on their own plans and customers' feedback, therefore the speeds and results you see here may differ.
- The phone model used is also one of the main factors influencing the network test outcome. We compared the first three 4G LTE smartphones in June last year and found that the hardware and software optimizations can lead to performance discrepancies among devices.
- Based on all the above pointers, the aim of this article is not meant to determine the best network among the three teclcos. Instead, the main objective of this article is to have an overview of the 4G LTE coverage and throughput across the three telcos from a general assessment point of view.
1) Synthetic Benchmark: Speedtest.net
The first test was to determine the latency, download and upload speeds. We used the Speedtest.net app as it is one of the most widely used apps for testing mobile Internet speeds. We ran the app on the New Media Express server three times to get the average score.
Points to note:
M1: Its 4G LTE network currently supports theoretical download speeds of up to 75Mbps and upload speeds of up to 37Mbps. Its typical download speeds at fixed locations is 10.4Mbps to 40.7Mbps.
The typical download speed range refers to the range of download speeds that users can experience 80% of the time based on specific test conditions and parameters that are stipulated on www.m1.com.sg/MBBspeeds. Testing time frame: July to September 2013
SingTel: Its 4G LTE network currently supports theoretical download speeds of up to 75Mbps. Its typical download speeds range from 7.5Mbps to 42Mbps for 80% of the time, over a 24-hour period at various locations with 4G (LTE) coverage. Testing time frame: April to June 2013. For selected supported handsets, SingTel's 4G LTE network can support theoretical download speeds of up to 150Mbps.
StarHub: Its 4G LTE network currently supports theoretical download speeds of up to 75Mbps and offers typical download speeds of between 11Mbps to 40Mbps 80% of the time at an outdoor stationary location.
|White Sands Shopping Centre||68.86 (indoor)
|Tampines Mall||40.25 (indoor)
|Changi Airport T1||56.93 (indoor)||31.56 (indoor)||18.53 (indoor)|
|Changi Airport T2||42.31 (indoor)||36.27 (indoor)||19.37 (indoor)|
|Changi Airport T3||45.49 (indoor)||25.35 (indoor)||13.63 (indoor)|
|White Sands Shopping Centre||37.90 (indoor)
|Tampines Mall||40.64 (indoor)
|Changi Airport T1||10.00 (indoor)||7.32 (indoor)||12.46 (indoor)|
|Changi Airport T2||11.69 (indoor)||23.31 (indoor)||14.83 (indoor)|
|Changi Airport T3||14.00 (indoor)||8.02 (indoor)||15.16 (indoor)|
|White Sands Shopping Centre||32.3 (indoor)
|Tampines Mall||31.3 (indoor)
|Changi Airport T1||55.7 (indoor)||44.0 (indoor)||40.0 (indoor)|
|Changi Airport T2||53.0 (indoor)||78.7 (indoor)||89.0 (indoor)|
|Changi Airport T3||32.7 (indoor)||83.0 (indoor)||41.0 (indoor)|
Based on our findings from Speedtest.net, it seems that M1 has a convincing lead over the other two telcos in downloads and ping tests. Our earlier tests in the northern zone pegged a close fight between M1 and SingTel while the tests in Orchard Road revealed SingTel having the upper hand. M1 once again had the lowest latency across most test locations.
Edited 20/12/2103: We highlighted StarHub's performance in the following categories (Download in Tampines Mall, Uploads in White Sands and Tampines Mall) as the speeds indoors and outdoors are very consistent compared to the other two telcos. StarHub, hence, has the strongest showing in the upload category among the three telcos.
Do note that these results are not indicative of their true network capabilities as there are many factors out of our control that can affect the outcome.
2) Real-world Download Speed Test: Dropbox Test
Our second test involved downloading a 18.8MB PDF from Dropbox, arguably the most widely used cloud storage service used to share files.
|White Sands Shopping Centre||12.32 (indoor)
|Tampines Mall||20.60 (indoor)
|Changi Airport T1||
|14.50 (indoor)||18.30 (indoor)|
|Changi Airport T2||12.61 (indoor)||29.11 (indoor)||27.70 (indoor)|
|Changi Airport T3||11.23 (indoor)||18.16 (indoor)||36.46 (indoor)|
3) Real-world Download Speed Test: Gmail Test
Our third test involved downloading a 20MB MP4 file from Gmail to mimic a typical scenario of downloading email attachments on a mobile phone.
|White Sands Shopping Centre||12.81 (indoor)
|Tampines Mall||23.25 (indoor)
|Changi Airport T1||18.41 (indoor)||23.70 (indoor)||18.35 (indoor)|
|Changi Airport T2||
|14.58 (indoor)||17.56 (indoor)|
|Changi Airport T3||9.38 (indoor)||16.61 (indoor)||14.46 (indoor)|
M1 clinched the fastest download timings in the Dropbox test while it is a tie between M1 and StarHub in the Gmail test.
As the files were downloaded from external servers, it is highly likely that the downloading process was subjected to many variables such as server location and available bandwidth (from the sender in the other end).
This is further explained in our previous article on SingTel 4G LTE for smartphones, where we saw faster downloading speeds on SingTel Store & Share cloud storage service which has local servers and is optimized for faster speeds.
As far as the real-world tests are concerned, M1 seems to have performed quite well in this comparison. StarHub's performance also seems to be a tad better compared to the northern zone and Orchard road area. As iterated throughout this article, the results reflected here do not represent the actual performance of the mobile network for everyone.
Network infrastructure upgrades and improvements are always a work in progress for the telcos, so the reported outcome in this feature article is only valid for the period of our testing. We hope the results are a useful baseline of comparison and information, but beware that device type used and its underlying hardware can drastically affect the outcome.
Stay tuned for more updates as we reach out to cover the six major zones in Singapore (Orchard road, North, South, East, West and Central) to test the network performance in these areas!