Back in February, Futuremark released a new version of its popular synthetic benchmark tool, 3DMark, for use across cross-platform devices. We've already tested the desktop gaming benchmark, Fire Strike, on a number of desktop graphics cards, and now a version compatible with Android has been released (unfortunately we're still waiting for versions compatible with iOS, Windows Phone 8 and Windows RT).
In this special feature, we'll be testing out the Ice Storm and Ice Storm Extreme benchmarks on a range of Android devices to see how they fare.
The Ice Storm benchmark is designed for smartphones, mobile devices and ARM architecture computers. It consists of two tests:
Test one is vertex shader heavy and has a generous amount of shadow rendering, while test two is pixel shader heavy and includes particles and light post-processing effects. Results for both tests are combined together for a final score. Ice Storm Extreme takes everything up a notch, with higher resolution textures and more particles and shadows, designed to simulate more intensive mobile gaming.
According to Futuremark, the scores are comparable across all the platforms that run this benchmark, but since we only have the Android version to test for now, we'll have to re-visit this test when it's made available to other mobile OS platforms too.
Requirements for running the Android version of this benchmark are as follows:
For those curious on how the full test would look like, check out this video:-
For this setup we'll be running both Ice Storm and Ice Storm Extreme on a variety of Android devices to see how they stack up.
The list of devices tested and compared are as tabulated:-
|Device||ASUS Nexus 7||HTC One||LG Nexus 4||Samsung Galaxy Note II (LTE)||Sony Xperia ZL|
|CPU||NVIDIA Tegra 3 quad-core 1.3GHz||Qualcomm Snapdragon
|Qualcomm Snapdragon S4 quad-core 1.5GHz||ARM Cortex A9 quad-core 1.6GHz||Qualcomm Snapdragon S4 quad-core 1.5GHz|
|GPU||ULP GeForce||Adreno 320||Adreno 320||Mali-400MP||Adreno 320|
|OS||Android 4.1||Android 4.1||Android 4.2||Android 4.1||Android 4.1.2|
The three Qualcomm Snapdragon S4 quad-core 1.5GHz, Adreno 320-equipped devices (HTC One, LG Nexus 4 and Sony Xperia ZL) were the winners here, overwhelmingly outperforming both the Nexus 7's NVIDIA ULP GeForce GPU and the Note II's ARM Mali-400MP GPU. In fact, our Adreno GPU based devices outscored their competition by almost 3 times!
Individually, both the HTC One and LG Nexus 4 were strong contenders, with the Sony Xperia ZL falling behind by about 10%. With almost identical specs, the disparity between the scores seems to be down to the different versions of Android OS each is running. This may be due to the GPU driver sets installed with each OS version, or possibly better optimization with the benchmark.
Both the Nexus 7's NVIDIA ULP GeForce GPU and the Note II's ARM Mali-400MP GPU performed about equally, with the Nexus 7 holding a slight lead on the easier Ice Storm benchmark and the Note II performing better at Ice Storm Extreme. This may be due to the Note II's higher RAM buffer performing better at the more taxing Extreme setting.
Right now, if mobile gaming on an Android device is important to you, there's absolutely no doubt: you need a Qualcomm Snapdragon, Adreno 320 GPU-equipped device. For intensive OpenGL ES 2.0-based applications - which Ice Storm and Ice Storm Extreme are designed to emulate - the Adreno GPU is just miles ahead of its competition.
Of course, NVIDIA will soon be releasing its Tegra 4 processors, of which NVIDIA has banked much of its success on its graphical processing prowess, so things may soon change.
It also remains to be seen how Android OS as a whole stacks up against iOS, Windows Phone 8 and Windows RT. So stay tuned for future comparisons when these benchmarks become available.
Before we wrap up, another overlooked fact is that there aren't many apps at the moment that skew towards needing a powerful graphics core to base your mobile device purchase off 3DMark results. After all, with over 40% of Android devices still stuck on Gingerbread edition of the OS, developers will continue to design apps that compatible with the widest base possible.
Certainly 3DMark results will help you understand who has a better graphics subsystem, but it's quite likely just going to be an interesting statistic for some time unless more gaming apps are readily seen tapping on to the GPU capabilities. With newer devices sporting ever increasingly powerful graphics engines, in time to come, we believe overall experience will also be affected by GPU capabilities since developers would then have a wider user base who've appropriate devices to benefit from the added graphical prowess. Perhaps in the near future, but it would probably not matter a lot to most consumers at this point of time.
Regardless, HardwareZone will be ready when the shift happens, so we'll keep the tests going to collect results and report our findings when the time is right.