Huawei Ascend P7 delisted from 3DMark rankings
August 29, 2014
We have removed the Huawei Ascend P7 from our 3DMark performance rankings for breaking our rules governing benchmark detection and optimization. It now appears delisted and without a score at the bottom of our Best Smartphones and Tablets list.
We have delisted the Ascend P7 because it does not comply with our rules. Specifically,
"The platform may not detect the launch of the benchmark executable. The platform must not...modify the usual functioning of the platform based on the detection of the benchmark."
The problem is explained and demonstrated by Joshua Ho of Anandtech in his article, "Examining Huawei's Benchmark Optimizations in the Ascend P7," which includes the data shown below.
Huawei Ascend P7 Performance in 3DMark Ice Storm Unlimited
- Play Store version score 7462
- Renamed version score 5816
As you can see, the Ascend P7's score when running the public version of 3DMark is 28 percent higher than the score from a renamed, but otherwise identical version of 3DMark not available to the public or manufacturers. We also confirmed these results in our own test lab.
This clearly shows that the Ascend P7 is detecting 3DMark by name, and not by workload. If the "CPU configuration is adjusted dynamically according to the workload in different scenarios," as claimed by Huawei in their response to Anandtech, then the score from the renamed version of 3DMark would be identical to the public version.
When a device detects 3DMark, and modifies its behaviour as a result, the score no longer reflects its real-world performance and cannot be used to make a fair comparison with other devices.
As Joshua says, "It's not really acceptable that Huawei is doing this, and I'm disappointed that they have chosen this path."
We share that disappointment and hope that this delisting will help persuade Huawei to realize the benefits of being fair and honest with their customers and the press.
We're committed to creating benchmarks you can trust
- 3DMark is the only mobile benchmark protected by rules for manufacturers.
- 3DMark is the only mobile benchmark with public documentation.
- Our Best Smartphones and Tablets list only shows publicly available models.
- Futuremark was the only benchmark developer to take action when news of benchmark optimizations in mobile devices first broke.
We invite members of the press to contact us for testing advice, tips on detecting optimizations or any other questions about 3DMark.
Download 3DMark - The Gamer's Benchmark
3DMark is a free Android benchmark app available from Google Play.
3DMark Steel Nomad will be free for all 3DMark users.
December 20, 2023
Using 3DMark to measure sustained performance.
December 13, 2023
New NPUs supported by Procyon AI Benchmark
November 16, 2023
A preview of 3DMark Steel Nomad
November 10, 2023
Qualcomm Uses Procyon AI Benchmark at Snapdragon Summit
November 1, 2023
3DMark celebrates its 25th year!
October 26, 2023
3DMark Adds iOS Support to Solar Bay Ray Tracing Benchmark
September 11, 2023
UL Solutions Launches 3DMark Solar Bay, New Cross-Platform Ray Tracing Benchmark
August 14, 2023
3DMark is now also available on Epic Games Store
June 20, 2023
Procyon® Is Addressing the Demand for Better AI Capabilities with Newly Launched AI Benchmark
June 15, 2023
Lenovo Focuses on Optimizing PC Hardware Testing with Procyon® Benchmarks
May 9, 2023
New Office Productivity Battery Life Scenario Released Today From Procyon®
May 4, 2023
Procyon® from UL Solutions Launch MacOS Benchmarks for Multi-Platform Score Comparison
May 2, 2023
3DMark adds feature test for AMD FSR.
March 30, 2023
UL Procyon® AI Inference Benchmark for Windows Released 22nd March 2023
March 20, 2023