Chipzilla in Hot Water For Cheating on Benchmarks

/ 2 months ago

You may or may not know that SPEC CPU 2017 is a popular benchmark used for high-end servers and data centres, and I guess either way, it doesn’t matter if you do or not. However, the Standard Performance Evaluation Corporation or “SPEC” is designed to provide an honest and fair benchmark for these systems, and has just binned over 2600 results for the Chipzilla Xeon chips… uh-oh!


The non-profit group SPEC has now said that it would stop publishing SPEC CPU 2017 results for Chipzilla PCs, which they claim have been running a modified version of the Chipzilla compiler. The group say they’ve been running a “sneaky optimisation for a specific workload” thus making their results look more favourable.

Standard Performance Evaluation Corporation

The benchmark in question is the Chipzilla API DPC++/C++ Compiler, which SPEC say has been modified for the 523.xalancbmk_r / 623.xalancbmk_s benchmarks. Jibberish to some of you I know, and largely it is to me to if I’m honest. But tinkering with software to produce higher benchmark scores is simplified enough that anyone can understand it. However, it has resulted in a 4-9% boost in the scores, which is significant.

There’s now a warning applied to thousands of results reading that “The compiler used for this result was doing a compilation that especially boosts the performance of the 523.xalancbmk_r / 623.xalancbmk_s benchmarks using insider knowledge.” Which is to say they’re completely worthless results. According to a Phoronix report, 2022.0 to 2023.0 models are affected, but similar systems from other years seem to be unaffected by the warnings.

Here is a dramatisation of the events…

Topics: , ,


By supporting eTeknix, you help us grow and continue to bring you the latest newsreviews, and competitions. Follow us on FacebookTwitter and Instagram to keep up with the latest technology news, reviews and more. Share your favourite articles, chat with the team and more. Also check out eTeknix YouTube, where you'll find our latest video reviews, event coverage and features in 4K!

Looking for more exciting features on the latest technology? Check out our What We Know So Far section or our Fun Reads for some interesting original features.

eTeknix Facebook eTeknix Twitter eTeknix Instagram eTeknix Instagram
  • Be Social With eTeknix

    Facebook Twitter YouTube Instagram Reddit RSS Discord Patreon TikTok Twitch
  • Features

Send this to a friend