"An ounce of honest data is worth a pound of marketing hype" - SPEC

The "marketing hype" in question:

Now, since I want to do a fair comparison between something that may resembles the C86-4G and an Intel processor, here's the details if I ever got a chance at it:

1. System configuration.

All processors are kept at stock settings and is configured so that no thermal throttling is detected during operation.

2. Testing procedure.

SPEC CPU 2006 will be the benchmark of choice. The testing procedure is as follows:

Compilation flags:

Optimization: -Ofast -fno-unsafe-math-optimizations -flto=<n_cores> -march=<march> -fomit-frame-pointer.

Portability: -DSPEC_CPU_LP64 -Wno-error=template-body (both base/peak).

For which the options are as follows:

For PGO runs, PGO options are enabled as follows:

PASS1_CFLAGS = -fprofile-generate
PASS1_CXXFLAGS = -fprofile-generate
PASS1_FFLAGS = -fprofile-generate
PASS1_LDFLAGS = -fprofile-generate
PASS2_CFLAGS = -fprofile-use
PASS2_CXXFLAGS = -fprofile-use
PASS2_FFLAGS = -fprofile-use
PASS2_LDFLAGS = -fprofile-use

Options such as bindN (N < n_threads) are configured so that bindN for N < n_cores maps to the physical threads.

Now, the testing procedure starts:

First, for SPEC06 STint, the testing procedure is as follows:

Launch base run and basePGO (as peak) run as above with 1 iteration.

Now, the peak tuning is generated by determining the effects of PGO on execution times (win/loss). After the PGO determination step for the peak tuning has been done, we do a full 3-iterations run and report the base/peak result.

For SPEC06 MTint, the testing procedure is as follows:

Launch base run and basePGO (as peak) run as above with 1 iteration, setting the number of copies to n_threads.

Next, launch the same configurations, but instead of setting the number of copies to n_threads, set it to n_cores.

Now, the peak tuning is generated by determining the effects of PGO and SMT on execution times (win/loss). After the PGO/SMT determination step for the peak tuning has been done, we do a full 3-iterations run and report the base/peak result.

All reported files for the final 3-iterations runs are made public (PDF, HTML, raw, flags, etc.), and the result is aggegrated to the tables as follows:

One can also plot graphs based on the test results.

The steps above should be simmilar for SPEC06 ST/MTfp scores, but optimization/portability options might need a change.