Hardlimit test bank

[cobito]

@krampak Well, that is a strange error. It is supposed that this Windows uses the same kernel as Windows Vista, although it is also true that I have not tested all the components of the test bench under Windows 7. And to be honest, I do not remember what the launcher was doing exactly. I will be away these days, but when I return I will take a look, if only out of curiosity to know what is going on there.

[cobito]

@krampak Today I was able to review the source code. The launcher does nothing but detect that the selectable mode is executable by the processor. If there is an error, it sends a signal to the window to display an error. Otherwise, it launches the execution of the test bank. That is, at first glance it seems that the CPU does not have the selected repertoire. Under normal conditions, if it is not compatible, an error message should appear in the window, but it seems that Windows 2008 manages the issue in a different way.

Initially, the failure comes from trying to execute µinstructions that this Xeon does not understand. Did you try running it in mode 0? I see that this Xeon model has support for AVX2 but I don't think the NT 6.0 kernel supports anything beyond SSE3. Reading around a bit, it seems that R2 is needed to access the vector repertoire. So that must be the problem. And in fact, that may be the reason why the error is not being displayed in a normal way (I had never tried running the graphical version on such an old Windows).

On another note (and with apologies for the delay; this week I couldn't do more), we have new micros.

More than a week ago, @garfield brought an i7-8750H with 6 cores, HT and a TDP of 45W. Today @rul3s has brought us what seems to be its twin brother from AMD: an Ryzen 5 2600X. I don't know if they have agreed on what, but this one also has 6 cores, comes with SMT and was launched on the market the same month and year as garfield's i7.

As it seems that the cover is a bit sad, I'm going to publish the mini-comparison here.

[krampak]

@cobito said in Hardlimit Test Bench:

@krampak Today I was able to review the source code. The launcher only detects that the selectable mode is executable by the processor. If there is an error, it sends a signal to the window to display an error. Otherwise, it launches the execution of the test bench. That is, at first glance it seems that the CPU does not have the selected repertoire. Under normal conditions, if it is not compatible, an error message should appear in the window, but it seems that Windows 2008 manages the issue differently.

Initially, the failure comes from trying to execute µinstructions that this Xeon does not understand. Did you try running it in mode 0? I see that this Xeon model has support for AVX2, but I don't think the NT 6.0 kernel supports anything beyond SSE3. Reading around a bit, it seems that R2 is needed to access the vector repertoire. So that must be the problem. And in fact, that may be the reason why the error is not being displayed in a normal way (I had never tried running the graphical version on such an old Windows).

On another note (and with apologies for the delay; this week I couldn't do more), we have new micros.

More than a week ago, @garfield brought an i7-8750H with 6 cores, HT and a TDP of 45W. Today @rul3s brought us what seems to be its twin brother from AMD: an Ryzen 5 2600X. I don't know if they agreed on what, but this one also has 6 cores, comes with SMT and was launched on the market the same month and year as garfield's i7.

As it seems that the cover is a bit sad, I will publish the mini-comparison here.

You're right, it works in SS3. In AVX and AVX2 it fails

[Magog]

I have uploaded a Dell 7470 with an I7 6600U.

Best regards.

[krampak]

I just uploaded another HP NUC like that (I'm still finding variants xD), this time an i3 4030U.

[rul3s]

I just uploaded a 2600x something a bit higher in revs
https://bm.hardlimit.com/result.php?bm=781be25ee54bb41e7440f5e2fac6559b1573

[krampak]

@rul3s said in Hardlimit test bench:

I just uploaded a 2600x that was overclocked a bit
https://bm.hardlimit.com/result.php?bm=781be25ee54bb41e7440f5e2fac6559b1573

How less curious

Ah, I haven't said anything, I didn't remember that the 1800 is 8/16.

[cobito]

Upon reviewing, I realized I had an interesting lineup of micros

@magog I was going to say that this model was already validated by @Namiga a year ago, but I went to compare the results and it seems that yours is quite a bit faster. Namiga's was at 2.8GHz when it ran the test and yours was 18% faster: at 3.3GHz. This can be attributed to limitations in the manufacturer's BIOS or something running in the background. But your CPU is pulling double the multi-threaded score. Looking at the performance per MHz of the architecture, it seems your result is quite close to other models like the 6500U, so your submission becomes a pretty representative sample of that model.

In another CPU lineup, Namiga brings us a Ryzen again, this time with integrated graphics. It's a Ryzen 5 2400G, a Zen (plain and simple) with 4 cores, 8 threads, and a maximum frequency of 3.9GHz. The composition of single and multi-threaded performance as well as in modes 0 and AVX2 doesn't resemble any AMD CPU and the Intel CPUs that appear in the comparison are very varied: from an "exotic" i5-5675R to the i7-4790K and all with big differences. Surely, the most interesting thing about this CPU is the integrated GPU although unfortunately we have no way to evaluate that. To not leave the analysis too bland, perhaps it's interesting to see what it resembles in each area:
· In single-threaded mode in FPU mode, the closest is the Core i3-7100.
· In single-threaded mode in AVX2 mode, it resembles the Core i7-4790
· In multi-threaded mode in FPU mode, it has some similarities with the Core i7-7700
· And in multi-threaded mode in AVX2 mode, it almost matches the aforementioned i5-5675R.

And to finish this round, as @krampak said, he's back with his NUCs. We are faced with an i3-4030U, a Haswell from the U series that was discontinued not too long ago. As a good U series, it comes with its TDP at 15W, which won't be a problem for its performance because its 2 cores (with HT) run at a fixed frequency of 1.9GHz. The processor turned out to be the slowest CPU in AVX2 mode of all the ones we have. In fact, the list of similar CPUs seems to have gotten a bit mixed up with the comparison. In summary, in FPU mode it's comparable to processors not of very high range of the old Wolfdale architecture. In AVX2 it's even below an AMD A9, which is nothing more than a Exacavator portable APU. That's right; the last one on the list. Although not to be too unfair to Haswell, that APU needed practically double the frequency to achieve similar performance.

Edited: By the way Krampak, the Xeon from Windows 2008 without R2 I added to the database but it shouldn't appear because the frequency was measured incorrectly in all tests (it measures 800MHz). I don't know if it's a problem with that Windows, that the machine is excessively loaded or some hardware issue (high temperature, incorrect configuration, etc).

[rul3s]

After this piece of news we will have to see how the 1903 affects the Ryzen with this benchmark.

[kynes]

@rul3s said in Hardlimit Test Bench:

After this piece of news we'll have to see how the 1903 affects the Ryzen with this benchmark.

I understand that in theory it shouldn't affect much, since the novelty is that instead of separating the threads as much as possible to avoid hot spots, which hindered communication between threads, now they will bring them together as much as possible, occupying a CCX before starting to "deposit" threads on the other, because of the way games are developed. From how the bench should work, I understand that with independent threads that don't communicate with each other because of the type of calculation they have to perform, the result shouldn't vary much.

[Fassou]

I passed the test to a laptop that's a few years old, and I see that the micro (4702MQ) was not, and it links to the datasheet of a Pentium

The second one in the list https://bm.hardlimit.com/search.php?s=4702

Best regards!

[cobito]

@fassou It's already added. It's a 2013 Haswell. It has 4 cores with HT, a max frequency of 3.2GHz, and a TDP of 37W. With the data available, the most similar CPU is an i5-4440, which makes sense considering that they are contemporaries with very similar specs. Although the MQ has a TDP of less than half that i5, which is surprising since in multi-thread, the i5 only beats it by 12%. Going to slightly different processors, the i3-7100U that came out 3 years later, has very similar single-thread performance.

[cobito]

The day before yesterday, @krampak brought an i7-7700HQ at stock frequency for the first time, even though there were already results at a lower frequency. I realized that some manufacturers (in my case Dell), limit the speed of the i7 HQs, I imagine for thermal reasons. I have a couple of i7-6820HQs here but since they are capped at 3.2Ghz, they are excluded from the statistics.

To the point: the 7700HQ is a portable processor with 4 cores and HT that reach 3.8GHz. With its 45W TDP, it manages to match the performance of an i5-7500, which is practically the same except that the desktop processor allows a higher TDP and does not have HT.

In the overall ranking, it appears in sixteenth position for single-thread and in twenty-third for multi-processing although if I'm not mistaken, it is the second in both rankings in portable CPUs.

[kynes]

@cobito said in Hardlimit test bench:

The day before yesterday, @krampak brought for the first time an i7-7700HQ at stock frequency, even though there were already results at a lower frequency. I realized that some manufacturers (in my case Dell), limit the speed of the i7 HQs, I imagine for thermal reasons. I have here a couple of i7-6820HQs but since they are capped at 3.2Ghz, they are excluded from the statistics.

To the point: the 7700HQ is a portable processor with 4 cores and HT that reach 3.8GHz. With its 45W TDP, it manages to match the performance of an i5-7500, which is actually practically the same except that the desktop processor allows a higher TDP and does not have HT.

In the general ranking, it appears in the sixteenth position for single-thread and in the twenty-third in multi-processing although if I am not mistaken, it is the second in both rankings in portable CPUs.

When comparing with the i5-7300HQ (my laptop) I am surprised by the similarity in the results, is it possible that this 7700HQ had HT disabled?

https://bm.hardlimit.com/compare.php?cpu1=Intel(R)-Core(TM)-i7-7700HQ-CPU-@-2.80GHz-591593&cpu2=Intel(R)-Core(TM)-i5-7300HQ-CPU-@-2.50GHz-591593

[cobito]

@kynes I obtained the data for what I'm about to comment from these results for the 7300HQ and the 7700HQ, both the best results in AVX2.

The 7700HQ has a frequency 8.5% higher than your 7300HQ while the former beats you by 5.4% in single-core. In principle, being the same architecture with everything identical except the frequency, a linear increase in performance should be obtained to the frequency, but this is not the case. If we look at the results of each test individually, both micros get a score per MHz very similar in the first three tests in single-core. The difference occurs in test#4 (the memory test).

7700HQ

7300HQ

When we go to multi-core, important differences begin to be noticed in tests #1 to #3. And in test #4 (memory), your PC gets 3 times more performance than @krampak's. When we look at the hardware details, we see why this might be happening: your laptop is using two memory slots while @krampak's is only using 1. That is, your PC is probably using dual channel while Krampak's is not.

That you get triple the performance in memory I think is not explained only by having dual channel, so there must be other variables: perhaps different chipsets, different latencies, BIOS with different characteristics or whatever: yours is an MSI and Krampak's is an HP so you go figure out what they've caught or stopped catching.

[kynes]

@cobito But my doubt is more about the absence of notable differences between a 4-core processor like the 7300HQ vs an 8-core processor like the 7700HQ. I understand that one should notice a noticeable difference by having hyperthreading activated, right?

[cobito]

@kynes I have seen few models where HT brings a noticeable improvement. I specifically set the mmt ratio so that I could evaluate HT/SMT. You can see the value in the multi-threaded results column in brackets. It is the ratio of the multi-threaded result per thread (that is, the multi-threaded result divided by the number of execution threads) and the single-threaded result of the same test. In a processor without HT (where one process runs per physical core), the result should be 1 (or close to 1): this means that by running multiple threads, the performance of each core is the same as if a single thread were running on one of them. If two threads were run per core without HT/SMT, the result should be 0.5, which is that the performance of each thread is 50% of that obtained in the single-threaded result. In one with HT, it should be higher than 0.5 since that would mean that the performance of several threads executed on a physical core is higher than the performance of a single thread executed on a physical core. But the truth is that in few processors have I seen HT results that substantially exceed that 0.5.

In this result of the i7-4790k or in this one of an i7-7700k, you can see a ratio of about 0.6 in tests 2 and 3. This means that in the i7-4790k, HT can increase performance by 20% compared to not having it. It is possible that this lack of tangible results from hyperthreading is a flaw in the design of the benchmark or that HT is nothing more than a gimmick that slightly improves performance only in some models; the benchmark in this aspect is quite simple since all it does is run several processes in parallel and evaluate the result of each one separately and then sum all the results.

Regarding the result of the 7700HQ in particular, I believe that the memory is acting as a bottleneck there. Perhaps more results and more diverse machines would be needed to draw clearer conclusions.

[cobito]

Bueno, bueno, bueno. Pues @hAyO has brought us the first Zen 2. It's the Ryzen 7 3700X and you can see the results on the front page.

[krampak]

Does @cobito still have the offline version (usb) of the benchmark available? I have a Celeron N4000 1.1Ghz but Windows 10 in S mode won't let me run hlbench xD

[cobito]

@krampak Yes, the link is in the thread of the Live-USB. The only drawback of that version is that it is not compatible with AVX. But for that N4000 it will not be a problem.