Hardlimit test bank

[Xevipiu]

@cobito said in Hardlimit test bench:

@Xevipiu What do you mean by processors that are not 100% native to AVX2?

There are processors that do not use the instructions 100%, but when they do, they give result inaccuracies.
But leaving this aside, using AVX vs AVX2, the results are abysmal.

You would have to cap them according to the instructions of each processor to go well, don't you think?

[cobito]

@Xevipiu I still don't understand. What do you mean by there are processors that don't use instructions at 100%? The benchmark is designed to do 4 tests: two of the SIMD type (where having AVX for example will improve the results), one of memory (where more cache and/or faster memory will give you better results) and a generic test whose result has little dependence on the instruction set.

If you want to compare two processors under equal conditions, you just have to pass the benchmark in the same mode for both. If you don't have a result in the same mode, you can look at test#4 to get an idea.

The operations that are executed in all modes are exactly the same. If in a processor in FPU mode it gives you 1000 points in test#1 and changing to AVX2 mode it gives you 15000 points it simply means that for the vector calculation of integers, AVX2 multiplies by 15 in performance. That is the reality. Another matter is that then you are not going to run multimedia software or games and you don't care about vector instructions. In that case, once again, you either have test#4 or the complete benchmark in a lower mode.

One of the objectives of HLBM is precisely to evaluate how much the recent sets improve performance and also how optimized the older sets are in current CPUs.

As I understand it, what you are asking is that I weigh the results of higher modes so that the final result is similar to the execution in a lower mode? In that case it wouldn't make sense to have the possibility of evaluating the performance of recent sets.

[Xevipiu]

For whom I go, if it is in the sense of evaluating the differences between instructions, what was said makes sense

[cobito]

@Xevipiu Let's see, the benchmark is very versatile and serves for many things. Among other things, it serves to evaluate:
· Instruction sets (with different modes)
· Implementations of Hyperthreading and SMT: in the results you can see the score per execution thread in the multithread test. If the micro has HT or SMT, the closer this result is to the monothread result, the better HT or SMT the micro has. If that result is closer to 50% of the monothread result, the worse HT/SMT it has.
· Amount and speed of cache memory (test#3).
· Comparison of processors to do a specific task: using the same mode on both micros or comparing test#4.
· Comparison of processors in general: using the highest possible mode.

That is to say, it is not a benchmark only to evaluate a micro with a generic and bland bench. You have much more information and with it you can know how well or badly a model goes for a multitude of tasks.

Now, it is true that in the ranking there are results that are disproportionate with respect to micros that are not so much slower, just because some support AVX2 and others do not. And in the real world, the difference between both models is not so abysmal. That is why it is also good that tests are passed in lower modes (even if the result is not for the top) because it is the best way to make more reliable comparisons.

When there are more results, a new top will appear only for mode 0. There the sieve will take out a few that are now in the first positions. But in order to be able to get that kind of statistics (and others) it is still necessary that more results are sent.

[Xevipiu]

I left you the result of a crappy MAD, the truth is that it disappoints me for what they are, I hope that the next generation will measure up

[cobito]

@Xevipiu The first Ryzen! Well, in multi-threading it seems to have eaten krampak's xeon. Although compared to the 7700K in single-threading, it pulls 10% less performance per MHz. There it really has stuck.

[Xevipiu]

The monore test of the 7700k, was only at 5.11Ghz with the previous version

[cobito]

News

There is a new version of the program that speeds up the start time. In return, the initial screen only shows the stock frequency. To see the real frequency, you have to validate the result. We have managed to get the real frequency to have an error <3%. It is recommended that there are no other programs running during the test (not even CPUZ or similar).

The central has also received changes. To know them, you can see the first post of this thread.

In addition, a top10 has been added for mode 0. This top is only for the brave who want to compare their micro one-on-one without fancy sets that inflate scores excessively. I invite you to pass the bench in mode 0 unless you are a chicken

For now there are only 6 results in the ranking of mode 0 with pretty old micros, so you have it easy to appear.

[krampak]

I have tested Mode0 with the E5-2620V4 but the calculated speed is 1.78Ghz, I am not sure if the CPU does not really go up to the theoretical 2.1Ghz or if it is an error in the benchmark meter.

Edit: It seems to be a problem with the test, as it does not capture the maximum frequency that the CPU reaches (although it also does not show the one it has at rest, rather an average):

[cobito]

@krampak The performance graphs of that result have some brutal ups and downs. Something is happening on that PC that is interfering with the program. If during the frequency measurement it has caught one of those downs, it will not show the real one.

In fact, the mmt ratio indicates that there is something running in the background that is consuming quite a few processor cycles.

[krampak]

@cobito said in Hardlimit test bench:

@krampak The performance graphs of that result have some brutal ups and downs. Something is going on in that PC that is interfering with the program. If during the frequency measurement it caught one of those downs, it won't show the real one.

In fact, the mmt ratio indicates that there is something running in the background that is consuming quite a few processor cycles.

There is a resident antivirus that I can't deactivate. I guess it must be that.

PD: What a lunch the Ryzen has had in mode0...

[krampak]

@cobito On an i5 6500 it just gave me 6Ghz of calculated frequency... and curiously (I don't know if it's a thing of the new version) it has surpassed the old result of the i5 7500.

https://bm.hardlimit.com/result.php?bm=9796222558dc06a8a250c56af2a06753144

[cobito]

@krampak What is the real frequency? This weekend I will make the last change of the season, to see if I can close the topic of the frequency once and for all.

The code of the test bench has not changed (nor will it change in the future without a very good reason), so if it has given more it is because it has performed better.

[krampak]

@cobito said in Hardlimit test bank:

@krampak What is the real frequency? This weekend I will make the last change of the season, to see if I can close the topic of the frequency once and for all.

The code of the test bank has not changed (nor will it change in the future without a very good reason), so if it has given more it is because it has performed better.

Well the real one would be 3.6Ghz at most (turbo), it is an i5 6500 (without K). When I finish the d*** Windows 10 update I will test it on another i5 7500. I also have a laptop with an i7 6560U that I will test in the afternoon.

[Xevipiu]

The day before yesterday, at 4.25Ghz, the MAD gave me 4.5Ghz

[cobito]

@Xevipiu Ok, the source of the failure should be the same as that of krampak. The more modern the processor, the more screwed it is. I have a solution in mind that I hope will work.

By the way @krampak, I've seen the result of the 7500 and that of the 6500 and notice the mmt ratio of the 7500: in tests 1 and 2 it gives around 0.8. That's very low for not having Hyperthreading. In contrast, the 6500 comes out above 0.9 in all cases (except test 3) which are normal values (for not having HT). Just like in yesterday's Xeon, in that 7500 you have something consuming CPU when the benchmark is running; that's why it gave a worse result than the 6500.

Basically the mmt ratio should give above 0.9 (close to 1) in micros without HT and above 0.5 in micros with HT. If it comes out less, something is interfering.

[krampak]

@cobito said in Hardlimit test bench:

@Xevipiu Ok, the source of the failure must be the same as that of krampak. The more modern the processor, the more fucked it is. I have a solution in mind that I hope will work.

By the way @krampak, I've seen the result of the 7500 and that of the 6500 and look at the mmt ratio of the 7500: in tests 1 and 2 it's around 0.8. That's very low for not having Hyperthreading. In contrast, the 6500 comes out above 0.9 in all cases (except test 3) which are normal values (for not having HT). Just like in yesterday's Xeon, in that 7500 you have something consuming CPU when the test bench is running; that's why it gave a worse result than the 6500.

Basically the mmt ratio should be above 0.9 (close to 1) in micros without HT and above 0.5 in micros with HT. If it comes out less, something is interfering.

Now it has come out at 0.96 (except test 3 which gives very little), even so there is very little difference with the 6500 despite taking out 200Mhz and a generation. I'm talking about Modo0, I can't compare the other because I've passed ModoVI to the 7500 (after seeing that Xevipiu passed in VI, now I understand so much difference. If I can, I'll go passing VI to the old ones).

[Xevipiu]

@Cobito The MAD can fail due to its multiplier that scales from x0.25 to x0.25, for example: 100.4 x 40.25 = failure yesterday its real frequency reading.

Also, when the application starts, it doesn't read the real one, but the startup one

I hope it helps you

[Fassou]

@cobito Data from the last test on hlbm.exe 1.0.5

https://bm.hardlimit.com/result.php?bm=49474e66f4e94dce583921ba03045b73153

Summary:

3200Mhz nominal of the model Vs 3199Mhz in app
3600Mhz by CPU-Z Vs 3611Mhz in HLBM Central

Much more approximate :thumbsup_tone1:

[cobito]

There is a new version that I hope will fix the frequency measurement errors. I think it is calibrated a bit low (maybe it gives you a measured frequency slightly below the real one). I don't have PCs on hand to be able to do all the tests I need, so over time I will correct it.