Hardlimit test bank

[cobito]

After nearly two and a half years without an update, version 1.3 of the program is now available. The changes are exclusively internal (nothing aesthetic). There are two important changes:

· The maximum number of threads increases from 32 to 2³²-1. Let's hope that, with the war of cores, this version lasts a couple of months, even if it's . As before, it will be allowed to run twice as many threads as detected, so if you have a processor with 16 or fewer execution threads, you won't notice the change.

· During debugging tests, it was detected that in certain configurations, the execution threads are launched very unsynchronized. This falsifies the results because at the beginning of test#1, not all threads have started executing, and at the end of test#4, executions start to finish when there are still tests in progress. In extreme cases, it was detected that the test bank even throws an error because there are several seconds of lag. This seems to be a change in behavior of Windows 10 compared to what it did two and a half years ago, although it has not been possible to reproduce it on all test machines. In any case, in this version, a synchronization mechanism has been implemented that launches the test bank on all threads exactly at the same time.

As usual, if you see anything strange, don't hesitate to comment on it.

The next changes will come from the Central. First, internal changes will be made because currently a C-written engine is used that has become a bit slow with the growth of the database, that is, the response speed of the page will be improved. Then new information, new rankings, different ways of presenting the data and a page in Spanish (in the style of the museum) will be included.

[Sylver]

Excellent work @cobito. I don't know how we can thank you for everything you create for the cause and for us. I really hope that the test bank gains relevance on the network and at least your work is rewarded because you really deserve it.

For the moment I only have the main PC available and a Q6600 that is already in the database, but if everything goes well, later I will return to my previous self, surrounded by gadgets and see if I have any curious ones that I can contribute.

Greetings!

[Clipper]

now it gives different values 28865 /119342 and before 27930 /118270 and I don't think it's because of the hard drive... before an ssd and now a mechanical one.
greetings

[cobito]

@Clipper The differences are 3.3 and 0.9% respectively. They are normal and depend on background processes. The largest difference you have found (in single-threaded) can occur, for example, because during execution, another process starts to consume and the processor lowers the frequency momentarily because it is using more than one core. That difference can occur, for example, by having the task manager open even though it seems to consume little or nothing.

The code of the benchmark itself has not changed. And of course, I have used the same compiler version.

Those who had the synchronization problem (I have seen that it is not common and so far it has only happened to me on Windows 10), will see that the results are now somewhat lower.

[Fassou]

@cobito said in Hardlimit test bank:

Those who had the synchronization problem (I have seen that it is not common and so far it has only happened to me on Windows 10), will see that the results are now somewhat lower.

In order to be able to compare the records better and detect patterns, does it also store the OS version?

Do you think I will get better results with W7, or W8 than W10?

Salu2!

[cobito]

@Fassou The Windows version has been shipping since version 1.2, but I've had development so neglected that it's not even shown. It's one of the changes that will come soon in the center.

For my part, I have noticed a difference between Windows 7 and Windows 10 where Windows 7 gets better results. I've never tried it on Windows 8.

I don't know why this happens, but I can imagine it. It comes with things like Cortana, a new engine for the graphical interface, automatic update checking, a new architecture for audio, etc. They are substantial differences compared to Windows 7. From my experience, it's harder for me to have the PC completely at rest with Windows 10 than with 7.

Indeed, showing the Windows version is important.

EDITED: On some occasions I've considered launching the processes with maximum priority, but that would leave the PC locked during the test and I don't know if it makes a very good impression. Perhaps I will add an option in the future although I have my reservations, because in the end the performance is what it is with everything that the operating system throws on top of it. If Windows has some consumptions and that makes the processor go slower, it's really what there is.

[Clipper]

I think Neptune could clarify the OS thing, since we practically use the same PC and there is an interesting difference. (I assume he used W7)
and with the administrator open, in multi-thread it takes a while to use 100% and in single thread it doesn't even reach 50%.
Regards

[cobito]

The version 3.0 of the central is now available. You can find the details in the first post of this thread. First of all, I wanted to thank you for the validations you have been providing over the past two years. Thanks to them, it has been possible to extract new information of interest and it will be possible to extract extra statistics in the near future. In summary, it is now in Spanish, some aesthetic changes have been made, new information has been added to the profile of each processor, a performance comparison has been added in the submissions (every time you make a validation, you will see how your CPU performs compared to the average of that model) and other changes here and there. The most laborious part has been that the entire internal engine has been rewritten from scratch. Now the loading of the pages is much faster and, above all, it will be much easier for me to be able to create new statistics and apply other changes (until now, it was a nightmare to touch the code). Regarding the new statistics, they are still to be made. They are planned: · By architectures · By number of cores · By TDP · By performance/W · By performance per MHz · By users The search page and the comparison page have not been updated to the new version nor are zero results shown in the processor profile; it is also still to be done. If you come up with something else, you just have to say it. As usual, if you see anything strange or have any suggestions, you just have to say it. As you can see in the performance graphs, it is interesting to have several validations per mode. This way, irregularities in performance can be glimpsed. So if you are bored, do not hesitate to run the benchmark on your machines: the more modes, the better and if it is a couple of times per mode, better than better.

[Yorus]

Great job!

I am looking forward to tasting the performance statistic/W, because for me efficiency and keeping temperatures at bay in a small space is very important.

[cobito]

@Yorus Well you already have it. The performance per W has been calculated in each test individually and the general scoring. I think in this aspect, the memory performance/W is quite irrelevant for the common mortals and the general result is distorted by this. So I would only look at tests #1, 2 and 3 depending on the use that will be given to the PC.

More rankings coming soon.

[Yorus]

Thanks @cobito!. It will be very useful for future renewals in my equipment, since lately I tend to use the second-hand market more than new hardware.

[cobito]

Added individual test and performance rankings per MHz.

Also, during the debugging process, I noticed that an i7-4500U that @namiga sent a couple of months ago (in all modes as well), had been overlooked.

[cobito]

Everything has already been migrated to version 3. You can see the details in the first post of this thread. Apart from the migration, the most important change is that the "Architectures" section has been added, where a list of all the architectures for which we have results appears. In general, there have been many changes. Surely the page with the most news compared to the previous version is the one of the comparisons, where additional and better organized information comes out.

From now on, we will gradually add the little things that are missing, mainly some rankings, additional information on the different pages and some descriptions.

Another major change will be announced soon that will come hand in hand with the signatures. But for that I will open a new thread.

[whoololon]

What a great job.
Save all that with seven locks, because that database is worth its weight in gold.

It would be great if, once you enter and validate the data, the choice of style for the signature was automatic, I mean that instead of copying, opening your profile, editing it and pasting it into the signature, it would do it all at once.

Besides, if you choose the ranking, you're out of the game.

But what I'm getting at is, thanks for the hard work you're putting in.

[cobito]

Thanks @whoololon. Actually this works thanks to those who participate.

The thing about updating the signature automatically is something that gives me a bit of fear for security reasons.

Regarding the ranking, it is actually not intended as a signature but to share results around. That is, you pass the test bench, you want to show off results and instead of putting a boring link, you put a cool image.

[whoololon]

Ah, so.

Thanks for the clarification.

For the next version, in the final message when collecting data, fix "Retriving data".

[cobito]

I have a question about some results. Let's see if anyone knows why this could happen.

The other day, @krampak uploaded some results of the Ryzen 7 2700x. Specifically, I was intrigued by this one executed with 32 threads. This model has 8 cores with SMT. Looking at the results of a validation at 16 threads uploaded by the same @krampak on the same day, presumably from the same machine, the results look comparatively curious.

Leaving aside the memory tests, in the single-thread tests, the validation at 32 threads gets 1.2% more than the one at 16 threads. With that, we know that the conditions have been the same (same background loads) and we also know that both tests were run at stock frequency.

In contrast, in the multi-thread tests, the validation at 32 threads gets 5.3% more performance than the one at 16 threads. The way the benchmark works in multi-thread is very simple: it launches a process per thread, does its operations, calculates its scores and finally adds up the scores of all the processes. That is, if it got more points, it's because it was able to do more operations in the same time.

I can only think of two possibilities:

1. That when executing 32 threads, the benchmark is monopolizing a greater percentage of the CPU, taking processing time away from background programs.
2. That it is making better use of the processor's segmentation.

If I remember correctly, this behavior was also noticed by @kynes a while ago. A third possibility is a bug in the program, but after reviewing the code, I can't figure out how it could be happening because, in addition, the result of the integer test is practically the same in both cases; with a bug in the program, the behavior would have had to be reproduced in all the tests.

Anyway, I leave it there in case you feel like pondering for a while.

[whoololon]

What I'm saying is that, with all the fuss about UserBenchmark, you need a serious site to compare processors... that's it.

[cobito]

As I mentioned in the other thread, the program is signed with a CA approved by Microsoft. So from now on, this becomes something serious.

The Windows Smartscreen still appears. The theory says that the program (but especially the certificate), have to gain reputation. There are three ways to gain reputation:
· Leave the executable in a public place. Over time, it will gain points. This is done.
· Download it from different sites. The more it is downloaded and executed, the more points it will gain. I read that if it is done from Internet Explorer or Edge it will be better. But in reality I think it makes little difference. The point is simply to download and run it (no need to pass it or validate). This is where you can lend a hand.
· Leaving the executable in any folder (like the downloads folder) so that the Windows Telemetry program can see it.

In this way, in a matter of days, the warning message will disappear.

@whoololon said in Hardlimit test bench:

What I'm saying is that, with the fuss that's been made about the UserBenchmark, a serious site is needed to compare processors... that's all I'm saying.

Well yes, it may be a good time to move the matter. I have some ideas...
By the way, the typo in the text that you commented on has already been corrected.

Regarding the central, there have been a couple of minor updates. The most important ones (the details in the first post of the thread):
· Now shows OC ranking in each fiche.
· Now shows the version in Spanish if a browser in Catalan, Galician, Basque, Asturian or Occitan is detected.

[whoololon]

I've been taking a look, and I'm not clear if the results shown, both in the micro description and in the ranking table, are the best scores (micro with OC up to the hilt in a specific configuration for tests like Xevipiu), or the average of all validated results for the micro, or only those that go with the serial frequency...

Thanks in advance.