Hardlimit test bank

[cobito]

@krampak a few days ago I was reading about this and the problem is widespread among small software development companies. I will study the possibility of signing the program that they say improves the situation.

I will also see if the limit of 0.97 is too strict.

[cobito]

It seems that the program as a whole is detected as a trojan in some antivirus. But the main problem is with Windows Defender. @krampak I am going to report the false positives to Microsoft but they are asking for the "detection name" that is found in the Windows Defender history. If you have it, can you pass it on to me?

I am going to install Windows 10 so that I can do tests on my own.

[cobito]

The new Hardlimit signatures are now available, where individual results of each test have been removed and only the totals have been left. In addition, additional hardware information as well as the ranking position is shown.

If you finish first, second or third in single and/or multi-threaded, a gold, silver or bronze medal will appear next to the ranking.

[cobito]

I've been investigating the issue with antivirus software. On virustotal, sometimes there are more detections and sometimes fewer (the day before yesterday there were 8 and today 11).

I've tested Windows 10 and indeed the "Antimalware Service Executable" process belonging to Windows Defender is a headache. However, Windows Defender itself finds all the test suite components as harmless. I've read that it's normal for that process to consume a lot of CPU normally, running any program so I can't do much about it.

I imagine that this problem can be reproduced with other test suites. The issue is that in HLBM there are "too many" mechanisms to detect if the test has been performed under adequate conditions, which other programs in the field don't have and that's why they apparently don't have a problem.

I've also checked that when the test suite is run for the second time, the malware detector is much less aggressive.

The reason its activity is seen in test 4 is not because of test 4 itself, but because after running that test, "hlbm-detect" is launched, which is meant to detect the system's hardware. It's a very different process from "hlbm-core" and that's probably why it wants to analyze it. However, this wouldn't affect the results since at that point, the test suite itself has finished.

I understand that in Windows 10 it's going to be difficult to reach the 0.97 (0.5) MMT ratio so in the next revision of the program, the Windows version will be added to the rest of the information sent. In case it's Windows 10, the requirements to consider a result valid will be lowered.

Until then I can't do anything more. It's a problem of Windows 10 and its implementation of Windows defender, not of HLBM or a false positive.

[krampak]

Perfect @cobito, good job!

[cobito]

Version 2.0 of the hub is now available. The changes are mainly aesthetic. The goal has been to make it look as much like the forum layout as possible. It is now also compatible with mobile devices.

In terms of functionality, the search fields now suggest processor models. Only 5 suggestions are shown so you won't see anything until the search is narrowed down to that.

Finally, a brief description is shown on the model tab. The description is still in an early stage of development.

Home

Example CPU tab

Example comparison

Example result

Once again, there have been a lot of internal changes, so you know where to look for any bugs.

[Namiga]

I like it much more now.
Thank you very much.

[Yorus]

It looks good, more integrated with the rest of the web. Let's see if I update my results with the new "cucumber" from the house.

[Fassou]

@cobito said in Hardlimit test bank:

Once again, there have been a lot of internal changes, so for any failure you already know.

At https://bm.hardlimit.com/cpu.php?cpumodel=Intel(R)-Core(TM)-i7-5500U-CPU-@-2.40GHz-198356 you get a micro like "error op t3= 0. Please, report"

I came by one of the links you posted, but I can't remember which one

[cobito]

@Namiga Thanks. Those comments help to continue...

@yorus Well, now the important thing is to collect as many results as possible. The next goal will be to give compatibility to old Windows and if I'm lucky, even to make a live-usb that runs all modes automatically without having to boot the PC's operating system.

@Fassou Look, I had it noted down from the beginning. It's already been fixed.

[krampak]

@cobito Do you have any idea why I can't pass the test1 within the ratio for the i7-6700 in mode 1 and 2? I have repeated other modes and they come out well, but I have never been able to validate modes 1 and 2 to 0.50 as far as I can see. I will try again later.

[kynes]

@krampak I don't know if this will happen, on some micros that have a higher turbo for 1-threaded applications it was impossible for me to validate it, it always stayed below the minimum.

[krampak]

@kynes said in Hardlimit test bench:

@krampak I don't know if this will happen, on some micros that have a higher turbo for 1-threaded applications it was impossible for me to validate it, it always stayed below the minimum.

It could, but it is still curious that it is only using those 2 modes, what you describe should happen in all modes.

[kynes]

@krampak right, I hadn't read you properly. I thought it was in all modes.

[cobito]

@krampak Well, that seems very strange indeed. I'll have to review the mmt ratio restrictions soon because as @kynes says, there are cases where it's normal for it to come out below 0.97/0.5. The only reason that comes to mind (a bit far-fetched) is that in those modes it gets hotter than usual in test#4 and stops the car a bit on the first peak of test#1. But looking at the performance graph, that doesn't seem to be the case.

If we look at the coincidences, it could be that some program in the background coincidentally did something during that execution. By the way, in a few weeks there will be a live-usb of the test bench to avoid interference from other programs and make it agnostic to the installed operating system.

If you repeat it and it goes wrong again, put the link to the result here.

[krampak]

Yes I have tried it again, same thing: https://bm.hardlimit.com/result.php?bm=70620578ee58b11e46bcbfbfbec8104e588

I will try with the live USB when it is available.

[kynes]

@krampak Have you tried with 4 threads instead of 8?

Mesplico, let's see if there is a program problem with the hyperthreading implementation of those micros, and with four physical cores it's not a problem. The ideal thing would be to be able to disable it in the bios, to see if it gives a coherent result without those threads.

[krampak]

@cobito I'm testing it at the Lan Party and I'm not sure if it's the PC or the server, but this is what happens when I click on validate results:

https://imgur.com/a/Q6vEI

[cobito]

@krampak ok, that's a bug from the change to the new version. It comes out because you're using an old version of bm. Download 1.1 and it will work.

[krampak]

I don't know how I didn't notice, it worked later @cobito.
By the way, how would you see creating a test bench page that showed all the CPUs sorted by performance (taking the best result of each CPU)?

PD: In About you have a mistake, it says: About Hardlimit Benchmack