Hardlimit Museum

[vreyes1981]

Good evening, @cobito, and thank you for your quick response. Well, you see, I have all the CDs from Micromania from February 2000 until they changed their publishing house. Since you already have the 2000/2001 period uploaded, I can contribute from 2002 onwards. Many of them are already uploaded on archive.org and on AbandonSocios. The bad thing is that since I am on temporary assignment, I don't have all of them with me. If you want, I can upload to your server the ones I already have uploaded and saved in my private cloud. You tell me. Oh! And thank you very much for enabling the museum again. It's a good shot of nostalgia.
Best regards.

[cobito]

@vreyes1981 I'm glad you like the museum

I have sent you a private message.

[cobito]

After more than 3 years without news, today we bring an update to the v86 virtual machine. A lot of things have changed and the frontend had to be completely rewritten.

This new version is much faster and more stable than before, although there are still some issues to be resolved with the beta version of Windows 95.

Thanks to zstd compression, loading is now almost instantaneous and the sound has improved a lot. You can try it in the Windows 95 Emulations section.

Next stop: Windows 98 virtualized in your browser.

[cobito]

This time, the work has focused on improving the maintainability of the code. Until now, the museum's database used an RDBMS created by me based on CSV files. This was fine as a programming exercise, but doing anything was absurdly complex while with MariaDB it is ridiculously simple (it seems I had an insulting amount of free time to maintain something so infernal). Leaving aside the hyperbole, I have started to migrate everything to MariaDB (MySQL), which is considerably reducing the complexity of the code. With this "small" investment of time, I hope to be able to focus on more interesting things for the content and presentation.

You won't notice this change when it's finished, beyond the fact that the paths of the different sites are going to change (not to mention making a permanent redirect).

I had planned to put this new version into production for the software section this weekend, but more fun things have come up like assembling a cucumber. Let's see if I can do it next weekend.

Apart from this internal change (which will last for several weeks in parallel with things more visible to the public), the first virtualizations of Windows 98 have been added. In addition, the people at v86 have had the infinite kindness to prepare a patch that allows modifying the RTC date, which extends the possibilities to use beta versions with "time bombs". In any case, there are still a few things to solve with the beta versions of both Windows 95 and 98.

[cobito]

The software and hardware section has been migrated to MariaDB and put into production. The process has been faster than initially planned. With this, we jump to version 3 although the library is still missing.
For the library, there are big plans that will be executed when the second phase of hardware renovation is completed. So for now, the museum is slowing down.

Apart from the internal restructuring of the code and the database, it has also been decided to abandon the use of JS-DOS for emulation and everything will be centralized in v86. This is done because the people behind JS-DOS are starting to take a commercial approach to their project that forces, under certain circumstances, to use their cloud services, which look very good, but go against having a completely independent service from third parties, that requires little maintenance and works in all circumstances even if they decide to abandon their project. In addition, I have seen that v86 performs much better than JS-DOS, so the decision was clear. The only drawback is that doing certain things that were simple with JS-DOS, will have to be figured out with v86, but that's part of the fun. With this, there are now emulations (like those of Windows 3.x) that are not working, but they will be coming.

So, leaving the museum running at a slower pace (not stopped), I want to focus on starting to get the most out of the new machine that helps to power Hardlimit since this weekend.

[cobito]

All final executable versions in v86 of MS-DOS and a good number of Windows 95 betas and versions are now available.

In addition, thumbnails have been added to each version.

[vreyes1981]

Good afternoon. I have noticed that in the "library" section, both in the "files" and "magazines" sections, the links are broken. Is this due to the migration to the new server?

Thank you very much.

[cobito]

Good @vreyes1981

It's because of the migration to version 3 of the museum. The library will be out of service until it migrates to the new version, hopefully before Christmas when, in addition, the Micromanía discs you sent me will be available.

[vreyes1981]

Vale @cobito, no pasa nada, creí que había pasado como la última, que te hackearon la página.

¡Muchas gracias!

[cobito]

We are recovering old spaces: The magazine library is back online (NOTE: Publications and issues are still being added; it is an automated process that will be completed in a few days).

With the magazine library, we started making functional and visual changes:

• The loading of magazine pages has been improved to the point that they are practically instantaneous.
• Thanks to the loading speed, it has been possible to create a new simple, fast, and much more practical magazine viewer than the previous one:
  · Now the entire magazine is displayed on a single page, avoiding the need to turn pages.
  · Pages are displayed in pairs at full width of the browser window.
  · It is still possible to zoom in to full screen.
• A new format has also been applied in the presentation of magazine lists, years, and issues with wider tables with screens starting at 1080p in mind and mobile devices.

At the moment, the new file browser where the CD-ROMs will go is being projected. I have a sketch although I still have to make several decisions before starting it. I want to try to make this the definitive one.

On another note, a new universal header has been launched, which changes buttons for tabs and includes the turn menu. That is, the tabs will be the same for all pages but the bottom menu will change depending on the needs of each space. At the moment, this new header is only in the museum but will spread to all pages over time.

[vreyes1981]

Olé!! little by little, si señor.

And I like the new design, with the magazine covers really big. What a blast of nostalgia.

[cobito]

The first feature of the year arrives and it's for the museum. We're launching a new disc browser that has become the germ for the future file browser that will be an extension of this. The new front-end has been used, so the views are optimized for both large screens and mobile devices. As it is thought, it should be possible to navigate in a comfortable and intuitive way all kinds of media: in addition to magazine discs, also operating systems, drivers, demos, compilers, etc. The content will be arriving. The entire table structure is already created and in the next iteration, the concept will be put into operation with a couple of discs or three. Due to space limitations, for the moment only the PCManía CDROMs are available. I will look for temporary solutions. I have a 2TB disc around that I might be able to use as a temporary medium. For its part, the disc card (medio.php) of which you have an example here, shows the cover along with information about the medium. The information that is shown (if available) is as follows:

• Name of the publication.
• Size of the medium.
• Format of the medium.
• System with which it was originally created.
• Volume
• Editorial
• Entity that prepared the data
• Program that was originally used for creation
• MD5 sum
• SHA-256 sum

Download links are also shown for the file (.iso,.zip) or files (.cue/.bin) of the medium that you want to download and the downloads are enabled, in principle, without limits. What comes next will depend mainly on the storage space that I manage to rescue from discs that I have stored around.

[_Neptunno_]

@cobito I love the hard work you're putting into the Museum and the new interface for the disk browser. As a professional in the digital preservation sector, I can only take my hat off to you.

Let me tell you something, because you'll like it: my company is dedicated precisely to digital preservation on a global scale. We work with clients ranging from the National Library of Spain (BNE) to international ones like Harvard University, the Holocaust Museum in the US, or HILA (Stanford University), among many other top museums and universities, setting up systems that cost hundreds of thousands of euros. And I'll tell you something: seeing what you're achieving with limited resources has an impressive value; the structure with the MD5, SHA-256 sums and the metadata cataloging is at a level of rigor that many institutions would envy.

That's why I'm so excited about it. From my humble opinion (keep in mind that I'm just a technician, but after so many projects, and especially the one at the BNE where I saw how millions of pages were published thanks in part to my work over the years for people to access them from home), your work seems to me professional and necessary. I know how difficult it is to digitize and give visibility to these archives, and yours is something to take your hat off to.

It's vital to give visibility and access to this content before it's lost forever. In fact, if you want, I'll talk to my company so that you can negotiate with them for a safebox and we'll take the Museum to the big leagues jajaja

Seriously, you're doing great work and the community will always be grateful. As soon as I get my hands on the 486 or the Pentium 166 I have around here, I'm sure I'll be using this material to the fullest!

Hugs!!

[cobito]

@_Neptunno_ You're going to make me red

You have no idea how happy and motivated you make me with what you say. In the technical aspect of the subject, I don't have as many doubts. It's the typical thing that can be done well in many ways and I probably am not doing it perfectly, but not badly either. But from the point of view of structuring and organization, I follow my intuition more than "technical" standards. My references are Archive.org and WinWorldPC. And well, the experience of this being the third time I try (let's see if the third time is the charm).

@_Neptunno_ said in Hardlimit Museum:

We work with clients ranging from the National Library of Spain(BNE) to others internationally such as Harvard University, the Holocaust Museum in the USA or HILA (Stanford University) among many other top museums and universities, setting up systems that cost hundreds of thousands of euros.

Having someone with this track record come and tell you that you're not on the wrong track is very motivating. As you are probably the biggest expert in Hardlimit (certainly, you are the biggest expert in the field that I know), any criticism or suggestion you have, whatever it is, please comment on it.

This is something I want to do well and since the economic cost (not in time) of development is zero and that the hardware resources can be scavenged around, it would be great if the formal aspect was done with rigor.

It's still very green, both in functionality and content. So as it evolves, you will probably see things that can be improved (if you haven't already seen them).

Thank you!

[_Neptunno_]

@cobito I'll do everything I can to help you, as you know it's a pleasure, even if you don't think I'm at the level of a preservation engineer!

I think you would enjoy learning a lot from my development colleagues; to program the software that manages not only Teras, but petabytes of information, there is an incredible amount of work behind it (not to mention a thousand things that were not known, such as the "Transfer Connector": a critical function that acts as a bridge for ingesting data in a secure and structured way). In that part, I have less to say, since my role is more of a system support, but I have also been in many digitization projects of all sizes and I can't help but see your museum with professional eyes. I find it admirable and, above all, very useful for the community!!

To give you an idea of the scale, the systems we set up are responsible for preserving "digital knowledge" for the next 200 years (or so I heard in a meeting a while ago, haha). We always say that if this had existed in the time of the Library of Alexandria, nothing would have been lost today. The data is stored in redundant systems that constantly audit the files to ensure they are healthy and that the disks maintain their integrity. If something fails, there are several more copies in the pools ready to come in while the damaged disks are replaced or "with a lot of running around".

Even in the face of catastrophes or wars, measures are taken. With the conflict in Ukraine, for example, critical copies were already being moved to safe locations (For example from UK to Ireland) to prevent the information from being lost in case of direct confrontation.

That being said, I must point out that I am only the "last guy" in my company! But these things are cool and I tell you about them as a bit of gossip from the world

Best regards!!

[cobito]

@_Neptunno_ What a find. A while ago I read about a new Rosetta stone (in the sense that it was recently created) where information was stored in a spiral reducing the size of the characters from visible to the naked eye to microscopic with a similar goal to the original Rosetta stone (to have a translation table). And then, to reproduce it and distribute it all over the world, as a way of having that redundancy.

After some searching, I found it. It's called The Rosetta Project and what they have done is create a disk that stores 13000 pages of information in 1500 languages in a way that the text can be read with a microscope (no digital information).

You probably know about it, because looking at the page, it's run by Stanford University which your company has worked with. You may have even participated in some related development!

Anyway, what I meant to say was that this was the only example I knew of an attempt to preserve knowledge on that scale. I would never have imagined that there were so many resources put into preserving knowledge at the level you describe.

Very interesting, indeed.

[_Neptunno_]

@cobito the most important part of my company is preservation, which is where the most resources are invested and is the "soul" of the company. Then there is the digitization department, which was the one that started everything.

At first we were fully involved in massive digitization, something quite "mechanical" (scanning material and generating metadata and bibliographic records of the images). We worked with specific scanners to obtain TIF files (300dpi although over time they went up to 400 and 600 with very specific cameras for this) and then generate the derivatives (JPG, PDF). Depending on the project, some were simple and others required massive renaming and complex structuring, such as periodicals (newspapers and magazines). To give you an idea, we have digitized everything from the AS newspaper to press from the beginning of the 20th century for the BNE, passing through projects with the Prado Museum or the University of Granada (some project of the Royal Hospital of Granada), among others.
But minor projects were also done for Town Halls of Villages and Urban Planning files, to put another example. But we are going to show you that it has been many years of doing everything.

My job for a long time was precisely that image processing, although I also did system support. In parallel, the company grew exponentially developing digital preservation software. That software is the one that manages everything now. We no longer just digitize documents, but we guarantee that they remain intact and accessible.

If you go to the BNE Digital Library, there you can see millions of pages in which I have contributed in a small way with my work. As a curious anecdote: I worked with collections of photos from the Civil War that were not open to the public due to copyright, but that the BNE had to preserve to release them in the future. Seeing those images was quite shocking

I'm sorry to disappoint you, but that project of the Rosetta Project at Stanford is something they develop on their own. That said, as a client we work with their infrastructure and, well... you would laugh if I told you that we have had to complain about being so stingy. They gave us virtual machines with system disks of 127GB (the default size of Hyper-V) and we had to protest so that they put in "more juice", because they were constantly filled with logs

Best regards!!

[cobito]

I have finally been able to organize all the museum content. I rescued a 2TB drive from the storage room that will give me plenty of room to continue with this. In addition, I have been able to make space on the backup drive, so the part that bothered me the most about using a drive in this way is resolved. At first, all the magazine discs are already there. @vreyes1981, your Micromanía discs are there, specifically from the year 2002 (full year) and 2003 (until April), which is what you had uploaded. If you have more material around, don't hesitate to put it (let me know if you do so I can add it). The database structure has also been finalized to take the next step: the file explorer. The initial phase will allow you to navigate the directory tree of all available drives and in the next turn of the Museum, it should be operational. I will see if I dedicate time to the test bench or the museum next.

[cobito]

This week we launched the first phase of the file browser. All files from almost all disks are already indexed and browsable. We left out a couple of MDF/MDS from Micromanía that we can't extract despite being able to extract most MDF/MDS. Here you have a disk where, in addition to the information that was already announced, now includes the list of files and folders browsable. In addition to the file explorer, it is possible to view detailed information of each file along with duplications, that is, files that could have different names, dates, and other attributes but whose content is the same. The information that is shown is as follows:

• Name of the file in the current medium and route.
• Original creation date in the medium.
• Size in ISO/IEC 80000-13 binary format.
• The type of content in a descriptive way (from here for most it will not show information; it will be added over time).
• MIME type (it is not 100% precise, but almost).
• A more detailed description of the content of the file, for example, if it is a self-extracting executable, details of the encapsulated content are given.
• An MD5 signature
• An SHA256 signature

Here an example of pkunzip.exe which is a pretty popular file. In this first phase, it has been a challenge to be able to determine the character set of the file system. Sometimes UTF-8 is used, other times CP850 and there are a couple of images out there that look like they came out wrong from the beginning due to some failure in the creation software (apparently, it was not uncommon in the 90s). In any case, the names of the files are shown correctly with their es, accents, and other regardless of the original format. We have 570,000 files. This comes from the media of the five publications that we have at the moment, which are being used as a reference for all development before adding much more. The second phase of the browser has also begun, which consists of extracting all extractable files. And of the extracted ones, repeat the operation recursively. At the moment we are compatible with more than 70 compressed formats from all eras that are determined by heuristics, not by extension or by magic number, which avoids that any compatible file escapes. About this, when this first version of the extractor is polished, it will be put into production. Over time, more formats will be added but for now we stay with those 70-80 to prioritize other aspects. In another order of things, a change has been made in the v86 configuration that makes the response of the virtual machines now much faster (example). It was something that I didn't notice by going locally, but now that all traffic goes through a VPS, I realize details of performance that can be improved (dependent on latency). And to finish, the algorithm behind the new page translation system has been corrected making pages render much faster now (this affects both the museum and the test bank). This change, in addition, has stopped breaking certain functions such as the magnifying glass in the magazines and the capture of the mouse pointer in the virtualizations.

[cobito]

It is now possible to view the contents of the extractable files. These include any type of file and the content can be other files or sections of binaries. The sections often contain binary data without importance, but in others, there is embedded relevant content such as images, sounds, animations, cursors, plain text and so on (mainly in DLLs). Here is an example of a.zip that in turn contains another zip. In the description you can see the format and the algorithm that is used in each of them. 70% has been indexed. I suppose that during the weekend the process will be completed. At the moment, they add up to 1.3 million files/sections to the ones we already had. With this, phase 2 is in principle finalized (waiting to complete the indexing, which is an automatic process) and the most arid part of the topic is closed until new formats are added (there is already a list for the next iteration). Possibly phase 3 will begin soon, which consists of being able to visualize the files from the browser. This is: images, videos, sounds, midis, mods, documents, etc, etc, etc. It is one of the coolest parts of the explorer for which it has been necessary to previously do what has been done so far and with which it can be converted into a powerful tool for digital archaeology. On the other hand, the front-end has been consolidated throughout the museum (except for hardware cards): now the new one is already being used in all sections, which gives it a better appearance on the desktop and fixes many things that were broken in the mobile version. In addition, videos have been added about each hardware and software that I uploaded at the time to Peertube: example.