Information & announcements

[cobito]

Well, gentlemen, today marks the end of the quarter from an administrative standpoint. A lot has been done, but just as much has been left undone. What remains:

• Museum disc/file section (@vreyes1981 I haven't forgotten about this or your Micromanía discs). I tried to get it more or less done before Christmas, but I didn't have the time. This will be a high priority when tasks resume. Unfortunately, the lack of disk space (and the current situation in the hardware market) means that for now we will maintain the previous functionality even though the content organization has been completely restructured (this is almost finished but I want to finalize a few things before making it public). The idea is to have everything ready for when buying a new NAS is more viable than it is now and the deployment will be quick.
• Forum translation: the process has been running internally for a few days. This has been a bigger challenge than translating subtitles because the forum text is formatted in HTML and although the instructions to the model are clear and concise, sometimes it gets some HTML tags wrong (something that shouldn't be complicated to correct programmatically). Also (and this mistake probably occurred in the subtitles), it seems that the model wasn't being entirely clear about the task it had to do and the questions, it didn't always translate them literally, but rather translated the answer to the questions. If in any subtitle you see something like the definition of "free software" when the speaker said "but what is free software?", it's because of this lack of specificity on my part. The implementation on how to display the translations is outlined and is functional at an early testing level. Here, ChatGPT has messed things up for me several times and made me follow paths that led nowhere (I lost a couple of days doing nothing). The fault lies largely (apart from mine for delegating things I shouldn't have), in the poor documentation of NodeBB in some aspects although through much testing, I have managed to get a rough sketch that looks pretty good.

These are the two most priority things. Then there is an endless list of things that will come after and with which I won't bore you.

Development will resume in mid-January.

As every year, I will try to make the thread summarizing this year where the things that were done will come out.

[vreyes1981]

Hello @cobito, I just read your message.

¡¡No paasaa naa!! When it's possible, it's possible. This is done as a hobby and we're not obligated or paid for it, so don't worry, don't suffer. The important thing is that the saved material is in good hands.

That said, if we don't see each other these days, I wish you a merry Christmas.

[cobito]

This summer, a couple of Peertube users complained that they couldn't stream live because their connection was very slow. One was from Germany and the other from Italy. It seemed strange to me because the connection is symmetric with hundreds of Mbps. After some research, I discovered that in certain cases, residential connections often don't have good peering with other residential connections. This can happen because the routes aren't optimal or because traffic for higher-tier services is prioritized. I ran some tests and, indeed, I could see that during peak hours (mainly after 8 PM), the bandwidth with one of the affected ISPs from here didn't even reach 1 Mbps.

With this situation in mind, I've realized that a residential connection isn't a good option for running the services we host here. Especially when there are strict bandwidth requirements.

At one point, I tested Cloudflare's Tunnel service. It's very easy to use, but there are limitations regarding the protocols it supports (among those it doesn't support is RTMP, which is used for live streams). Additionally, the terms of service don't clearly specify traffic limits, beyond requiring responsible use.

So, I looked for options that are more complex to set up but much more versatile, and I found OVHCloud's VPS. I've been running some tests, and I think we'll go with that option: they installed Debian (the distro used by the rest of the machines), it has plenty of processing power, and a sufficient amount of RAM and storage. Furthermore, they clearly state that the bandwidth is 400 Mbps and traffic is unlimited. All of this should be sufficient for now.

The idea is that the Hardlimit data center will remain where it is, but all traffic will route through OVH (a server in France, which was one of the few available when signing up for the service). With that, I hope the issues related to poor routing/peering will disappear. Additionally, it avoids exposing the public IP of my personal connection in the way it's currently exposed. Honestly, I'm surprised my provider hasn't given me a warning after the several DoS attacks we've received.

With this change, the domains will also be moved to a different registrar. We're currently using no-ip with their Enhanced DNS. Since using the VPS will make dynamic DNS unnecessary, it no longer makes sense to keep the domains with no-ip, so they will also be transferred to OVH.

For now, the VPS will function exclusively as a reverse proxy, though it's possible that some Peertube processing load will eventually be moved there, since CPU power turns out to be very cheap, while storage, RAM, and GPUs have astronomical prices. So, the heavy lifting will still remain here.

I'm sharing this purely for informational purposes. The transition will take place over the coming weeks and there will likely be some service interruptions. Beyond that, the change should be transparent to you.

[_Neptunno_]

@cobito I absolutely loved this post! Mainly because, to be honest, if I had read this a couple of years ago, I would have been completely lost in the technical parts. That said, I'm not an expert now either, but over time at work I've been picking things up, doing my own research (and chatting with DevOps coworkers who are real experts), and now everything makes much more sense to me.

Looking at your network diagram and how you've set everything up, I'm really impressed by the hard work behind it. Comparing it to my little home lab (where I've learned what I can by trial and error, setting up a reverse proxy with NPM and managing my dynamic IP with DuckDNS), I realize that even though my setup has plenty of room for improvement, it's still cool to see it working. But yours is on another level. Hats off to you for your work and the effort it takes to maintain the forum and all its services... what a massive undertaking!!

Actually, at first I didn't fully understand why you ruled out Cloudflare (we use it a lot at my company, though I'm not directly involved in managing it). But after giving it some thought, I realized it's simply not viable for your case for two major reasons you've probably already considered:

• The RTMP protocol: Cloudflare (in its free tier) is optimized for web traffic. For PeerTube live streams, you need RTMP, and Cloudflare doesn't support it on their free proxy; you'd have to go for Enterprise solutions that cost a fortune.

• Streaming limitations: In their terms of service (policy 2.8), they make it clear that you can't serve large amounts of video without a specific plan. The moment they detected you were moving terabytes of video through PeerTube, they'd cut it off quickly.

Given that, I completely understand why you went with OVH, which we also work with. We have some Proxmox nodes hosted there, and even several MooseFS clusters with their chunks (a real beast that stores hundreds of disks with petabytes of data). I'm sharing this because I enjoy talking about the things I'm discovering; the world of systems and storage is crazy. By the way, we also have some servers on Voxility. I don't remember certain configurations right now, but we have several nodes in the USA, Canada, Europe (UK, Germany, and another place I can't recall), and Singapore.
Anyway, we're moving a lot of systems to AWS, but honestly... the DevOps team knows that part better than my systems department, lol.

Anyway, sorry for the "wall of text," and I'm sure I'm not telling you anything you don't already know, lol, but I really wanted to share that feeling of being able to understand such a technical post and realizing I'm learning a little bit more every day.
Honestly, I'm super burned out at work because of other things, but at least I'm picking up some knowledge and having a bit of "fun"

Cheers!!

P.S.: If I drop any "technical nonsense," please don't hold it against me!!

[cobito]

Thanks @_Neptunno_

I've always been drawn to the idea of having my own homelab... Since my first 56kbps connection back around 2000, I was already tinkering with Counter-Strike servers, though with that bandwidth, it only allowed me (locally) and one other player to join.

Much later (actually just a few years after), when ADSL speeds began to double, Jazztel implemented Fastpath on their connections up to 20MBps (with uploads close to 1Mbps), I allowed myself to set up a Counter-Strike Source server for 16 players. And it was quite a success!

After that, I started hosting my personal blog on a Raspberry Pi. That's when I began learning Linux at an administrative level. And when I took over Hardlimit (which was still running on vBulletin), I got my hands on a Netbook (Atom N270 with 1GB of RAM) and it ran like a dream. Those were simpler times when only PHP and MySQL were the standard

Little by little, I've made things more complicated for myself until reaching what I have now. And the truth is, by doing so, you learn a ton about everything; for example, until 6 months ago, I had no idea what a peering agreement between providers was, and I lived in the innocent belief that my connection was (in relative Mbps terms) equivalent to a data center's

Continuing on the topic of VPS, CDNs, and the like, the problem with Cloudflare is that even with a paid (basic) plan, they don't clearly state the bandwidth or traffic limitations. And, indeed, Peertube moves quite a few terabytes a month, which leaves me with doubts and uncertainty.

I prefer the VPS approach, because in the end, I manage it like any other machine, and in OVH's case, they make all features and limitations completely clear. Plus, it has a very good price, all things considered.

And going back to the migration, last night I ran some initial tests setting up a basic NAT. This morning I spent a few hours trying to get Nginx running from the VPS, but there are many things that become incredibly complicated, mainly with Peertube and Mastodon since, among other issues, I need access to a mount point with the data, and in the end, it results in a pretty messy configuration that's hard to maintain and inefficient. And not only that, but Mastodon, by design, doesn't allow a reverse proxy outside of localhost, so on top of that, I'd have to modify the platform's internal code. Basically, it's a complete hassle and, in reality, the benefits are minimal.

So ultimately, I'm just going to stick with a simple NAT.

From now on, all traffic is routing through OVH. Those of you connecting from Spain won't notice any difference. People coming from other countries and continents, however, will see a significantly higher service quality.

Regarding the domains, the folks at No-IP are a bit of a pain; instead of automatically generating a migration key for you (like normal registrars do), you have to open a ticket and wait for them to reply with the key. So I guess this will happen next week.

[cobito]

I thought this was under control, but it wasn't (that’s what you get for doing things for the first time). There have been a lot of outages this weekend; my apologies for not planning this properly or giving timely notice. The issue is, in principle, resolved, and from now on, things should run smoothly without any further interruptions.

In short, OVH has an anti-DDoS system (which cannot be configured or disabled) that treats UDP packets as inherently suspicious. When a certain threshold is crossed (I’m not sure if it’s based on data rate or packet rate), the system blocks the source IP. Since my server was using UDP to connect to the VPS, any slight increase in normal activity would trigger it and take down all services. I """simply""" had to switch to using the TCP protocol.

It’s the price of learning (half a weekend, in this case).

[cobito]

On the 14th or 21st of this month, Hardlimit will be down for maintenance. Several tasks will be carried out:

• Internal cleaning of all equipment.
• Peertube will be updated to version 8.1.
• Minor hardware changes will be made to the NAS: replacement of the case, PSU, SATA controller, and fan. This change is primarily for thermal and noise reasons. We have chosen an Inter-Tech IM-2 Expander case, a 550W Pure Power 13 M PSU, and a Pure Wings 3 120mm fan. Since the board has a soldered CPU and passive cooling, I expect the only audible noise source to be the drives.
• The Redis server will be decommissioned and merged back into the frontend server, but with a dedicated disk just for it. This is done for power savings and noise reduction. If this change impacts performance, a solution will be proposed after summer. It's true that the 32GB of RAM will be a bit tight, but as long as the Ryzen 9 doesn't let us down, we'll be fine.
• The Peertube metadata disk is starting to show SMART warnings, so the content will be moved to the Mastodon disk (we've noticed both have less activity than expected, so there shouldn't be any performance impact). It's not critical since, in addition to having backups, it's information that can be regenerated, but the change will be made anyway.

On another note, the end of the quarter is approaching. The first quarter of the year was overly focused on the museum due to the persistent issue I had with the file browser. Fortunately for the museum (and the swamps), a rainy winter has given us more hours than initially planned, and it will be practically wrapped up, just needing to tie up a few loose ends.

As a result, the second quarter of the year will be the test bench quarter.

There are several planned changes at the program level, to the point where version 2.0 will be released. The idea is to keep the current metrics intact for backward compatibility with results and to add a new test.

New features will also appear in the program, both suggested by you and developed in-house. The plan is to release beta versions that will allow result validation, with these new features being added incrementally. Once the version is finalized, it will be signed to avoid Windows warnings.

Details will be revealed release by release.

Finally, minor changes are expected for the museum (lowest priority), along with some forum updates related to integration with other platforms, mainly Peertube, as well as promotion of its English version (which, by the way, has received minor fixes).

[cobito]

PeerTube has been updated to version 8.1.3 today. Additionally, NodeBB was updated to version 4.9.1 earlier this week. With this latest version, we've been experimenting with forum federation, but it's quite fragile, so for now we're sticking with the current setup.

Regarding the museum, you can now download files individually. We've added some Windows 95 and 98 CD-ROMs to expand our media library, and several improvements and fixes have been applied to the search and file explorer. Additionally, the indexing of extractable files is now complete. We have nearly 4.5 million files in total, of which just over half a million have been processed to extract their media (images, videos, MIDI, etc.). This means all the magazine CD-ROM files have already been processed, leaving only the extracted files and the newly added media (Windows 95/98). Going forward, development on the museum will take a back seat and will only receive minor updates and new content.

Regarding the network infrastructure, OVH has completely messed up with billing issues, and we've been without a VPS for the past two or three days (despite having paid until January 2027). I have a lot to say about this, but I'll keep it brief: I've never seen such brutally incompetent technical support, and I have zero hesitation in saying so because the negligence has been nothing short of epic. As soon as I get the chance (whether they refund me for the unused months or the contracted services simply expire), I will be leaving. So for now, and until they sort out this mess, all traffic is routing directly through my ISP.

[cobito]

We’re starting the spring quarter, and with it comes our first screw-up and first major task:

Yesterday I was dealing with some of the issues I’ll discuss below, and in the process, the forum database got corrupted. I had to fall back on a backup, and (here’s the screw-up), the most recent one was from March 18 (three weeks ago). Backups are supposed to run weekly, but between updates and time off, I missed the Redis one. As a crutch for my limited cognitive capacity, the forum actually runs on two different databases across two different DBMSs. That allowed me to recover the lost posts, but their timestamps are set to today. Furthermore, over the past three weeks, users registered to validate results in the testbed, and that data is unrecoverable.

And why was I tinkering with things yesterday? Well, because a bunch of scrapers felt the need to rip off all the content from the Peertube instance from hundreds of IP blocks originating mainly from Asia. We’ve finally been able to largely mitigate the attack, but due to the sheer volume of IPs, some requests are still slipping through. That’s causing everything to run slower than usual, particularly in the museum and the testbed. We’ll just have to hang in there for a bit until all the malicious IPs are banned (it’s a slow but automated process).

Right now, Hardlimit is under its most intense attack to date and will experience periodic outages.

In short, the server has been targeted by various attack attempts, not just Scraping/Crawling/Flood/DDoS, but also attempts to exploit zero-day vulnerabilities, privilege escalation, and the like (platforms like WordPress tend to be a sieve). However, these leeches are so aggressive that most attacks (though not all) are detected immediately. So far, they haven’t achieved anything (beyond taking down services by crashing the server) thanks to the web layer being properly isolated from the data layer. Still, monitoring the situation and applying fixes has been consuming way too much time. I hope that after implementing these latest measures, I can finally focus on more interesting things.

Essentially, IPs are being banned en masse, and I’ve been tweaking kernel parameters to avoid hitting swap space as much as possible. Right now, ZRAM is saving us from a much worse situation, and Crowdsec is saving our ass from leaving the server completely unusable.

On this front, there’s only so much we can do. We’re a small fish, the ocean is too vast, and AI has given sharp teeth even to the dumbest fish. These days, keeping internet-exposed machines secure is a real challenge. But for now, we’re holding our ground.

Setting aside the parable, the drama, and the whining, as I mentioned, I’m going to focus on the testbed, and more specifically, on the software itself. There’s a somewhat defined roadmap (more vague than defined) that, if completed, would take us to the second major version.

It’s been a long time since I last worked on it. The first step will be to dust off the development environment, which lives on a virtual machine, review the code, etc.

The first significant change I want to make is migrating from a multiprocess model to a multithreaded model. The reason I want to make this rather boring change is to simplify the testbed startup: currently, process synchronization is relatively fragile (in certain environments, the bm tends to fail) and delays the start of the tests. Additionally, this change will allow us to improve safeguards against result manipulation.

Overall, this will result in a more robust program, simpler internally (hopefully), and with identical behavior regarding the reproducibility of results across previous versions.

I’ll likely start on this next week and probably dedicate the entire week just to administrative tasks: reviewing the VM, getting familiar with the structure, code, etc.

That said, with each new feature (such as implementing the threading model), a beta version will be released. As was the case with the first version, anyone who participates and leaves feedback will be listed in the program’s acknowledgments window (if they aren’t already) and will have the option to change their name if they wish. Those who don’t participate but are already listed will remain there.

The goal is to release a new version roughly every two weeks, though this will depend heavily on various factors.

In parallel, I’ll be making minor tweaks to the museum, the translation system for both the forum and subtitles, and other small issues. And yes, damn cybersecurity will continue to hog my time.

[cobito]

A brief update:

The attack continues in bursts. It causes average loads of >150pu, bandwidth of >450Mbps, and hundreds of ffmpeg processes to process the requests. Disk I/O has been successfully reduced to a negligible amount, so despite the ridiculous numbers, the various configurations that have been applied, along with the fact that those requests are given minimum priority, allow the server to hold up and maintain a very good response speed during these bursts, which can last several hours.

We're in good shape on this front.

Regarding the testbed: migration to the threading model has begun. Test #1 and Test #2 have been completed. #3 (the most complex) and #4 remain. Additionally, in this batch of changes, all the hardware detection code and related components will be migrated to the program's new architecture. This means it's quite likely that the first beta will be released, at the earliest, in two weeks.

Once the program is in good shape, the rest of the changes should start arriving more quickly. Basically, the strategy is the same as with the museum, where it took a while for major changes to roll out, but once everything was in place, implementing improvements became much easier.

To wrap up: this weekend, a new controller was installed in the NAS. This saves me an external drive (which is now internal) and has allowed me to add individual activity LEDs, so I can check the load status at a glance. Beyond that, this change doesn't affect anything else.

[cobito]

Testing

• The migration to the threading model has been a failure. Tests #1, #2, and #4 are running fine, but #3 (compression) exhibits anomalous behavior in multithreaded mode (yielding results far below expectations). I've tried a wide range of approaches, but nothing works. I'm going to stop dedicating time to this process for now (though I might revisit it later).

• I'm focusing on tidying up the code by separating the main components to decouple them (currently, everything depends on everything else), but this has to be done with extreme caution to avoid altering the behavior. It's a slow process that won't introduce new features, but it's essential for maintainability. This means that, even though I'm not posting updates on this front, a significant number of hours are being (and continue to be) dedicated to it every week. The first beta will be released once we have something presentable. In short, 90% of the time resources are being focused on this.

Infrastructure

• As I already mentioned, OVH left us without a VPS quite some time ago. We're moving to Arsys, and currently all traffic is already being routed through a VPS. All that's left is to migrate the domains. Coincidentally, the day after we started the migration, OVH responded (after over a month) with a reply that leaves much to be desired in tone, but it does resolve the core issue (they are refunding me for a service I paid for but never received).

AI

• We've started using a new model for forum translation, which puts us at the very top in this area. It requires extreme resources and is reserved exclusively for new posts. Additionally, we've reserved a processing slot that allows for the retranslation of one post per day. Peertube continues to use the previous model.

Administration

• To wrap up, the forum goes down occasionally. I'm looking into it, but it's likely due to the heavy load we've been experiencing lately.

[cobito]

Infrastructure

The cause of the forum's outages has been identified: a faulty RAM module. This is one of those things that drives me up the wall, but on the upside, it's an 8GB DDR3 module, so the memory crisis probably won't affect me much here (at least I'm not seeing outrageous prices).

I'm looking for one of the exact same model, and in the meantime, services have been redistributed as best as possible. Until it's replaced, we'll try to make do with the 104GB of RAM that remain operational (and, by the way, they're running at practically 100% capacity).

Regarding this, I'm closely monitoring the machine in case it's not just that module or if the issue lies elsewhere (motherboard, socket, memory controller, or whatever).

Test Bench

Code refactoring is ongoing. If there aren't too many setbacks, the first beta with a few new small features should be out in one or two weeks.

Front Page

It's been pretty neglected in general. I've focused on new features (forum federation, an agent intern with video summaries, the RAM failure that took up a lot of my time, etc.). I think it's more worthwhile to spend time on these things than on the front page, so in the medium term, there will be few posts per week.

Forum

As I already mentioned, it's federated and can exchange content with the Peertube instance. I encourage you to visit the threads with the selected videos. There's one on lightweight desktop environments and another on video game news, where the topics are presented in a rigorous and well-explained manner.

Regarding the integration with federated threads, they've also implemented showing them in recent threads. This will make the content translate (at the moment, those threads are in limbo). However, it's not clear yet whether they will appear in unread threads when there are new posts (even if they're local). The issue doesn't seem to be philosophically or technically easy to resolve.

Security

Attacks continue in bursts, but they have slowed down. Optimizations have reached the point where the bottleneck is now bandwidth (>400Mbps serving only scalpers). I'll look for a way to minimize it without affecting other things as much as possible.

Additionally, as some of you may know, several major security vulnerabilities have been discovered in both the Linux kernel and other components. We're up to date here, but lately we've been seeing things we hadn't seen before, and as I mentioned in another update, we need to stay very vigilant.

[cobito]

A few hours ago, a PeerTube security patch was released that also addresses another vulnerability discovered three days ago. The Hardlimit instance was updated both three days ago and a few hours ago.

The vulnerability allowed code injection, SQL injection, plugin installation, and the exploitation of processing tokens to manipulate videos published by users. Additionally, they detected a manipulation of the Google Analytics domain that was reporting traffic data to an illegitimate site (we do not use this).

The checks recommended by the platform developers have been performed, and no signs of penetration or manipulation have been detected.

The instance is now secure against these vulnerabilities, but all user sessions had to be reset, so you will need to log in again.

[cobito]

Today's tasks are essentially complete.

The web proxy component has been moved to the VPS. Following this, cache testing has been underway, and it now primarily caches content from the Peertube instances (videos and images), the Mastodon instance, and the museum. This should significantly improve the loading speed of certain files, particularly the museum's emulations, image loading, and video playback under higher traffic. Additionally, this is expected to reduce I/O activity on the NAS and the other drives.

From a security perspective, this is a major change as it isolates the internet-facing machine from everything else: nothing else has direct external contact except for the proxy-cache, which doesn't actually store any data.

Regarding software, the following updates have been made:

• Nodebb from 4.11 to 4.12 (improvements to the visibility of federated threads).
• Peertube from 8.1 to 8.2 (ability to transfer channels to other users, along with fixes for several important bugs).
• Mastodon from 4.4 to 4.5 (quoting other posts is now possible, provided the author allows it).

As for the hardware, XMP has been disabled on the Ryzen 9 9900X (dropping from 6400MT/s to 4800MT/s, a 25% reduction). This is done to lower the voltage from 1.4V to 1.1V and is purely a matter of thermals and longevity: if those RAM modules fail, we'll be facing a major issue. This will have a direct impact on the AI models. At the moment, while handling live tasks (processing them as they come in, without revisiting past ones), there is still spare capacity. Once the temperatures start to drop, we will switch back to the XMP profile.

[cobito]

The translator now handles federated threads. Until now, it wasn't possible to fetch them because NodeBB didn't expose any list that included these threads, but with the latest update, that has changed. As a result, you will no longer see the annoying "This post is being processed/translated..." message indefinitely. For now, it has been tested on some older threads (not all are included since we're running in "low power" mode), but it will be applied to all new ones.

Additionally, I've implemented a small utility I need for server management, which I'm sharing with you in case it comes in handy: visiting extra.hardlimit.com/ip will return your public IP as a plain text string, with no HTML formatting.

[cobito]

Quick update:

• It seems the Nodebb team actually did handle the embedded videos in cross-posts correctly. It was my fault for not enabling an option. Since yesterday, they've been showing up in new threads.
• We've brought on a new intern to process threads: someone named Smith. This is because if HIAL-9000 is both posting and processing the threads, it won't see its own threads during processing since they don't appear in its "unread" list (typical silly oversight that cost me almost an hour yesterday going over the scripts to figure out why HIAL-9000 wouldn't process its own threads ).

[pos_yo]

@cobito It's impressive everything you're implementing just "for fun". Great work, thanks!

[cobito]

Alright, last post of the term:

The end of the term is approaching, and with it, the holiday break, which will be longer than usual this year.

Benchmark

In the end, the benchmark has been left half-finished and will remain in beta for a few months. I want to keep adding features to it before finalizing it.

Basically, new features have been added to the interface along with secondary tools, but the Core remains unmodified despite receiving many changes that haven't been included. There are several decisions I still need to make.

The reason for not releasing more updates is that the program's code was (and to some extent still is) a genuine headache. It's much better now than before, but it's something that has been neglected all this time, unlike the rest of the pages, which have received some attention over the last 5 years. In fact, most of the Core's code was practically the same since the early stable versions (around 10 years ago?).

In short: many hours have been spent on invisible work, but it will help speed up the release of updates from now on.

It's been impossible for me to dedicate time to it in the last few days, so there probably won't be any more betas until after the summer.

Homepage

It's been the great neglected one. After experimenting with video summaries, I've seen that AI is superior to me in understanding, correcting, and condensing concepts. The homepage basically consists of writing a short news summary, which I've done manually until now. And honestly, after seeing that, I've developed an incredible amount of laziness to keep doing that task. It's like having a brand-new excavator with a full tank of fuel, sitting on its tracks, while a worn-out pickaxe and shovel wait for someone to pick them up to dig a trench. What a pain to dig a trench with a pickaxe and shovel! The downside of all this is that one of the homepage's purposes has been to show that there are people behind the scenes maintaining the site. Delegating the cognitive effort of this task to a machine will erase that "symbol" of activity in one stroke. On the other hand, I don't think it's necessary right now to show signs of activity just for the sake of it.

In general, the homepage has been something that hasn't motivated me much for a while, because it's hard to find real news (the world of personal computing is essentially dead), because sensationalism is the norm, and because sometimes you have to verify unlikely (but potentially true) information. It takes a lot of time and effort, and the value it brings is debatable. If a machine can do that job and even do it better than me, why not?

I still have to make the final decision, but it's highly likely that upon my return (sometime in September), the homepage will undergo a radical internal overhaul. It's the only page that hasn't received development hours in the last year, and since it's going to be redone, I think this is the way it will be approached.

Forum

I'm trying to inject some activity into it from the Peertube instance. It seems like the right place to publish text-based content, and in fact, I was considering migrating the posts from my old retro blog to the forum. I was also thinking about creating posts about thematic videos (The Computer Chronicles comes to mind right now) in a new retro subforum, since it's a branch with a lot of momentum at the moment.

In this regard, I'm still on uncharted territory. If social media and aggressive SEO ended the forum era over a decade ago, I'm pondering how a forum fits into the current AI era. Providing content as a starting point for discussions isn't complicated as long as there continues to be content generation on the Peertube instance. Being a Q&A hub remains a possibility for very specific and poorly documented topics. But attracting people to participate directly is a challenge.

We'll see if I can revive some old formulas...

Museum

It's been completely neglected this term. It remains the center of gravity for my interest because it's where I feel most comfortable experimenting without the fear of breaking things. But I can't leave the rest neglected, and besides, one also gets tired of always working on the same thing.

I'd like to shovel content in and push to the limit all the development I did a few months ago to see how it performs at scale, but the lack of storage prevents me from doing so. Still, I have some ideas to squeeze every last byte out of the 2TB allocated to this project.

Infrastructure

We have three urgent needs: more storage, more GPU processing power, and better energy efficiency. These are three machines that, as of today, cost an unaffordable fortune. During the next term, the acquisition of this hardware will depend exclusively on price trends, which puts us at the earliest in 2027.

Meanwhile, the hoarding frenzy by AI scalpers has greatly reduced to the point where, at the moment, we're running at nominal levels for both bandwidth and CPU usage. And to be honest, I'm genuinely curious to know the reasons behind this abrupt drop in activity compared to the momentum AI is currently experiencing, regardless of how beneficial this drop is for us. Knowing where personal computing is heading in the coming months directly depends on what the AI "industry" does, and given how extremely opaque that field is, having these discreet data points always helps a bit to understand the situation.

I don't have anything special planned for the next term: naturally, the benchmark program will be the priority. Essentially, I'll let myself be guided a bit more by whatever I feel like doing at the time (without neglecting anything and with some planning), and it will also depend on how feasible it is to purchase the planned hardware, since we're a bit limited in that regard at the moment.

If I can find some time, I might release a bit more until the first week of July, but not much.

Happy summer!

[cobito]

Last night, the hard drive on the Ryzen 9 9900X failed miserably. That left us without the forum or translations all day. It's been a disaster. I've spent about 10 hours getting everything back up and running, which involved finding a working hard drive, installing the system, and trying to recover the forum's database from the dead drive. It was impossible to recover the database, so I had to fall back on a backup from Saturday. That means any threads and crossposts created since then are lost.

For now, our bots (Smith and HIAL-9000) are offline. I need to get the entire AI infrastructure back online, which will be a slow process, so it'll be ready either today or tomorrow.