Hi MD friends! It's only been a month since the “hobby mode” announcement, but there have been some developments that should be addressed.
Almost immediately after I announced “hobby mode” in the last update, the site began experiencing performance and stability issues. Analytics suggested this was due to a sudden and continuous spike in traffic, but with no corresponding mention on a YouTube video or article.
With our heavy use of databases and filters, I always knew that wiki pages were quite intense on the web server but things had been stable (at least once we moved the site behind Cloudflare and used a firewall rule to block a lot of bots.) But this new wave was too much for the server to handle and it was going down like a soccer player near the penalty box.
I experimented with PHP configuration settings like max_children and different scaling (dynamic, ondemand, static, etc.) but the result was the same - slow loads, time outs, and server hangs needing a hard reboot. Even Cloudflare’s “under attack” mode, which requires every visitor to pass a captcha, didn’t help. We also enabled some new bot protections which weren’t available before.
Eventually I decided to rebuild the wiki server on a new instance with upgraded specs (and more money per month) - this machine is our 5th instance (wiki05) and has proved to be more stable and performant than any one before, which is exactly what we want considering the price and work put into it. It is not just a move of the same machine to more powerful hardware but a complete recreation of the wiki based on a backup similar to the ones we make public several times a year.
But the sailing was not all smooth from there. Even on the new server, with more bot protection, we saw spambots overwrite pages with absolute slop and a few URLs mixed in.
The most visible was the site’s start page, which was quickly reverted and a new ACL added so that only site admins could edit that page. I searched the usernames of the spammers in the internal User Manager to see which email address was used and the pattern of the usernames, then filtered the user list by similar email domains and manually deleted ones which matched the pattern - if your legitimate account was caught up in this, please accept my apologies and do recreate your account if needed.
Next the spammers focused on two pages in particular: the Sony MZ-E33 player and Getting-Started/Recording, which is a page which may have existed before the guide refactor but I forget the specifics and so does Dokuwiki. I repeated the same steps of creating restrictive ACLs and removing spammer accounts, but one man cannot defeat a swarm of dedicated bots. And this is why we’ve never been able to successfully allow anonymous edits despite registration being a barrier to entry for contributions.
Luckily, Dokuwiki has a plugin to use Cloudflare’s captcha mechanism on logins, which does seem to have worked (for now….)
This segues into talking about Cloudflare itself. Its proxy is a service which is very widely used to block bots and DDoS attacks which take down a site, but is not without its detractors. I’ll leave more detailed commentary for a personal blog post but needless to say, a lot of people have issues with Cloudflare. They don’t like that it blocks a lot of older or esoteric web browsers, requires Javascript, and that an outage would take a lot of the internet offline.
I share those concerns, and the site was working fine without the Cloudflare proxy for several years. Until the bots overwhelmed the server and blocking via Cloudflare was the quickest and most convenient way to bring the site back online and not have resources vacuumed up by selfish or malicious actors. It’s also free, which is nice considering how much the server infrastructure (not just for the wiki, but also Web MiniDisc Pro, the LP encoding servers, development servers, etc.) costs.
In the hobby mode announcement on Reddit, one commenter suggested a chatbot to answer basic questions before people choose to post on social media. This is a good idea, and Dokuwiki recently added a chatbot plugin to allow an LLM to answer questions based on the wiki’s content. But this isn’t something I’ll be pursuing in the near future.
First, I doubt it will actually do much to prevent low-effort questions. If someone finds the front page of the wiki, there are big links to our Getting Started guides, FAQs, and other resources which already aim to answer those questions. And if people have questions after reading that, they’re usually pretty good questions that don’t burn out or frustrate community members.
Second, it will take time and money to configure an LLM plugin and (apart from this long update post) that isn’t where I’m putting my resources right now. Especially right after an outage caused in part by AI scraper bots.
Finally, the debate around AI is so polarized and toxic that I don’t want to drag the wiki into it. I have complex and shifting opinions of the tech, but the internet is rarely a place for nuance and more often a place for harassment.
I’m sure you’ve heard by now that the UK has passed a law which places onerous burdens on sites and services which host adult content.
Of course, we don’t host anything explicitly adult (although a few album art images may be NSFW in nature and I’m sure some people do find pics of MD equipment alluring…) but as a wiki we could find ourselves the victim of spam or defacement which is adult in nature.
And although neither the site nor myself are based in the UK, and I have not been there for over a decade, I am a British citizen with family in the UK. And should I ever go back to visit I would rather not risk being pulled aside on arrival at Heathrow and facing fines or other trouble for operating this site.
I’ll be doing some research on how at risk I am for operating the site as it exists today. If I am told that the risk is above-zero, I will simply block UK IP addresses from the site. I will not consider any type of age verification service or removing content from the wiki, nor will I make it read-only. The legislation of one country will not affect those outside of it, especially on a site which mostly lists battery types for 30 year old music players.
I hope that my fear is overblown and no action will be needed. But in full transparency, this is where I stand on it.
With all of that out of the way, it’s time to disappear into the ether again for a while. Hopefully nothing else will come up which requires another update so soon, be that legislation or outages.
On that note, please do not email the admin@minidisc address to report incorrect data or suggest changes - it’s a wiki! Edit the page! We have a contribution guide which you can refer to for how to add to the site. We do always welcome emails regarding issues with registration, outages, or spam attacks, but simple page edits can be done by anyone with an account.
I’ll continue to monitor the site occasionally for spam edits and outages, but an email to let us know may get a faster reaction.
Thanks and until we meet again…