GF needed a new pair of headphones for her MP3 player. The cheap POS phones she was using developed a bad connection, just like the cheap POS pair before that. And having a functional portable music system matters more now that she's taking the MRT to work every day instead of scootering to school. The conventional earbuds don't do much to block out the train noise, and she had to turn up the player way too loud to hear anything resembling music. So I wanted to suggest a earplug-type headphone to help block noise. However, the headphones must also be capable of acting as a lanyard for the MP3 player, hung around the neck. So in order of importance:
Couldn't exactly take the GF running through the geek-warren of Nova. There's plenty of fancy boutiques on the Taipei east-side, but fancy is not what you need when you're looking for commodity electronics. The headphone selection at 燦坤 branch on 忠孝東路 was a bit less than pathetic, with only a few junky Chinese brands. Went to Mitsukoshi for dinner. There was a FNAC branch in the A9 basement so we went by after dinner to check it out. They used to have a pretty good CD selection, but they cut that way back to put in more high-end audio and specialty electronics like notebooks and cell phones. No point in selling commodity junk at a premium location. Their headphone selection wasn't great, either, but at least they had some reasonably decent Japanese stuff (made in China these days, oh well). I've been considering the Sony MDR-EX51, but it didn't have the neck-lanyard. Still, hanging the MP3 player or cell phone around the neck has been the Asian trend for a while and I expected that they'd have something for the market.
Finally after searching all over the store (and after one very scary but ultimately harmless call about GF's grandfather in the hospital) we finally found the Sony MDR-NX1 headphones, with the EX51's earplug-type design but with a neck cord. Perfect, except it was NTD1750 and they only had the red color. The color was fine with the GF, and it's been a while since I got something nice for her anyway, so what the heck.
Of course, when we did go to Nova with her mom on a shopping expedition we found out that the Sony headphones could be had for four hundred NTD less at any of the little audio shops in the tech-mall. Not to mention that Panasonic and Audio-Technica also had similar products available for a couple hundred NTD less than the Sony. Oh well, so long as the GF's happy with it, right?
The Penny-Arcade Forums are down. Now what am I suppose to do at work?
Obviously the addicted hordes must be hounding the sysadmin for updates. So he's turned the placeholder page into a Downtime FAQ.
Okay, update on progress. I uncovered what (I believe) was causing all of our problems. An overzealous RAID controller is trying a bit too hard to ensure data integrity. On periods of high load (ie, when you guys would all hit the forums) the combined load between the disk checks and regular server load was too much for the power supply. The result was the machine writing bad bits into RAM causing a whole host of problems. I need to tune the settings to reduce the priority of those checks, and then I can pick back up with the work I was planning on. The work will end up doing some load distribution that was sorely needed, but I didn't have enough motivation to do before the data started corrupting. After all, nobody likes downtime.
So apprently the thought of a Test Server never occured to the guy. The forum's been falling down regularly over the last few months and he can't be bothered to actually do any work about it until the machine started randomly spewing bits and the database blows up. How about some preventative maintenance, eh?
Now, some serious data was lost, and unfortunately MySQL does nothing gracefully, so I am uncertain if I will be able to recover some of those lost posts. Of note the loss of the ban thread saddens me. We maintain an internal list of bans and the reasons behind them for the staff to see, but I liked keeping everyone in the know. There is also a list of users who got deleted accidentally, for my own sake I'll list it here (BucketMan, CovertOperative, SenorAmor, Bogart, & TheSentinel). MySQL told me that we lost at-least 50 users in one of its "table repairs,"
What the fuck is the point of having a database if it can't gracefully recover from failure? The whole theory of relational databases is based upon the idea of data integrity. And it's not really much of a "repair" if it's eating data in the meantime, is it?
Q: If MySQL sucks so much, why not just move to PostgreSQL now and not worry about building a new forum software?
A: A tempting idea. phpBB does have builtin support for PostgreSQL, however, after many years of hacking this code, a lot of the changes we have in place are not built to be database independent. A good example is the jail code Ramius wrote before he left. Ramius wrote some very slick MySQL code which is optimized for MySQL 4.0+, the problem is, the code won't run on a PostgreSQL server. By the time I found all those little broken things and fixed them, I would be close to rewriting everything. The bigger issue is that we would just be delaying the inevitable. phpBB has stalled somewhat from a development standpoint. Despite their recent attempts to resume doing bugfixes and other fairly simple tasks, we need to jump ship soon.
"Real slick code" that's married to a data-eating system and not portable to other SQL databases? I'd pull that kind of crap if I were writing a one-off program for my own use, but I'd never release that kind of shit into the light of day. And so much for documentation and maintenance, once the last guy leaves nobody else knows what to do with his fancy hackz.
What it comes down to is this; if I wanted a "traditional" forum software I would just patch phpBB to use postgresql and be done with it. phpBB powers the largest forum on the internet, with some hacks it would serve us too. I want something more though. The current theory in large webforums is to tune your web and database server software and throw more hardware at your problems. The better sites will even introduce things like memcache to scale better. I think that we need to rethink the way forum software is written. We have learned a lot in the past 6 years, and I think it is time we use some of that knowledge. I have nothing against PHP, it is a good language, but Ruby on Rails provides a better framework for what I have in mind.
If you like the idea of helping out with a project like this, shoot me an email. Due to the nature of the project, I am looking for fairly experienced programmers...
Maybe they should do some basic tuning before trying to reinvent the wheel? Even if you have a fancy new code you still need a scalable front-end if you want to grow. There's nothing wrong with a good caching server and a good caching webserver system can be put in front of whatever forum backend they end up with. Heck, it's not as if Ruby is even all that speedy. One suspects zealotry whenever someone mentions Ruby on Rails as the solution, especially when the guy would rather rewrite entire systems rather than fix the problem at hand. To add to the zealot-quotient, make it Open Source, but we only want experienced programmers. Because they have nothing better to do.
To-do List
...Rebuild FreeBSD 6.1 Beta - Done!
Because running a beta operating system really helps your stability and uptime.
I'm sure all his friends and cow-orkers worship him as the righteous Geek God, but crap like this makes me wish more work would get outsourced to India so wannabe hackz like him can starve to death on the street or get a real job. Not that our admins are that much better, as our database server blew up last weekend and we couldn't get it fixed for two days because our three-letter-initialed service contractor only works M-F. Oy vey.
Went to Düsseldorf for three days and two nights, all included. Unfortunately, all it included were day-ful of meetings for work, so I literally zero time for sightseeing. Didn't help that we were staying out-of-town near our distributor's office, in a little village surrounded by fallow winter fields and denuded trees. Also got a good look at the Zürich airport, considering we were stuck there for five hours waiting for our transfer to Düsseldorf. The airline was kind enough to issue us each a 5CHF refreshment voucher for our wasted time. I think even our line-workers in China get better treatment than one franc an hour. The vouchers were just enough for a cup of coffee for each of us, and the vendor guy was cool enough to toss in a candy bar to soak up the leftover amount. Hey, the GF said European chocolate would be a good gift, so that's one thing taken care of. Had to take a bus out to the commuter plane and walk up the stairs, which gave me a chance to see falling snowflakes for the first time in I don't know how long.
Did get to stop by an office on Königsallee in Düsseldorf city center for a meeeting, but didn't get the chance to browse the fancy shops along the thoroughfare. The city does have a large and modern airport, completely rebuilt after a fire ten years ago. Wandered around and bought a Düsseldorf keychain for the GF while waiting for our flight out. Us Asian folk were getting a bit tired of the coldcuts+cheese on bread of the typical German lunch, so our sales manager splurged for sushi, an offshoot of a restaurant in town. It wasn't bad, although it did cost more than a meal at the much-superior Japanese places back home. Funny how after all that we still ended up with a plate full of cold food, although the miso soup sure hit the spot (for two euros it damn better).
It was a working trip, after all, so no complaints on the lack of R&R. Did get the chance to have a few genuine German brews, see the Rhine, and check out our European distributor facility. Plus the chance to finally pile up some points on my Cathay frequent-flier card. I'm happy enough with that. On the flight back, we ran into a crowd of Chinese folks returning from an exhibition in Düsseldorf, each with multiple massive suitcases full of show samples and their own purchases. Made the flight on Swiss Air seem more like your typical Asian flight, complete with midnight cup-o-noodle runs. Whatever it takes to fill up the planes, I guess.
Was taking the stairs up to the apartment one day when I noticed a small potted plant left on the garbage cans in the stairwell. I guess the owner was too lazy to open the can to dump it in. Decided to take it home for some reason. A bit of greenery to help recycle the Taipei air, I suppose. Even after regular watering, the succulent-type plant was always kinda limp and never grew or did much, not that I was expecting a song and dance from a plant or anything. So I went away on holiday for CNY, and when I got back I find that the plant had all of a sudden bloomed with pretty magenta-and-white flowers. So instead of being just a charity case it's now quite the astute find. Yay me.