|
You're looking at the results of searching across 14MM posts. The hardware is sufficient, but VB, untouched, is kind of a dog. We recently switched to MySQL fulltext search instead of the VB native search (should've been done ages ago) so the boards in general are a lot faster than they used to be, but specific types of search cause some nasty lockups due to MySQL inefficiencies..
We've got an angel process that monitors the boards every minute looking for a timeout, and when it does, it nukes the offending query.
Next steps are to replace native VB search with Sphinx or something like it, and possibly set up a dedicated replica for searches. The boards are also going to be moving from FreeBSD (which has known issues with MySQL) to Linux and newer hardware (although the problems you guys see today are not hardware-related, they're due to inefficiencies in how VB does things).
Big improvements coming. All research indicates Sphinx will be a silver bullet.
This is not something unique to GFY--all big boards run into this (assuming they're running VB+MySQL). If you're a Somethingawful forums user, there's a discussion in their Questions/Comments/Suggestions forum that goes into some heavy detail on the shortcomings of MySQL and search.
Their approach has gone from crippling search to disabling it altogether at times due to horrible performance.
If any gearheads are interested in some detail from that and aren't members of SA, I can paste some snippets for you. In essence, though, they're going to end up going the same route we are and take search out of the database.
|