20150813

published 13 Aug 2015

Today marks the completion of a gruelling 78 hour database schema change to support multiple scores per-user-per-beatmap. Why did it take so long and what did it involve, you ask? It was an index change on the main osu!-mode scores table, rewriting 38gb of data in an asynchronous process to avoid any table-locking. While pt-online-schema-change makes this easy to do - and tries its best not to degrade database performance - there were still times of increased load resulting in score processing delays. I invested some time in improving the queue mechanism to ensure scores are always received (with pp awarded later) wherever possible.

Continued MySQL 5.7RC roll-out on a slave database (used for country and mod-specific rankings, as well as being the food source for Elasticsearch). Trying to get the upgrade process as perfect as possible as I move to deploying it on the primary slave and master, hopefully next week. Also used this as an opportunity to do performance testing under standard load - quite important as the previous 5.7.x build I tested resulted in HUGE performance regressions.

Yesterday I invested some time in adding log output for the update process. The output is verbose enough to diagnose any update issues people may come across, and will help progress towards my ultimate goal of making a 100% fail-safe install/update process, bootstrapped inside osu!.exe itself. I’d say it’s already around 99% infallible, but as long as users are reporting infinite update loops I won’t be happy.

Haven’t done a Q&A for a while, so let’s make it happen tomorrow. Ask me anything about infrastructure, databases or servers. Tweet with the #FridayQA tag so I can easily gather your questions!