The observant will notice that I have been a bit less active over the last couple of weeks. This is partly due to the fact that I have been moving/settling in to a new living and office environment, and catching up with a lot of people I haven’t seen in a while. Things are getting back on track though, and I have been busy not only working on my own new features/fixes, but also merging the contributions from Ephemeral and woc2006.

• Added the ability to reset key bindings to defaults. This is one of those feature requests I see and instantly wonder why it isn’t already implemented. I then proceed to drop everything and implement it immediately.

• Fixed remaining issues with “Loading…” players in multiplayer games. This was being caused due to a constant not being the same on server and client, meaning if a certain number of presence requests were made at once, they would be silently ignored by bancho (causing those players to remain in a loading state forever).

• Widescreen support in Match Setup thanks to Ephemeral. This means that as far as I can tell, every game mode/screen in osu! is now widescreen compatible. Quite a good achievement!

• A few futher tweaks were required to hyperdash after getting feedback from pro players. This was an interesting one to follow up on, because while a lot of players voiced how much they disliked the new changes, they could not say why. Luckily I did find someone who could pinpoint the issue at hand – specifically the behaviour of the “dash” button during hyperdash – and it will be pushed out as a fix in the next release.

• I did a lot of internal clean-up on error handling scenarios, to make processing error reports a lot less painful. Also fixed a few scenarios where osu! wouldn’t automatically restart after it says it will.

• “Finally” updated the main menu copyright graphic, which is no longer really that. I removed the (in my opinion) unnecessary copyright text and made it more friendly and compact.

• I did also start work on a “Free Mod” mode for multiplayer, allowing each player to choose their own mods. There’s still quite a bit of implementation and testing to get this working perfectly, but expect it to hit the test build in the next couple of days!

Touching back on the start of this post, I am now in Japan for the near future. Give me a yell if you are in Japan (or planning to be over the next few months) and we may be able to catch up for a coffee, or game of jubeat ;).

As a side note, I was able to write this entry completely on my iPhone using the newly released Textastic code editor. Pretty awesome stuff.

Just a quick note that i will be skipping this week’s “week in osu!” due to being busy in real life and working mostly on the CtB hyperdash fixes with the time I had. Hopefully the post I made on hyperdash will make up for it :).

Over the last week, I have been working on fixing long-standing issues with the “Hyperdash” algorithm in Catch the Beat mode. Hyperdash is a gameplay tweak that was added to CtB in order to ensure that all patterns are catchable – no matter how far apart they are. It does this by makring certain fruit (objects) which would previously be impossible to catch as “Hyper” fruit. On catching these fruit, your catcher speeds up to the precise speed required to reach the next fruit in time.

This has always worked in roughly 99% of cases. Today I will detail the two main issues which caused the flaws in the remaining cases. For brevity, I have left out some other more obvious cases which were fixed earlier on last month.

Same Direction Chain

The catcher needs to catch three (or more) fruit in a line, where all but the last are hyperfruit. In the initial algorithm, the current X position of the catcher was always considered to be centered on the destination fruit by the time required, but because hyperdash would disengage earlier than this, catching all fruit were not possible.

The first fruit is caught as expected.

The second fruit is caught, but the catcher is dashing in overtime to get there. The green silhouette is where the algorithm expected the catcher to be.

Because the algorithm expected the catcher to be further than it actually was, the third fruit is not catchable. Hyperdash was required earlier on in this pattern.

Fixing this issues involves keeping track of:

• The current direction of the catcher
• The excess time after reaching the destination.

If the catcher has been moving in the same direction over more than two fruit, the excess required movement (if any) is added to the required distance, allowing hyperfruit spawning where required.

Hyperdash disengage times were also made more precise to ensure reaching the perfect position to catch the fruit.

Sub-frame Movement

The most edge case meant that some fruit may be possible or impossible to catch only by luck and frame limiter settings. Internal calculations where done on a sub-frame level, meaning:

possible_movement_distance = (object2.startTime - object1.endTime) * velocity;
require_movement_distance = abs(object2.x - object1.x);

if (possible_movement_distance < required_movement_distance)
	hyperdash(object1);

The problem here is that the possible_movement_distance was calculated based on the number of milliseconds separating the two objects. As osu! internally runs at 60fps in the worst-case scenario, possible_movement_distance may have been less or more than the actually available movement, due to where calculation frames end up falling (which is completely uncontrollable and random).

Correcting this is a matter of rounding available time downwards to the nearest frame + 0.5, which ensures that in worse-case scenario the time calculation will be 0.5 frames more lenient than required (ie. the objects will still be catchable):

frame_time = 1000/60; //60fps (16.66667ms)
actual_frames_available = floor((object2.startTime - object1.endTime) / frame_time) + 0.5;
possible_movement_distance =  actual_frames_avilable * frame_time * velocity;

When the game’s frame limiter was set to 60fps, frames were guaranteed to be a full 16.6ms, rounding calculations upwards and allowing slightly more movement. They would therefore match the old algorithm and be catchable without hyperdash.

When the game’s frame limiter was set to 120fps or unlimited, frames could be any length, causing higher precision in calculations, and introducing the possibility of sub-frame errors as seen above. While the fruit should technically still be catchable, rounding errors meant that they were not catchable around 50% of the time.

I spent the last week stumbling around trying to keep up with both real life as well as osu! commitments. I don’t have much to show this week so instead I will go a bit more in detail than usual on what I did get up to.

• I got really annoyed at cross-compatibility of symlinks which are committed to git, so took a day to get rid of every symlink (more effort than you’d think!). The real issue only shows itself when you are using a git repository shared across multiple PCs using Dropbox.

  Most symlinks were in place for large data stores which may be mounted across disks or PCs (using NFS). Removing them meant replacing with nginx location/aliases where possible, or absolute paths in other cases.

  For example: ./repo/www/mp3/ → /data/large/mp3/

  In this example, we see ./repo/www/mp3 inside a git repo pointing to an absolute path which may be a local folder or mounted hard disk. In the case of storing these in git, there is no initial issue – symlinks are a file with a special link attribute set and contain the path to their destination.

  After checking out the repository on a unix based system, symlinks will correctly point to their absolute destination, so if you have a folder available at /data/large/mp3/ it will work instantly after deployment via git. This works great for automatic server-side deployment without having per-system configuration containing locations of certain sets of files.

  I store all my git projects inside Dropbox, which means I can use multiple PCs simultaneously without a tiresome commit->push->pull to get both PCs in sync. When symlinks are synchronised across to windows PCs via git, windows is unable to keep the link attribute intact (it contains no such attribute), and the links become non-linked text/binary files. This causes a conflict between the two dropbox connected PCs and results in really weird scenarios which generally break git until you pause one PC’s Dropbox syncing and reset any file changes.

  Long story short, I would highly recommend avoiding symlinks inside git unless completely required :).

• Database was getting big, so I took a day to prune unnecessary data and do a clean-up of indices. I aim to do this about once a month (basically as soon as things start to fill up) to both keep the overall filesize within acceptable limits and keep the database running in an optimal state.

  This generally involves:

  • Review any new tables added. Decide if data needs to be stored permanently – if not, start culling old data on a daily or weekly basis.
  • Use some smart index queries to find any unused (or very infrequently used) indices on new or old tables. These can usually be removed and result in a positive impact to insert performance due to less disk writes.
  • For data paths which are read and write heavy, it may be possible to move indices to slave database servers, leaving the master free to write only the base data, but still have performance benefits when querying slaves.
  • Check for any tables which have abundant free space. This generally happens when large amounts of data are temporarily written to tables (sometimes due to a bug, something due to a short and sudden increase in traffic). Cleaning these up is as simple as running an OPTIMIZE TABLES n; command, and defragments remaining data in the process.

  End result was a 10% reduction in database size!

• Ephemeral started widescreen support for the editor, and I finished it. Feels pretty nice to use! Also if you run osu! fullscreen, the editor will always default to borderless fullscreen now, meaning an almost imperceptibly smooth flow to the editor.

• A lot of time spent trying to get Catch the Beat “Hyperdash” to work correctly for all patterns. Previously there have been issues in cases where objects next to each other should be hyperdahsed, but aren’t due to sub-frame calculation errors. Still haven’t finished work on this, and it is holding up the next public build, so is my highest priority at the moment. You can read about the specifics of fixing hyperdash in my separate blog entry.

I hope everyone has an enjoyable Christmas break. I took a day or two off, but spent the rest of the week focussing on some scalability issues creeping up on osu! – the main one being the initialisation process when first connecting to bancho.

• Using the concept of caching user information locally, I reduced the amount of data from server→client by 90%. osu! can now play in solo mode without ever receiving more than userIds of all online players. When users need to be displayed or interacted with (chat / multiplayer / online users display) this information is requested on-demand. I am still experimenting with the best balance between local cache size and expiry times, but you should already be able to see the improvements when first starting osu! and connecting to bancho. THis should help out those with slow internet connections quite a lot.

• I reworked much of bancho’s internal to treat all clients (osu! / irc) evenly. Previously there were a lot of special cases to handle multiple logins from the same account, including

  osu suffixes on all osu! client’s username, and the inability to have an IRC client share a username with an osu! client in many cases.

  These changes not only allowed dropping that ugly username suffix, but also means that a single user can have multiple IRC clients all running under a single unified username. In the process, I also removed the ability to change IRC usernames, so it should be a lot easier to identify people too (even though you could only add a suffix previously). The code for handling user connection is also a lot cleaner, and will allow for special cases like tournament spectating without having one osu! account per running client.

• I fixed most of the remaining widescreen compatibility issues, and also went through a lot of hard crash bug reports and fixed them. osu! should be a whole lot more stable after this week’s updates. In addition, I changed the way crashes are handled so a lot of errors (>80%) that would previous hard-crash osu! are now handled (and reported to me) without the game dying completely.

• I made some fixes to input handling which allow prioritising certain groups of game elements on a situational basis. The main reason for doing this was to allow universal volume adjustment – now available either using up/down arrows or mouse wheel, and in dialogs where it wasn’t previously by holding ctrl+shift while using the controls – but also allows for a lot finer control of input handling, especially in cases where there are many scrollable areas, buttons and overlapping layers on screen at once. Quite large internal changes, but quite invisible to the end-user.

• I have started work on fixing long-standing Catch the Beat hyperdash issues, where fruit would occasionally be impossible to catch due to sub-frame calculation errors. It is going to take a bit more time to get things finely tuned without affecting the difficulty of existing maps, but should be ready for the next public build, if not the one after.

Let me take a moment to note that I have recently been streaming dev sessions on twitch.tv. Make sure to follow me if you’re interested in coming and watching, asking questions or whatever. It’s a lot of fun, and I play some epic music playlists to boot :).