20150812

published 12 Aug 2015

Hey there, I’m smoogipooo and I’ll be hosting your daily progress report for today.

Preface

As osu! moves forward with OpenGL one of the biggest challenges we face is maintaining compatibility for older systems. With osu! and especially with OpenGL we strive to maintain compatibility until the statistics show that >99% of the userbase won’t be impacted by a change. Statistics have shown that this is not the case for the move to OpenGL, and we have also received numerous reports of OpenGL not working well on specific systems.

The cause

All the problematic systems had one common trait - the GPU manufacturer’s drivers were not installed and the only Display Adapter in the Device Manager was the Microsoft Basic Device Adapter and likewise gl_info.txt showed GDI Generic as the renderer. We realized that the problems indicated that osu! was being rendering through software. Upon diving into OpenTK I found that while all the graphics contexts had the GENERIC_FORMAT flag, none of the graphics contexts had the GENERIC_ACCELERATED flag which is required to obtain hardware acceleration. This type of acceleration is called MCD acceleration, and surprisingly enough Quake 3 managed to obtain MCD acceleration for reasons still unknown to me (black magic?).

The solution

For some cases it is enough to alert the users to install their drivers, however for many other cases the driver is incompatible with the user’s Operating System. Unable to reach a solution to this, a few days ago I mentioned the possibility of translating OpenGL to DirectX as DirectX is supported by Microsoft and compatibiliy won’t be a problem. Upon looking at how Google Chrome and Mozilla Firefox do WebGL I was directed towards ANGLE which seems to provide exactly this translation.

Well… Not exactly - there are a few caveats:

  • All OpenGL code must be changed to OpenGL ES 2.0, as ANGLE only supports ES 2.0 and 3.0.
    • This means that there is absolutely no fixed-function pipeline and everything must be vertex buffered - there is no immediate mode. osu! still uses the fixed-function pipeline for many of its core functions.
    • Slider rendering must be completely rewritten as they made extensive use of the fixed-function pipeline.
  • Vertices must be buffered and sprites must be batched in order to achieve the same level of performance as Cutting Edge. Tom94’s recent efforts with this has brought performance above Cutting Edge on the OpenGL renderer!
    • This also brings us closer to fixing sliders however we will explore slider rendering through shaders to further improve performance.
  • Shaders need to be rewritten to conform to both OpenGL ES and DirectX standards, complete with loop unrolling. Bloom is currently working but I haven’t had as much success with blur.
    • The shader framework has also undergone quite a few changes to accommodate their new usage throughout the game.

This is not a replacement for OpenGL and will be offered as a compatibility mode only for systems that can’t otherwise obtain a hardware accelerated renderer through OpenGL. OpenGL will provide better performance than DirectX through ANGLE and will remain as our choice of graphics library.

Conclusion

The project is currently in its final stages - buffering and batching are working, shaders are mostly working and most primitive elements are drawing. Buffering and batching everything is already cleaning up the codebase quite nicely so we’re enthusiastic about that too. All that’s left is to fix up sliders and clean-up any remaining code. I leave you with a pretty nice (in my opinion) effect I discovered after the red and blue channels of textures were inverted:

comments

20150811

published 11 Aug 2015

Another day spent mostly putting out database fires. Ongoing database migration efforts mean we are dealing with around double the normal system load, which has caused previously fixed performance bottlenecks to rear their ugly heads once again.

While I have not yet had a chance to completely write up the problem, those who have been following from the start will know what I’m talking about. I’m not sure when I will find the effort to fully explain the amount of effort which has been thrown at this particular issue, but it’s safe to say it will continue until we find the cause.

Today I figured our upgrade path from Percona Server 5.6.x to MySQL 5.7.x. Not as straightforward as it could be, due to the official MySQL upgrade process not playing nice with the custom changes that Percona makes to information_schema and performance_schema, but I got it working in the end.

Code changes to accompany the ongoing database migration that will work hand-in-hand to support multiple scores per user per beatmap are still ongoing. Today I fixed some shortcomings which still existed in bancho.

comments

20150810

published 10 Aug 2015

Hope you are all having a productive and energetic start to your weeks.

Another quick Monday update, as I spent most of today catching up on completely uninteresting tasks that I will spare you the details of. This week will hopefully bring some exciting changes to the table which have taken a while to get ready for public consumption.

I can say that we may have made a breakthrough in graphics compatibility for the next osu! release (currently what you see on cutting-edge). While we are testing internally, the whole team is hopeful that this will bring us as close as we are going to get to 100% compatibility.

More to come on that soon, by means of a first-time guest post :).

comments

20150808

published 08 Aug 2015

fireworks over tokyo bay, just an hour ago!

comments

20150807

published 07 Aug 2015

I received a few emails recently asking why there were hold-ups with API bug fixes and improvements. Looking back at the commit log, it was quite apparently how forgotten the API has been in recent times, so I spent half a day going through the most active issues and fixing what I could.

So why do things like this get “forgotten”? It’s not like I’m purposely avoiding the API in this case. With the new osu!web in active development, it just doesn’t make sense to continue adding new features to the old framework. But I get you – completely abandoning something like the API can stall development for your external projects, and therefore I plan on committing more time to keeping the API offerings growing in the old framework until we have the new systems up and running.

Going forward, I plan to make the API offered for public access completely shared with the API used by the actual osu! site and game. This means that not only will there be less code paths for us to manage, but that third-party developers will be able to access (mostly) everything you can find in-game or on the official site. There will of course be some limitations in place where required for safety/privacy reasons, but I hope to make things as open as possible in a hope to foster some amazingly imaginative and useful additions to the osu! ecosystem.

So if you are a developer and have something you’d like to see added right now to the API, leave an issue on the above repo (make sure to search first to make sure it doesn’t already exist; if it does, add a comment with your purpose for the feature to give it more priority)!

Have a great weekend guys. I’ll try and do either some streaming or Q&A over the weekend time-permissive, so keep a watch on twitter :).

comments