My Persee just got faster!

Posted on July 2018

It's been a bit quiet for me on the Orbbec front. But finally things are moving again. This time it is mainly because of the latest version of the Astra SDK 2.0.9 Beta3d released on June 26. Why is this relevant? Simply, because a few things were running too slow, for me at least. Luckily, with the latest update my Persee just got faster!

In this article I just give you and example of what performance improvements mean in a practical application. I earlier had to modify the core AstraController to speed things up. Now I swapped back to the original way of using Unity events and their bindings.

Setup

Let's list the biggest differences that affect typical game development. My benchmark game running on Persee consists of:

Mono-color depth image representing the user as a silhouette (ColorizedBodyFrame)
3 joint overlay sprite images (BodyStream, BodyFrame)
Animated sprites moving on the screen that the player has to hit
(Additionally) I tried swapping the mono-color depth image to background removed color image too

Basic stuff, that is. (I won't be able to share the image because of confidentiality, please don't ask.)

Comparison of SDK versions

Simply put, I just compare the same game with SDK version 2.0.8 with my previous performance tricks of removing event handling vs. the latest SDK version 2.0.9 with variable image sizes. My main concern has been the framerate so far.

AstraController's debug text has helpful measurements for testing performance, so let's look at the variables available there:

Tot means the amount of milliseconds to prepare a full frame, which should be 33 for 30fps
AU means the amount of milliseconds that sensor update loop takes
FR seems to be the amount milliseconds perhaps just for drawing the data out of the sensor, but this is always around 0.3-0.4ms, i.e. too small to have any impact
AUI is basically derived from the above as AU-FR, so let's skip that too

Now we only got two measurements to look at, AU and Tot, besides the overall framerate of the game. So, let's test!

	Event-removed 2.0.8 with 160x120px depth image	Event-removed 2.0.8 with 320x240px depth image	Clean 2.0.9 with 160x120px depth image	Clean 2.0.9 with 320x240px depth image	Clean 2.0.9 with 160x120px depth image and background removal
AU (ms)	6	45	9	11	9
Tot (ms)	19	65	19	19	18
Game framerate (fps)	30-50	12-18	50	48	55
Good enough for interactive games?	No (occasional lag destroys gameplay)	Just no	Yes!	Yes!	Yes!

This means Orbbec Astra SDK 2.0.9 has some quite fantastic performance improvements! Now without any tricks and cheats you can use the normal clean SDK to run proper depth image and even background removal on Persee.

My choice for representing players in any game running on Persee is definitely the 320x240px depth image. The only problem with the background removal is that even with 160x120px depth image the color and depth images do not match when moving arms or legs quick. This makes it look strange in any action type of game that I usually make.

Hopefully this info help you to pick your visualization method for player representation in your games. Good luck!