Kinect is dead, long live Orbbec!

by Dr Mike

---

Posted on December 2017

---

---

It was two months back when my trusted ally and a close personal friend, Kinect, passed away. It was a moment of sadness, despite the fact that the death had been pending for quite some time. You see, Kinect was suffering from Never-Ending Preview Syndrome (NEPS). Microsoft didn’t develop Kinect SDK 2.0 for Windows beyond Preview version (for various reasons you can read about elsewhere). Few applications developed by others made it really big. Yet, that SDK, Preview or not, was so well made that I came to miss those beautiful APIs. We had good years together.

Although game shops and other retailers still have shelves full of Kinect sensors, and my pipeline of Kinect projects is as long as ever, Microsoft’s decision of not producing any new devices will effectively stop new cool applications from emerging at the speed we had before. No serious product development would pick a dead platform to start something new with. Obviously.

Is there afterlife for Kinect? Yes, sort of. The core technology remains somewhere inside HoloLens and the origin of Kinect’s skeletal tracking is still available in other libraries, for instance Nuitrack is being developed further.

Just now, the developers of the most well-designed 3D sensor hardware, Orbbec, pierced through the markets with their OpenNI enabled body tracking SDK with Unity support. Now things get practical! I got Orbbec Astra and Persee long ago and the only way to get any apps going was to do some good old C++. This won’t do for a modern application developer simply because of the trouble piecing the old SDK together with any modern front-end technology, Unity being the de facto choice. Of course it’s doable in a couple of ways, but hardly a pleasure. So now, finally, Orbbec supports Unity!

At this moment, there is a huge bunch of Kinect applications out there needing an update to a newer hardware, Orbbec being the closest thing. But what’s the difference when moving on from Kinect to Orbbec? Let’s take a look at the specifics from application development point of view with focus on porting an old Kinect-driven game app developed originally with Unity to an Orbbec sensor.

Here is Orbbec’s Persee and Astra Pro side by side. But what’s the difference and which one would you need? Other Orbbec’s Astra models are similar but with lower specs or smaller size, so typically you’d pick either one of these.

Specs comparison

First, let’s compare the technical specs: Microsoft’s Kinect vs. Orbbec’s Astra Pro and Persee.

The price I won’t consider that much since each of the sensors are in the class of US$100-240. But one thing is worth highlighting here. If we look at the cost of a full stand-alone installation, Persee has the obvious advantage as you can skip having any laptop on site entirely. You can get the whole thing going with any HDMI-enabled display using Persee sensor/CPU combo. Could you run Kinect on a hundred dollar laptop? Probably no.

A couple of notions on the specs:

• Body tracking is similar enough to build almost identical applications with Kinect and Orbbec. The few extra joints that Kinect had, namely fingers and feet were never very reliable for any need I encountered.
• You should give up all hopes of your turning Kinect face tracking app to Orbbec (at least for now).
• Orbbec sensors are really tightly packaged. You could (in theory) fit almost 6 Astra Pro devices in the volume of one Kinect. Even Persee with its minicomputer is about half the volume of Kinect. This aspect is not to be underestimated especially when designing installations where the sensor is hidden inside the cover of the rest of the product.
• Range and field of view are so similar you won’t notice much difference from the normal 3 to 4 meter distance. Kinect enables body tracking at a little bit closer range, that’s all.
• Orbbec frees us from the annoyance of Kinect’s USB3.0 compatibility problems (which too often prohibited setting up a low-cost laptop-based installation). Orbbec sensors only need USB2.0. Orbbec’s hardware architecture enables computing to take place on the device so that all of the raw infrared data does not need to be transferred onto the PC for processing.
• Installations based on Orbbec sensors require always one less power cable, which is always a good thing.
• Orbbec beats Kinect in depth image resolution. Although the difference does not look very big, having 90,112 pixels more makes a big difference in advanced imaging applications and for developing precise recognition algorithms
• Orbbec is no match to Kinect in video image quality. On the other hand, few applications I’ve made really require HD image quality.
• Persee transforms the usage model of the entire system from PC (keyboard, mouse) to a remote control Android TV-like usage.

Example case

So that was the basics. Now, let’s take an example, our classic Fishing game installation for starters. How would this installation look like using Orbbec sensors?

I’d definitely pick Persee which would enable our installation to:

• Avoid using any laptop/desktop PC on site.
• Use the video stream like with Kinect, although it might look slightly worse because of lower resolution.
• Use the body tracking to find hand and head joints like with Kinect.
• “Take photos” i.e. screen captures like with Kinect (we never needed HD resolution for this purpose).
• Issue coupons over WiFi connection like with a PC. We used a simplistic Google Drive implementation on Windows, and there wouldn’t be a problem installing the Google Drive on Persee either.
• Use one less power cable (I’d only need one for Persee, one for display).

It looks like there wouldn’t be any significant drawback in porting the Fishing game to Persee. Only advantages, actually. Persee is the way to go!

Bottlenecks

Even though the above may look rosy in the world of Persee, things are not all perfect. Here’s a short list of capabilities that would probably face some problems combined with a few issues I’ve encountered so far:

• Local disk space available on the device is minimal. This prohibits many types of applications that require data recording (color or depth image). You could add another USB stick or an external hard disk to overcome this bottleneck.
• WiFi connection gets pretty slow occasionally. Once I moved Persee close to my router, the problem was solved. This can create a serious problem for apps that need continuous online connectivity, e.g. streaming, uploading data etc. But there is a LAN cable port in the device, but that can create another practical problem to your installation.
• The Unity sample scenes with all display options enabled seem to run only at about 10-15 frames per second. I’ll have dig into the performance topic in another article.
• The built-in CPU (Quad-core Cortex A17 1.8GHz) and GPU (Mali-T7 600MHz) cannot be of course match for modern apps and games that demand high quality graphics and are designed for VR-ready PCs. Persee has a different purpose altogether. And combining other devices to work with Orbbec is simply out of scope. Don’t think Persee as a PC replacement, but as a 3D sensor equipped smartphone.

General recommendations

To summarize, although all these sensors discussed here look pretty similar at first glance, there are some fundamental differences, especially in terms of practicality of installations. We can take this simple rule of thumb for selecting the replacement for Kinect that I apply in project planning phases with my clients:

• Pick Orbbec Persee if your old Kinect installation is simple stand-alone type, doesn’t need much local storage, processing or super fancy graphics.
• Pick Orbbec Astra Pro if your old Kinect installation requires a PC for other purposes anyway, such as lots of disk space, heavy or parallel CPU processing, high-end GPU or such which is beyond what Persee can offer.

Obviously, the thinking behind this is the aim of minimizing the cost of swapping the sensor in a typical product development process. You can keep most of the Unity content and only change the middleware connecting your Unity app with the sensor. Some rework will be needed on joint tracking, color and depth image dimensions and specific differences in API level method calls between the middleware components. But it shouldn’t be a huge job, unless you really must port a large application where the 3D sensor is just one small part from Windows to Android altogether. That would get heavy especially if you have custom inter-connected components working outside Unity.

But that’s enough for starters. The next couple of articles will show you how to get started with Orbbec+Unity and get knee-deep into coding Kinect-like apps on Orbbec’s superior hardware.