Astra Pro distance range

Posted on July 2018

This time I just wanted to clarify something a few readers have been asking: what the actual depth measurements that Orbbec Astra Pro gives? Some seem to think it is something like 6 meters only. Well, what would a scientist do other than go and run an empirical test?

Here is a simple code snippet for visualizing distance ranges in a nice way. You can use it as your personal CT scanner to slice the real world.

Basically, what I've done here is based on the Unity SDK's default DepthTextureRenderer which allows us to turn every depth pixel into a color pixel on the screen. I've modified it to show different color for all the pixels within a specified range in millimeters. Also I added simple calculations for finding the far and near pixels as well as the average distance of all pixels. One thing to note here is that there are a lot of empty depth pixels that we should not count in. I visualize those in a different color, so we can see the difference.

Let's call our creation DepthDistanceDebugger. Here's what it does in plain C#. First, let's do some settings in Unity Editor:

Copy the Scenes\TestScene to name of your choice
Toggle off all the visualization scripts under Main Camera game object except DepthMapViewer
In DepthMapViewer toggle off the DepthTextureRenderer script
Add the below DepthDistanceDebugger script to the DepthMapViewer

using UnityEngine;
using UnityEngine.UI;

public class DepthDistanceDebugger : MonoBehaviour
{
    public Slider highRange;
    public Text highRangeValue;
    public Slider lowRange;
    public Text lowRangeValue;
    public Color rangeColor;
    public Color emptyPixelColor;
    public Text textDisplay;

private Texture2D texture;
    private Color[] textureBuffer;

private short[] depthFrameData;
    
    private void Start()
    {
        textureBuffer = new Color[320 * 240];
        depthFrameData = new short[320 * 240];
        texture = new Texture2D(320, 240);
        GetComponent<Renderer>().material.mainTexture = texture;
    }

public void OnFrame(Astra.DepthFrame frame)
    {
        if (frame == null || frame.Width == 0 || frame.Height == 0)
        {
            return;
        }
        
        EnsureBuffers(frame.Width, frame.Height);

frame.CopyData(ref depthFrameData);

short far = -1;
        short near = short.MaxValue;
        int emptyPixels = 0;
        long total = 0;

for (int i = 0; i < depthFrameData.Length; i++)
        {
            short depthValue = depthFrameData[i];

// Pick any value that is bigger than before
            if (depthValue > far)
            {
                far = depthValue;
            }

if (depthValue == 0)
            {
                emptyPixels++;
            }
            else
            {
                // Pick any non-empty value that is smaller than before
                if (depthValue < near)
                {
                    near = depthValue;
                }

// Add to total
                total += depthValue;
            }
        }

// Calculate average of all non-empty pixels
        long average = total / (depthFrameData.Length - emptyPixels);

// Reset the nearest value in case it was never set so it looks reasonable on screen
        if (near == short.MaxValue)
        {
            near = 0;
        }

if (textDisplay != null)
        {
            textDisplay.text = "Far: " + far + "\r\n" +
                "Near: " + near + "\r\n" +
                "Average: " + average + "\r\n" +
                "Empty pixels: " + emptyPixels + " out of " + depthFrameData.Length;
        }

MapDepthToTexture(depthFrameData);
    }

private void EnsureBuffers(int width, int height)
    {
        int length = width * height;
        if (textureBuffer.Length != length)
        {
            textureBuffer = new Color[length];
        }

if (depthFrameData.Length != length)
        {
            depthFrameData = new short[length];
        }

if (texture != null)
        {
            if (texture.width != width ||
                texture.height != height)
            {
                texture.Resize(width, height);
            }
        }
    }
    
    void MapDepthToTexture(short[] depthPixels)
    {
        short minMillimeters = 0;
        short maxMillimeters = short.MaxValue;

if(lowRange)
        {
            minMillimeters = (short)(lowRange.value * 1000);
        }

if (lowRangeValue)
        {
            lowRangeValue.text = minMillimeters.ToString();
        }

if (highRange)
        {
            maxMillimeters = (short)(highRange.value * 1000);
        }

if (highRangeValue)
        {
            highRangeValue.text = maxMillimeters.ToString();
        }

int length = depthPixels.Length;
        for (int i = 0; i < length; i++)
        {
            short depth = depthPixels[i];

float depthScaled = 0.0f;
            if (depth != 0)
            {
                depthScaled = 1.0f - (depth / 10000.0f);
            }

if (depth == 0)
            {
                textureBuffer[i] = emptyPixelColor;

}
            else if (depth >= minMillimeters && depth <= maxMillimeters)
            {
                textureBuffer[i] = rangeColor;
            }
            else
            {
                textureBuffer[i].r = depthScaled;
                textureBuffer[i].g = depthScaled;
                textureBuffer[i].b = depthScaled;
                textureBuffer[i].a = 1.0f;
            }
        }

texture.SetPixels(textureBuffer);
        texture.Apply();
    }
}

Then a few more things to complete the layout:

Move or hide the default debug texts
Add a text for showing our distance values
Add two sliders and texts for displaying their values. Range 0-10, one with value 0 and the other with 10
Resize and replace the DepthMapViewer as you wish

Finally, bind the UI elements to our script.

Then the interesting part. Testing time! We'd obviously need two test cases:

An area where we have more than 10 meters of space in front of the sensor
Some objects nearby for short range test... or your hand will do

And against what are we testing? Probably the official numbers provided by Orbbec: 0.6-8 meters. Ok, so if you did your Editor settings correctly, you'd be seeing something like this. It's all yellow for measured pixels and or cyan for empty ones. I've turned the sensor toward a door to the hallway which has way more than 10 meters empty space until the back wall of the next room.

Then let's drag the first slider down to about 3.5 meters. Nicely everything in the background turns to the original grayscale color whereas everything near remains yellow.

Then let's try the nearest pixels. Let the minimum be at 0 and maximum at 860mm. Looks about right. The nearest pixel seems to be at 831mm where my diagonally aligned TV screen appears first in the image.

Then let's try something in the middle. If I go stand at the doorway you see not much more than my silhouette as I set my sliders to make the range to be 5-6 meters. At this point is is getting obvious that the depth pixel farthest from the sensor is constantly at 9851mm.

Now if I drag the minimum range to above 9851mm I get nothing but the original gray and empty pixels. The next room's back wall looks black now as it hit the maximum range of the depth scale.

Let's slice a bit back just below the farthest depth value, here 8767. Now the back wall appears yellow exactly as expected.

How about the close range then? Let's try to block the sensor completely by moving the hand nearer and nearer to the camera. You can just about see my thumbs up gesture in the middle of the image. Nearest pixel is at 472mm.

Now the interesting thing: As my hand gets closer to the sensor, just after 443mm it goes blank. You won't see anything much about my hand anymore at any closer distance.

And then the part where I tell you "do not try this at home". As I keep closing in to cover the sensor altogether, the DLLs go nuts. Some funny unhandled exception in the SDK, I suppose.

Now, if you keep testing like this you'll come to the same conclusion as I did:

The maximum depth pixel value is always 9851, i.e. 9.85 meters
The minimum non-zero depth pixel value is 443 (I tried plotting the numbers to text file as well)

We can conclude that the maximum is solid whereas the minimum is a bit more obscure. But at least we can get almost 10 meter range of the raw values of the depth pixels. Compared to the official range of 0.6-8 meters we do get depth pixel values at 0.44-9.85 meters. But I suppose the reliability is not good below 0.6 and above 8 meters to be officially recommended as the depth measurement range.

Now we know.