使用 Open GL 讓 LIPSedge AE400 3D 相機產生點雲圖

Point cloud is essential to 3D cameras. Let's see how we can populate the point cloud using AE400. In this article, we use AE400 SDK along with OpenGL and GLFW to render the point cloud. We will focus on how to turn 3D frames into point cloud's vertices and image texture. For OpenGL and GLFW, we will provide helper functions for you. You can have more information on their website (OpenGL, GLFW).

Code

* Download full code example from github

1. Let's create a empty project folder. Then create CMakeLists.txt with content below

cmake_minimum_required(VERSION 3.8)

project(ae400-pointcloud CXX)

add_executable( ${PROJECT_NAME} pointcloud.cpp)

set_property(TARGET ${PROJECT_NAME} PROPERTY CXX_STANDARD 11)

find_package( OpenGL REQUIRED )
target_link_libraries( ${PROJECT_NAME} ${OPENGL_LIBRARIES} glfw realsense2)

2. Create an empty file pointcloud.cpp for all the code we need to write. Then Create file example.hpp, copy and paste all the helper script from here.

3. In the main function in pointcloud.cpp. First we need to create the window and setup prepare a glfw_state variable for tracking the app states. Also, register a callback function for tracking the mouse movement to controller pointcloud's view projection.

window app(1280, 720, "Pointcloud Example");
glfw_state app_state;
register_glfw_callbacks(app, app_state);

4. Declare a video pipeline and start the camera. Also, declare pointcloud and points variables to store the pointcloud info.

rs2::pipeline pipe;
pipe.start();
rs2::pointcloud pc;
rs2::points points;

5. Whilte the video is streaming, get the color and depth frame from AE400.

while (app) 
{
    auto frames = pipe.wait_for_frames();
    auto color = frames.get_color_frame();
    auto depth = frames.get_depth_frame();
 }

6. On each frame captured, we first make pointcloud to map the color frame. Then calculate the depth frame and texture. To see what texture should be use at each vertices in depth frame. Pointcloud class has map and calculate function for that.

while (app)
{
   ...

   pc.map_to(color);
   points = pc.calculate(depth);
 }

7. With the helper script, we update the texture for openGL to use.

while (app)
{
   ...

   app_state.tex.upload(color);
}

8. Finally we use the vertices and texture to draw the pointcloud.

while (app)
{
   ...

   draw_pointcloud(app.width(), app.height(), app_state, points);
}

Let's take a closer look at draw_pointcloud function. After setting some openGL draw attributes, use glVertex3fv() and glTexCoord2fv() to actually draw the points.

Compile and Run

cd pointcloud 
mkdir build 
cd build 
cmake .. 
make 
./ae400-pointcloud

Here's the pointcloud viewer. You can use mouse to control the rotation or zoom in/out. Zoom in for a closer view you can see all these tiny points in it.

Full Example

You can referencethe full example in the original article in LIPS Help Center. If you have any questions, please post them to LIPS Forum.