App Note: Using the Helios2 Chroma

Introduction

This App Note contains instructions on using Helios2 Chroma. This unit is made up of a Helios2 camera and a Triton camera, and allows a user to combine a color image with a 3D point cloud. This unit comes pre-calibrated from factory, and requires no user calibration. 

Requirements

Helios2 Wide Chroma RGB 3D Camera

Items used for this App Note:

  • Helios2 Wide Chroma (HTW-RGB-001)
  • 2x M12 to RJ45 IP67 Cat6a cables
  • M8 8-Pin GPIO AC Adapter Power Supply (needed for the Helios2 Wide only)
  • LUCID 2-CH 10G/1G RDMA PoE+ HCA Network Card
  • Arena SDK with ArenaView MP v1.0.65.17 or higher

Using the Helios2 Chroma with ArenaView MP (AVMP)

Device connection

On connecting, Helios2 Chroma will show up in ArenaView MP (AVMP) as shown below:

If you click on the drop-down arrow to the left of the camera name, you can see the Helios2 and Triton cameras that make up this Helios2 Chroma. In our case, these are the TRI122S-C and HTW003S-001.

Starting the stream

When you click ‘Start Stream’, AVMP shows a stream of a colored point cloud, as shown below:

The individual camera framerates can be adjusted in the Features pane, and the displayed framerate of Helios2 Chroma is the lower of these two framerates (if they are not equal).

Saving Images

AVMP lets you save the colored point cloud from Helios2 Chroma in PLY format. Additionally, you have the option to save the corresponding RGB image from the Triton and PLY image from Helios (without color), together with a 2D image of the resulting overlay.

To save a colored point cloud, go to Controls > Save Image > Save & Load. Here, you will have the option to save the Depth image (as PLY), RGB image and the Colored Point Cloud 2D (in a file format of your choice) together with the Colored Point Cloud after clicking Save.

Decoupling the Helios2 Chroma

If you wish to use the Helios2 and Triton cameras separately, you will need to decouple them first.  

To do so, in the Controls pane, go to Camera Features -> Helios-Chroma. Toggle ‘Decouple Helios-Chroma Devices’.  

Using Helios2 Chroma with the Arena SDK

The Arena SDK includes the following examples made specifically for the Helios2 Chroma:

  • Cpp_HeliosChroma_Overlay, and
  • Cpp_HeliosChroma_Overlay_SDK

Cpp_HeliosChroma_Overlay

This example combines one RGB image from the Triton with one point cloud image from the Helios2 and uses the calibration information already stored on both cameras to combine the images into a colored point cloud using OpenCV’s projectPoints function.

However, before capturing images on either camera, the program first checks whether both the Helios2 and Triton camera belong a ‘DeviceGroup’, and whether they belong to the same DeviceGroup (by comparing their DeviceGroupSerialNumber). If either of these checks fail, the program will not proceed.

These settings are checked as shown below:

bool isApplicableDeviceTriton(Arena::DeviceInfo deviceInfo)
{
	// color Triton device with device group membership needed
	return deviceInfo.ModelName().find("TRI") != GenICam::gcstring::npos && deviceInfo.ModelName().find("-C") != GenICam::gcstring::npos && deviceInfo.DeviceGroupEnable();
}
 
bool isApplicableDeviceHelios(Arena::DeviceInfo deviceInfo)
{
	// helios device with device group membership needed
	return (deviceInfo.ModelName().find("HLT") != GenICam::gcstring::npos || deviceInfo.ModelName().find("HT") != GenICam::gcstring::npos) && deviceInfo.DeviceGroupEnable();
}
 
bool isApplicableDeviceGroup(Arena::DeviceInfo tritonInfo, Arena::DeviceInfo heliosInfo)
{
	// Helios and Triton devices need to be in the same device group
	return tritonInfo.DeviceGroupSerial() == heliosInfo.DeviceGroupSerial();
}

If the cameras belong to the same DeviceGroup, the program proceeds to obtain the calibration information stored on the Triton and Helios2 and stores them in a struct. Next, the program uses this struct and OpenCV’s projectPoints to combine the Triton image and the Helios2 image and save it as PLY file.

Cpp_HeliosChroma_Overlay_SDK:

This example performs a similar operation as the previous example except that it uses LUCID’s Overlay API instead. This API offers functions that wrap some of the functionality presented in the previous example, which simplifies the process of obtaining overlayed point clouds on Helios2 Chroma.

After ensuring both the cameras belong to the same DeviceGroup and obtaining an image from each camera, this example first uses Arena::ImageFactory::CreateOverlayInfo to obtain all calibration data from the cameras. This information is then used by Arena::ImageFactory::Overlay to output the colored point cloud as an Arena::IImage*; that is then saved as a PLY file.

These functions are used in the following way in the example:

// get overlay information from the devices
std::cout << TAB1 << "Get Overlay information\n";
 
Arena::OverlayInfo* pOverlayInfo = Arena::ImageFactory::CreateOverlayInfo(pTriton, pHelios);
 
// overlay RGB color data onto 3D XYZ points and save the resulting image
std::cout << TAB1 << "Overlay the RGB color data onto the 3D XYZ points\n";
 
Arena::IImage* pImageOverlay = Arena::ImageFactory::Overlay(pImageHelios, pImageTriton, pOverlayInfo);

Sample Images

Below are sample images obtained using Cpp_HeliosChroma_Overlay_SDK.

The point clouds were opened in Meshlab, which is an open-source tool for viewing 3D meshes.

RGB image from Triton (file: Cpp_HeliosChroma_Overlay_SDK_triton.jpg).

Depth image opened in Meshlab (file: Cpp_HeliosChroma_Overlay_SDK_helios.ply).

Overlay 2D image (file: Cpp_HeliosChroma_Overlay_SDK.jpg).

The overlay image opened in Meshlab (file: Cpp_HeliosChroma_Overlay_SDK).