Getting Started with Triton Smart and Brain Builder

Prerequisites#

This document describes how to get started using the LUCID Triton Smart camera (TRS123S-CC) together with Brain Builder for AITRIOS software and LUCID’s ArenaView MP.

Software:

LUCID Software Requirements

  • Arena SDK with ArenaView MP (Windows beta for Triton Smart) OR ArenaView MP (Lite version–Windows beta for Triton Smart)
  • Triton Smart Support Package

External Software Requirements

  • Brain Builder for AITRIOS
  • CodeMeter

PC Requirements:

  • OS: Windows 10/11 (Linux support coming soon)

Camera:

Software Descriptions:

Software / FileVendorDescriptionDownload Link
Arena SDK with ArenaView MP (Windows beta for Triton Smart)LUCIDArena SDK (Full APIs) with AreanView MP. Full SDK package for use exclusively with Triton Smart.Triton Smart "Getting Started / Downloads" Tab
ArenaView MP (Lite version–Windows beta for Triton Smart)LUCIDArenaView MP only. Windows only, for use exclusively with the Triton Smart. (Use this if you already have Arena SDK installed)Triton Smart "Getting Started / Downloads" Tab
Triton Smart Support PackageLUCIDContains C++ / Python code examples, example AI models, sensor firmware. Also contains Disp_InputTensor.exe Windows utility for capturing input tensors on the Triton Smart.Triton Smart "Getting Started / Downloads" Tab
Brain Builder for AITRIOSSonySoftware to train and export AI models for the Triton Smart.Brain Builder for AITRIOS
CodeMeter WIBU SystemsLicense manager for Brain Builder. You will need this for both trial and paid licenses for Brain BuilderWIBU Systems: CodeMeter

Installing ArenaView MP & support package#

You cannot install the Arena SDK with ArenaView MP package if you already have the Arena SDK with ArenaView installed. However, if you have the Arena SDK with ArenaView installed, you can install the Lite version of ArenaView MP.

Visit the Triton Smart Product Page, scroll down, and click the “Getting Started / Downloads” Tab. Download the Triton Smart Software Package and one of the ArenaView MP packages (login required).

Triton Smart ArenaMP software

If you have the previous version of Arena SDK installed on your PC, install the the Lite version of ArenaView MP. The version will only install the ArenaView MP GUI and is compatible with the previous Arena SDK version v1.0.55.11.

If you do not have a previous installation of Arena SDK on your PC,  install the “Arena SDK with ArenaView MP”. This will installed the full beta SDK, including ArenaView MP GUI. Be sure to run the installer with Developer options. Note that this SDK is designed solely for use with Triton Smart.

Installing Brain Builder and CodeMeter#

Currently only Windows 10 / 11 platforms are supported for Triton Smart. Download the Windows versions of both Brain Builder and CodeMeter following the steps below.

Installing CodeMeter#

Brain Builder requires CodeMeter to manage its software licenses. This includes the free 30-day license. To run Brain Builder, you must download and install CodeMeter. Visit WIBU’s CodeMeter download site to download and install the software https://www.wibu.com/support/user/user-software.html#CodeMeter_User_Download

Codemeter software screenshot

Screen shot of CodeMeter User Runtime for Windows download (https://www.wibu.com/support/user/user-software.html#CodeMeter_User_Download)

Installing and Activating Brain Builder#

Visit Sony’s Brain Builder for AITRIOS website and download the appropriate version for your PC platform: https://developer.aitrios.sony-semicon.com/en/studio/brain-builder#downloads Scroll down to see the different download options. The download packages on this site contains a trial license of Brain Builder for AITRIOS for 30 days free of charge.

Build Builder Download selection

Screen shot of Brain Builder versions provided by Sony (https://developer.aitrios.sony-semicon.com/en/studio/brain-builder#downloads). Download either the Windows CPU or GPU version

Once you have downloaded the Brain Builder ZIP package, unzip the package and install the Neurala Brain Builder EXE file – Neurala_BrainBuilderForAITRIOS_##.##.#_Windows_CPU/GPU_amd64.exe.

 

Activate Trial License

Once installation is complete, you will need to run the trial license CodeMeter file located within the Brain Builder Zip file (.WibuCmRaU file). This file will run CodeMeter and after it is complete, you will have full-access to Brain Builder.

Run Trial codemeter File

Prepare the camera for the first time#

The Triton Smart requires an AI model to be loaded before it can produce an input tensor. The Triton Smart Support Package (ZIP) includes pre-trained AI models that you can load onto the camera to start image tensor capture.

About the sample AI models locations:

The Triton Smart requires an AI model to be loaded before it can produce an input tensor. The input tensor characteristics depend on the type of AI model that is loaded:

  • Classifier AI model uses a 256 x 256 px input tensor
  • Detector AI model uses a 320 x 320 px input tensor
  • Anomaly Detection uses: 512 x 512 px for Fine / Ultrafine,  256 x 256 px for Normal / Coarse

The Triton Smart Support package ZIP file contains pre-trained models that adhere to the above specifications. You need to upload the specific pre-trained model that corresponds to your planned model type.

  • Sample Classifier AI model is located in \Networks\BrainBuilder\CL_cat_dog\BB_output\
  • Sample Detector AI model is located in \Networks\BrainBuilder\OD_cat_dog\BB_output\
  • Sample Anomaly Detection model is located in \Networks\BrainBuilder\AH_hazelnuts)\BB_output
Pre-trained ai-models

You will need the following files:

File:Location:
network_info.txt (Used to create fpk_info.dat)TritonSmart_Support_Package_v#.#.#\Networks\BrainBuilder\***\BB_output
BB_network.fpkTritonSmart_Support_Package_v#.#.#\Networks\BrainBuilder\***\BB_output
labels.txtTritonSmart_Support_Package_v#.#.#\Networks\BrainBuilder\***\BB_output
loader.fpkTritonSmart_Support_Package_v#.#.#\imx501_files
firmware.fpkTritonSmart_Support_Package_v#.#.#\imx501_files

*** = Choose one of the three folder to use.

Convert network_info.txt to fpk_info.dat#

  1. Power on the camera and connect it in ArenaView MP by toggling the switch at the top left.
  2. In ArenaView MP, locate the “Controls” panel menu and select Camera Features > Smart Model Installation.
  3. Specify network_info.txt in the “NETWORK INFO CONVERSION” section. You can find this file in located in the \TritonSmart_Support_Package_v#.##\Networks\BrainBuilder\(OD_cat_dog, CL_cat_dog, or AH_hazelnuts)\BB_Output\ folder
  4. Click the “Convert” button to save the converted fpk_info.dat in your specified working directory.

Note: You must set a working directory for ArenaView MP to convert network_info.txt to fpk_info.dat. If you have not set your working directory, go to the Controls menu and select “Save and Load”.

Upload the files to the Camera#

  1. Go to the “FILE UPLOAD” section and specify the firmware.fpk, loader.fpk in \TritonSmart_Support_Package_v#.#.#\imx501_files\
  2. Specify the converted fpk_info.dat in your specified working directory.
  3. Specify the BB_network.fpk and labels.txt in \TritonSmart_Support_Package_v#.##\Networks\BrainBuilder\(OD_cat_dog, CL_cat_dog, or AH_hazelnuts)\BB_Output\ folder
  4. Finally, click the Upload button to transfer the files to the camera.
Triton Smart File Upload locations Examples

Acquiring input tensor images for dataset#

Once you have uploaded all the files on the Triton Smart, you can start capturing images for your dataset. The Triton Smart Support Package includes the executable Disp_InputTensor.exe which can be used to capture images tot the correct specifications based on the sample AI model you loaded. The file is in the folder \TritonSmart_Support_Package_v#.#.#\Bin\ .

Disp_InputTensor screenshot

The Windows Firewall warning screen displays when you run it for the first time. You must allow access for the program to run. If you are unable to allow access, contact your IT administrator.

You must toggle the camera off in ArenaView MP before running Disp_InputTensor.exe. If you do not, ArenaView MP will not release the camera and other program will not have access to the camera.

Double-click Disp_InputTensor.exe to acquire images. Use the Hotkey “S” to save images.

Using Brain Builder for AITRIOS#

Brain Builder for AITRIOS has multiple versions. The Trial version includes access to Anomaly Recognizer, Classifier, and Detector. Brain Builder Basic version includes access to only Classifier and Detector.

The Brain Builder for AITRIOS package includes sample datasets located in the \SampleDatasets\ folder. The video below uses the \JapaneseCoins_Detector\ sample dataset image files.

Create your project and upload your dataset#

  1. Click Create new Project and label your project
  2. Click Create new Dataset and label your dataset
  3. Select your model type. In the video example below choose Detector.
  4. Select and import your images for your dataset.

Create your classes and label your images#

There are two ways to create a class, which is a label used to categorize similar objects or object features. You can add classes by clicking the “Manage Classes” button in the Dataset main window. You can also add new classes by clicking the “+” button on the top right of the label annotation window. In the video example below, both ways of adding classes is shown.

Training your model#

Once you have completed labeling all your images you can now train your model.

  1. Select Train on the left side menu.
  2. Select the desired comprehensiveness of your model. It is recommended to choose “Quick” for proof-of-concept for the first time. If you are satisfied with the results, retrain your model using “Thorough“.
  3. Click “Train“.
Training Brain Builder AI model

Exporting your model#

Once your model has finished training you can export the model (ZIP) file from Brain Builder. The file name will be same as your Project Name.

  1. Select Export on the left side menu.
  2. Click “Download Brain“.
Export Brain Builder Trained Model

Upload trained model to Triton Smart#

This section is similar to how we previously uploaded a sample model to the Triton Smart.

  1. Unzip and extract all files from your exported Brain Builder file, e.g. “Your_Project_Name-imx500.zip”.
  2. In the extracted folders, find the ZIP file with a name similar to your dataset label, e.g. “Your-Dataset-Label.zip”.
  3. Unzip and extract all files from “Your-Dataset-Label.zip”.
  4. Enter the extracted folder and find labels.txt, network.fpk, and network_info.txt.
  5. Open ArenaView MP, and toggle on the Triton Smart.
  6. Convert network_info.txt to fpk_info.dat as shown above
  7. Upload the files to the camera: follow the same steps as defined in “Preparing your camera for the first time” section. However, instead of using the sample files, use the files located in your extracted ZIP files. (Reminder: Firmware.fpk and loader.fpk can be found in the Triton Smart Support Package in the folder: \imx501_files\)

Unzip the files to find labels.txt, network.fpk, and network_info.txt

Run AI model on camera#

Once all files have been loaded onto the camera, click the Stream button in ArenaView MP to stream the camera. The bounding boxes can be enabled/disabled through the “Smart Display Settings” Panel (Controls -> Image Processing -> Smart Display Settings)

You can use ArenaView MP to run the AI model live.