Troubleshooting timeouts caused by incomplete images (Windows)

Re-enable all cores on your system if they are limited

run -> msconfig -> Boot -> Advanced Options

Uncheck the box for Number of Processors and click ok in the dialogue box.

 

 

 Set the maximum jumbo packets and receive buffer on the adapter

Right click on the Ethernet Adapter -> Properties -> Configure -> Advanced

Set Jumbo Packet to 9014 Bytes

Set Receive Buffer to 4096

 

 

 

Increase buffer size in the SDK using StartStream(100)

start_stream(number_of_buffers=None)

Causes the device to begin streaming image/chunk data buffers. It must be called before image or chunk data buffers are retrieved via device.get_buffer() otherwise, a BaseException will rise.

Args:

number_of_buffers :

Number of internal buffers to use in the acquisition engine. The default value is None, and the minimum accepted value is 1. It can be:

 

1. 1. 1. a positive integer. Relatively small numbers are recommended. Zero or a negative int values will raise ValueError exception.

   
   

   1. None. This is the default value, which is equivalent to device.start_stream(device.DEFAULT_NUM_BUFFERS).

Raises:

 

1. 1. 1. ValueError :
         • • number_of_buffers is zero or a negative intger.

   
   

   1. TypeError :
      • • number_of_buffers type is not int.

Returns:

 

1. 1. None

 

 

Turn off power Management on the NIC

 

 

Check power settings in the BIOS

 

1. 1. 1. Ensure PC power saving feature is off

   
   

   1. 1. In BIOS, Disable SpeedStep

   
   

   1. In BIOS, Disable C-States

Missing data/incomplete images with ArenaSDK can occur when EIST and C-State are enabled in the systems power management settings.

C-states are states when the CPU has reduced or turned off selected functions.

Enhanced Intel SpeedStep Technology gives your operating system the ability to switch the processor’s speed and voltage up and down, to preserve power when not much is being computed.

Both C-states and EIST power saving settings should be disabled to resolve the missing data issue with ArenaSDK.

Boot into the systems BIOS settings –

Windows 10 Operating System

Example PC Specifications: Supermicro X10DRi motherboard with Intel Xeon E5 2620v4

Settings → Update and Security → Recovery → Advanced Startup → Restart Now

The computer will boot into a special menu.

Troubleshoot → Advanced Options → UEFI Firmware Settings → Restart

Once in the BIOS settings, there should be an explicit option for  “Intel SpeedStep” or “EIST” that can be disabled.

 

Then under CPU Power Management in the BIOS, to disable CPU C-States change Package C State Limit to C0/C1 state and deactivate CPU C3 Report, CPU C6 Report and Enhanced Halt State (C1E).

 

 

 Disable Audio

We have found that sometimes window sounds can effect the network:

 

• • • Disable Audio

  
  

  • • Uncheck “Allow Notifications to play sounds”

  
  

  • Set NetworkThrottlingIndex to 0xFFFFFFFF from registry editor. Please note you MUST reboot after this is modified
    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Multimedia\SystemProfile\NetworkThrottlingIndex to 0xFFFFFFFF.

 

Then check the lucidlwf.log and a PktMon log floods of resends just before a timeout or incomplete:

 

We will be able to use Microsoft’s utility PktMon. This utility is built into Windows 10 as of October 2018, version 1809. This utility is not available on earlier versions of Windows 10 or Windows 7.

 

Saving a PktMon Log

 

• Open Command Prompt with administrator permission
• (Optional) Navigate to a convenient directory

• Start recording a log with PktMon:

1pktmon start --etw

• Perform camera actions
• Stop recording a log with PktMon:

1pktmon stop

The above steps would record traffic on ALL interfaces, which may not be desired.

 

Recording on a specific interface

 

• Use the following command to list your computer’s interfaces

1pktmon comp list

• Find your desired NIC and take note of the interface’s ID:

• Record with pktmon on the desired interface ID:

• Use the following command to convert the etl to pcapng format

1pktmon pcapng PktMon.etl -o output.pcapng

The PktMon utility from Windows 10 May 2020 (version 2004) or higher is REQUIRED to perform this conversion step.

• Open your new pcapng file in Wireshark and you’ll find GVSP packets again

 

Other Notes

 

• Use the -f parameter to output to a different filename (default is PktMon.etl)

1pktmon start --etw -f myoutput.etl

• Use the -p parameter to capture a different packet size (default is 128 Bytes, -p 0 will capture the full packet size)

1pktmon start --etw -p 512

• Use the -s parameter to change the ETL file size (by default the circular buffer is 512MB)

1pktmon start --etw -s 1024

• Use the -m parameter to make PktMon create a new file when the current ETL file is full

1pktmon start --etw -m

• In Windows 10 2004 and Windows 10 20H2, use -l multi-file parameter (instead of -m) to make PktMon create a new file when the current ETL file is full.

 

Increase DeviceLinkThroughputReserve and PacketResendWindowFrameCount

 

There is a node called PacketResendWindowFrameCount which decides how far back resends can be requested for images. By default the number is 4 which means SDK will attempt to request packets for as far back as frame n-3 while receiving the current frame n. Please keep in mind that there still has to be enough bandwidth reserve within that window for the camera to actually send those packets.

Camera uses reserve bandwidth to resend the packets. By default DeviceLinkThroughputReserve is set to 10%.

What you can try is to increase the  PacketResendWindowFrameCount to 8 and slightly increase the reserve bandwidth. You can first try with 10% and then slowly increase or decrease.

 

Dual Channel Memory configuration

We recommend dual channel memory configuration for your RAM.
Below is an example on how you want to populate RAM slots to make use of dual channel. In the below example DDR4_A1 and DDR4_B2 will get first priority. Check the manual of your motherboard to make sure you are using the dual channels.