Timeouts and incompletes usually happen because of a network disruption causing a flood of resend. If resends occur, either the camera did not transmit the whole image, or the host PC did not successfully receive the complete image. Below are the troubleshooting steps to avoid or eliminate the cause of resends.
Check the DeviceLinkSpeed in ArenaView
The following values should be reported in the DeviceLinkSpeed node if your camera is properly configured:
For 1Gbps, DeviceLinkSpeed = 125000000 Bps
For 5Gbps, DeviceLinkSpeed = 625000000 Bps
For 10Gbps, DeviceLinkSpeed = 1250000000 Bps
In case the DeviceLinkSpeed is not reporting correct value, make sure that ethernet adapters Speed & Duplex is set to Auto Negotiation or try a different port or a cable.
Re-enable all cores on your system if they are limited
run -> msconfig -> Boot -> Advanced Options
Uncheck the box for Number of Processors and click ok in the dialogue box.
Set the maximum jumbo packets and receive buffer on the adapter
Right click on the Ethernet Adapter -> Properties -> Configure -> Advanced
Set Jumbo Packet to 9014 Bytes
Set Receive Buffer to 4096
Increase buffer size in the SDK using StartStream(100)
start_stream(number_of_buffers=None)
Causes the device to begin streaming image/chunk data buffers. It must be called before image or chunk data buffers are retrieved via device.get_buffer() otherwise, a BaseException will rise.
Args:
number_of_buffers :
Number of internal buffers to use in the acquisition engine. The default value is None, and the minimum accepted value is 1. It can be:
- a positive integer. Relatively small numbers are recommended. Zero or a negative int values will raise
ValueErrorexception. None. This is the default value, which is equivalent todevice.start_stream(device.DEFAULT_NUM_BUFFERS).
Raises:
ValueError:number_of_buffersis zero or a negative intger.
TypeError:number_of_bufferstype is not int.
Returns:
- None
Turn off power Management on the NIC
Check power settings in the BIOS
- Ensure PC power saving feature is off
- In BIOS, Disable SpeedStep
- In BIOS, Disable C-States
Missing data/incomplete images with ArenaSDK can occur when EIST and C-State are enabled in the systems power management settings.
C-states are states when the CPU has reduced or turned off selected functions.
Enhanced Intel SpeedStep Technology gives your operating system the ability to switch the processor’s speed and voltage up and down, to preserve power when not much is being computed.
Both C-states and EIST power saving settings should be disabled to resolve the missing data issue with ArenaSDK.
Boot into the systems BIOS settings –
Windows 10 Operating System
Example PC Specifications: Supermicro X10DRi motherboard with Intel Xeon E5 2620v4
Settings → Update and Security → Recovery → Advanced Startup → Restart Now
The computer will boot into a special menu.
Troubleshoot → Advanced Options → UEFI Firmware Settings → Restart
Once in the BIOS settings, there should be an explicit option for “Intel SpeedStep” or “EIST” that can be disabled.
Then under CPU Power Management in the BIOS, to disable CPU C-States change Package C State Limit to C0/C1 state and deactivate CPU C3 Report, CPU C6 Report and Enhanced Halt State (C1E).
Disable Audio
We have found that sometimes window sounds can effect the network:
- Disable Audio
- Uncheck “Allow Notifications to play sounds”
- Set NetworkThrottlingIndex to 0xFFFFFFFF from registry editor. Please note you MUST reboot after this is modified
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Multimedia\SystemProfile\NetworkThrottlingIndexto 0xFFFFFFFF.
Then check the lucidlwf.log and a PktMon log floods of resends just before a timeout or incomplete:
We will be able to use Microsoft’s utility PktMon. This utility is built into Windows 10 as of October 2018, version 1809. This utility is not available on earlier versions of Windows 10 or Windows 7.
Saving a PktMon Log
- Open Command Prompt with administrator permission
- (Optional) Navigate to a convenient directory
- Start recording a log with PktMon:
1pktmon start --etw
- Perform camera actions
- Stop recording a log with PktMon:
1pktmon stop
The above steps would record traffic on ALL interfaces, which may not be desired.
Recording on a specific interface
- Use the following command to list your computer’s interfaces
1pktmon comp list
- Find your desired NIC and take note of the interface’s ID:
- Record with pktmon on the desired interface ID:
- Use the following command to convert the etl to pcapng format
1pktmon pcapng PktMon.etl -o output.pcapng
The PktMon utility from Windows 10 May 2020 (version 2004) or higher is REQUIRED to perform this conversion step.
- Open your new pcapng file in Wireshark and you’ll find GVSP packets again
Other Notes
- Use the -f parameter to output to a different filename (default is PktMon.etl)
1pktmon start --etw -f myoutput.etl
- Use the -p parameter to capture a different packet size (default is 128 Bytes, -p 0 will capture the full packet size)
1pktmon start --etw -p 512
- Use the -s parameter to change the ETL file size (by default the circular buffer is 512MB)
1pktmon start --etw -s 1024
- Use the -m parameter to make PktMon create a new file when the current ETL file is full
1pktmon start --etw -m
- In Windows 10 2004 and Windows 10 20H2, use -l multi-file parameter (instead of -m) to make PktMon create a new file when the current ETL file is full.
Increase DeviceLinkThroughputReserve and PacketResendWindowFrameCount
There is a node called PacketResendWindowFrameCount which decides how far back resends can be requested for images. By default the number is 4 which means SDK will attempt to request packets for as far back as frame n-3 while receiving the current frame n. Please keep in mind that there still has to be enough bandwidth reserve within that window for the camera to actually send those packets
Camera uses reserve bandwidth to resend the packets. By default DeviceLinkThroughputReserve is set to 10%.
What you can try is to increase the PacketResendWindowFrameCount to 8 and slightly increase the reserve bandwidth. You can first try with 10% and then slowly increase or decrease.
Dual Channel Memory configuration
We recommend dual channel memory configuration for your RAM.
Below is an example on how you want to populate RAM slots to make use of dual channel. In the below example DDR4_A1 and DDR4_B2 will get first priority. Check the manual of your motherboard to make sure you are using the dual channels.
