DCMTK and 10 Gb/s

During testing on 10 Gb/s network with the tools mentioned in the previous post I experienced very low bandwidth usage during the tests: ~ 1.5 Gb/s instead of the 8 Gb/s I measured with IPerf and with my own Java tool for network transfer speed measurement.

Investigation showed that DCMTK unfortunately always set TCP send and receive buffers by setsoc

bufLen = 65536; // a socket buffer size of 64K gives best throughput for image transmission
...
setsockopt(sock, SOL_SOCKET, SO_SNDBUF, (char *) &bufLen, sizeof(bufLen));    (void) setsockopt(sock, SOL_SOCKET, SO_RCVBUF, (char *) &bufLen, sizeof(bufLen));

This was probably good approach before TCP autotuning was introduced, but since it is available in newer kernels (>2.6) setting SO_SNDBUF or SO_RCVBUF is switching off the TCP autotune feature that results terrible performance over 10 Gb/s (see https://www.psc.edu/index.php/networking/641-tcp-tune)


When I commented out these setsockopt lines in
./dcmnet/libsrc/dulfsm.cc
./dcmnet/libsrc/dul.cc
the newly built tools could reach ~5.2 Gb/s bandwidth utilization.

DCMTK adds 1 sec delay

When measured transfer speed with a simple dcmtk server (C-MOVE SCP/C-STORE SCU) and client (CMOVE SCU/C-STORE SCP) I was surprised to see that there is always a 1 sec delay before the actual C-STORE transfer starts.

Checking the source code revealed that in dimmove.c at selectReadable function:

timeout = 1; /* poll wait until an assoc req or move rsp */

then invoke with assoc and subassoc in the assocList:
ASC_selectReadableAssociation(assocList, assocCount, timeout))

that leads to DcmTransportConnection::fastSelectReadableAssociation
where it instead of a single select on the assoc and the subassoc it does a select for first on the association, then after a 1 sec delay on the subassociation.

This one second timout for select that is experienced as 1 sec extra if someone relies on the callback implementation for the C-STORE SCP on the client side.

When I implemented the C-CTORE SCP in its separate thread (and not using the callback in the DIMSE_moveUser) obviously this 1 sec delay was removed.