Center for Magnetic Resonance Research, Department of Radiology
Remote Assisted Image Reconstruction for Multi-Band Accelerated Sequences
You are here
An optional remote reconstruction server is available to improve the reconstruction speed for especially demanding studies. It allows for the possibility of offloading the multi-band image reconstruction calculations to a separate server (with a CUDA GPU or dual Intel Xeon 5500/5600 or better CPUs) via a local network connection.
Since modification of the MRIR/MARS and custom Linux networking configuration is required and we do not have the resources to assist/support this, it is only recommended for experienced users, or users with access to local Linux support resources.
Some example timings for reconstructions of a sample MB8 accelerated acquisition (HCP resting-state FMRI protocol, truncated to 200 repetitions for 153 seconds total duration) with different hardware configurations are shown below:
Figure 1. Remote assisted reconstruction speed comparison of various hardware configurations for VB17A (Avanto/Trio/Verio Step4 MRIR).
Reconstruction timings measured on a VB17A Trio system with a Step4 MRIR are shown in Figure 1. In this case it can be seen that a 153 s acquistion that normally would take 368 s to reconstruct using the local MRIR can be reconstructed in 163 s using remote reconstruction with a Tesla C2050 (Fermi) or better GPU, connected via a 10 Gigabit ethernet link. Using a 1 Gigabit link only provides a marginal improvement as it is easily saturated. Remote CPU recon (over 10 GbE) is also significantly faster than the local MRIR recon in this case, owing to the significantly faster CPUs in the remote server (Xeon X5680 vs. E5440).
Figure 2. Remote assisted reconstruction speed comparison of various hardware configurations for VD11D (Aera/Skyra 64-channel MARS).
Reconstruction timings measured on a VD11D Skyra system with the 64-channel MARS are shown in Figure 2. Compared to the VB17A Step4 MRIR, the online reconstruction is significantly faster due to the faster CPUs in the MARS (Xeon vs. E5440). Indeed in this case using remote recon over a 1 Gigabit link is slower than using the local CPUs, since the 1 GbE link becomes a bottleneck. With remote GPU recon over a 10 Gigabit link the reconstruction completes in real time.
The default image processing pipeline for VB17A (Avanto, Trio, Verio, et al.) is shown in Figure 3. Acquired data are received by the MRIR, stored to a local (RAID) buffer, and processed in parallel using multi-core CPUs. The fastest current VB17A MRIR (Step4) has dual quad-core Intel Xeon CPUs and either 16 or 32 GB of RAM. Previous models (Step2, Step3) originally delivered prior to VB17A had dual single-core AMD Opteron CPUs and either 8 or 16 GB of RAM.
Figure 3. Standard VB17A image reconstruction pipeline (black boxes = Siemens product methods).
The image processing pipeline for VD11D/VD13A (Aera, Skyra, Prisma) is essentially similar to what is presented in Figure 3, however the MPCU and MRIR have been combined and replaced by the MARS, which has somewhat faster dual quad-core Intel Xeon CPUs and 32-128 GB of RAM depending on the number of receiver channels purchased with the system. The 64-channel and 128-channel models also include a single NVIDIA Tesla C1060 GPU.
The calculations for multi-band unaliasing are CPU-intensive (dealing with large matrices), which places a substantial additional load on the MRIR, which is already heavily burdened by standard reconstruction tasks. When high multi-band acceleration factors are used to achieve very short TR, this increases the load on the MRIR to the point where it can no longer reconstruct images in real time even without the extra load of the unaliasing calculations. When multi-band unaliasing is performed, reconstruction durations can exceed the acquisition duration several times over.
One solution to this problem which does not require replacing MRIR/MARS hardware is to send the aliased multi-band data to another computer, which performs the unaliasing calculations and returns the unaliased data. This pipeline is shown in Figure 4.
Figure 4. VB17A image reconstruction pipeline with remote assisted multi-band unaliasing (black boxes = Siemens product methods).
Offloading the multi-band unaliasing calculations to another computer reduces the load on the local MRIR, and since the remote computer is not supplied by Siemens it is possible to use significantly newer, faster hardware. Xeon E5 CPUs and NVIDIA Kepler GPUs, for example, are supported by the remote reconstruction server software.
The link between the MRIR and the remote reconstruction computer can be configured in three possible ways:
- Scanner internal network. The MRIR, MRC, and MPCU communicate over a private 1 Gigabit network. The switch for this
network is typically located in the scanner control room near the MRC (VB17A systems only). It is possible to simply connect another computer to this
switch and use it for remote recon. This is the easiest option, but it is not recommended. For the purpose of multi-band
unaliasing, a 1 Gigabit network is bandwidth limited, and this network is also used to send reconstructed image data
and feedback between the MRIR and MRC. Competing with these data streams is not ideal. This is not available for VD11D/VD13A
systems (unless a switch is added).
- MRIR built-in secondary 1 Gigabit ethernet port. All VB17A MRIR models have a built-in secondary 1 Gigabit ethernet
port which is normally unused. It is possible to use this as a dedicated link to the remote reconstruction server by simply
connecting a cable to the unused port. Using a crossover cable or an unmanaged switch to connect directly to the server is
recommended in this case, since a longer path through routers and firewalls will introduce costly latency. 1 Gigabit
ethernet will nevertheless be bandwidth limited for most protocols. Note that this port does not exist on VD11D/VD13A systems (MARS).
- Dedicated 10 Gigabit link. For users who are comfortable opening the MRIR/MARS chassis, it is easily possible to
add a single 10 Gigabit ethernet card to a Step4 MRIR or any MARS. This will allow the reconstruction to run at maximum
possible speed. Step-by-step instructions and drivers for installing and configuring a Myricom 10G-PCIE-8B-S 10 GbE adapter
are provided with the server software distribution. This is by far the preferred option; in combination with a Fermi or
Kepler GPU in the remote recon server, the time needed for multi-band unaliasing calculations becomes negligible.
Reconstruction speed will only be limited by the Siemens product reconstruction running on the local reconstructor.
For VD11D/VD13A systems this can be real time or better; for VB17A may be somewhat slower due to the slower MRIR.
NOTE: Check with local Siemens service before modifying any scanner hardware! This information is provided in the hope that it will be useful, but reliance upon any opinion, advice, statement, memorandum, or information presented here shall be at your sole risk.