Compute Resources

You are here

The CMRR provides an excellent computer support environment for research. The on-site support, physical infrastructure, instrument computers, networking, compute resources, data storage and backup all contribute to a modern, highly reliable, and productive research environment. On-site support. Knowledgeable and experienced computer support staff for Unix, PC, and Mac computers are on site to assist with software and hardware installation, debugging, and process development. Physical infrastructure. Onsite file servers and compute servers are located in a secured server room. The data center in the CMRR is a 750 sq. ft. room with redundant power (100KW Uninterruptible Power Supply and Diesel Generator) and redundant cooling systems (chilled water and glycol). The power and cooling systems are monitored 24x7 to maintain an optimum environment. Instrument computers. Each of the MR instruments at the CMRR has a console host workstation along with physiologic monitoring and paradigm presentation computers. The scanners with Varian consoles (9.4T-31cm, 4T, 16.4T) use Linux workstations while the Siemens consoles (3Ts, 7T/AS & 7T/PS, and 10.5T) use Windows based PCs. At some instruments, Unix workstations are also available for real-time image processing and data storage. Networking. The CMRR is connected to the University of Minnesota and the Internet by a 10Gb uplink and each of the magnet rooms is connected to the server room by a 10Gb connection. Selected high-traffic servers have 10Gb interfaces. The CMRR network currently supports speeds of 1Gb to the desktop for all researchers. The University also manages the local point-of-presence for Internet2, meaning the CMRR uses Internet2 to all Universities when it can. Compute resources. The CMRR has a total of 95 servers online, including compute servers, file servers hosting data, and support servers. We have 148 CPUs, 1006 processing cores, and 4.1Tb of main memory available. Our largest server has 512Gb of memory and 32 CPU cores, and we have 10 more servers with 128Gb+ of memory. We make use of parallel queue management software (SGE, slurm), and 140 CPU cores are available in a single queue for parallel processing. Also, 13 servers contain powerful Nvidia Tesla GPU coprocessors to speed up coil modeling and image reconstruction. Data storage. The CMRR has 595Tb+ of RAID disk storage space for data, used to store DICOM and raw data sets, intermediate results, and other user data. Backups. This data is regularly archived by an automatic backup system using rsync and ZFS backup file server snapshots. A LTO-4 tape robot with 80 slots allow encrypted off-site storage of backups and long-term archives in case of disaster.