By Patty Giorgio, ITS Communications
Research at the University of Michigan is a $1.3 billion enterprise, according to the 2014 Annual Report on Research. An increasing amount of this research involves moving large data sets, between researchers locally, nationally, and internationally. The network at U-M is able to meet the needs.
Research applications have unique network requirements, often in terms of significantly increased end-to-end bandwidth, but sometimes involving latency or jitter bounds that differ from what is needed for networks utilized for normal business operations. Another difference is that networks designed for business typically require significant security infrastructure to protect business services and desktop applications. These security measures cause problems for high performance research applications. For this reason, the U.S. Department of Energy’s Energy Sciences Network (ESnet) created Science DMZ (DMZ stands for demilitarized zone).
ESnet has designed the Science DMZ architecture with equipment, configuration, and security policies that are optimized for high-performance scientific applications. In FY14 U-M deployed a Science DMZ on the Ann Arbor campus to ensure researchers have the network environment necessary for their growing research efforts.
Shawn McKee, Research Scientist, Physics, College of Literature, Science and the Arts, is one of the researchers at U-M utilizing the Science DMZ. McKee is working on ATLAS, one of two main general purpose particle detector experiments at the Large Hadron Collider (LHC), a particle accelerator at CERN in Switzerland. This experiment requires analysis and sharing of large amounts of data with researchers around the world, on average about 10 Petabytes per year and growing.
According to McKee, “Traditional network firewalls create a lot of problems with movement of this type of data, making access to the Science DMZ important to work on projects like the ATLAS Experiment. Just to do physics, we need access to these types of tools. Researchers are trying to find a needle in a haystack, and tools that help us find new ways to search for and parse data quickly decrease our time to discovery.”
Handling big data transfer requires a high capacity network designed for these high performance research applications.
In 2012, U-M received a grant from the National Science Foundation to enhance the university’s network infrastructure in support of research. This grant (through ARC and ITS) helped finance a 100Gbps upgrade for connections between the MACC, MDC, and Internet2 in Chicago. It also covered upgrades to 10Gbps connection to the 3D lab in the Duderstadt Center as well as installation of several network performance monitoring devices, perfSONAR, located at core sites and other strategic locations.
Beyond the grant funding, U-M also made investments in upgrades to the core network, with 100Gbps links between nodes, as well as upgrading connectivity to 30 buildings from 1Gbps to 10Gbps.
“We must ensure the core network is never a bottleneck for U-M faculty and researchers. This means not only providing the necessary capacity and speed, but also providing tools to assist with diagnosis of network problems,” said Andy Palms, executive director of ITS Communication Systems. “The addition of perfSONAR devices on the network core allows faculty and researchers to test the speed of their network connections and quickly identify if there are issues with the network.”
McKee agreed, “perfSONAR gives us visibility into what the network is doing. When things move slowly, people automatically think it is the network, but sometimes it is problems with storage or applications. Now we have tools allowing us to see what is happening with the network that will either rule the network out as a problem or provide information to diagnose the issue.”
Network performance can be checked from a desktop to the edge of the university network, where network traffic is handed off to the ISP. Many other higher level academic institutions also use perfSONAR appliances, allowing for testing of network performance to another institution.
A typical perfSONAR Toolkit installation includes the Network Diagnostic Toolkit (NDT). NDT provides an on-demand service which launches a java applet on your local machine that runs network throughput tests to the perfSONAR appliance. This allows NDT to determine inbound and outbound network speed, the slowest link on the end-to-end path, ethernet duplex settings, tell you if congestion is limiting end-to-end throughput, and if there is excessive packet loss due to faulty cabling.
Access perfSONAR testing information at: www.itcom.itd.umich.edu/backbone/perfsonar/connection-test.html
For more information about U-M Science DMZ: http://www.itcom.itd.umich.edu/backbone/science-dmz/