About This Item

Ask the MINDS@UW Librarian

DiskRouter: A Flexible Infrastructure for High Performance Large Scale Data Transfers

Show full item record


Kola, George; Livny, Miron
University of Wisconsin-Madison Department of Computer Sciences
Mar 15, 2012
The steady increase in data sets of scientific applications, the trend towards collaborative research and the emergence of grid computing has created a need to move large quantities of data over wide-area networks. The dynamic nature of network makes it difficult to tune data transfer protocols to use the full bandwidth. Further, data transfers are limited by the bottleneck link and different links become the bottleneck at different times resulting in under-utilization of other network hops. To address these issues, we have designed a flexible infrastructure that uses hierarchical main memory and disk buffering at intermediate points to speed up transfers. The infrastructure supports application-level multicast to reduce network load and enables easy construction of application-level overlay networks to maximize bandwidth. It can perform dynamic protocol tuning, use higher-level knowledge, and is being in real-life to transfer successfully several terabyts of astronomy images and educational research videos.
Permanent link
Export to RefWorks 

Part of

Show full item record

Search and browse


Deposit materials

  1. Register to deposit in MINDS@UW
  2. Need deposit privileges? Contact us.
  3. Already registered? Have deposit privileges? Deposit materials.