GridFTP: Success Stories

"One Australian user left nearly 1TB of data on our systems that we had been struggling to transfer via standard FTP for several weeks. The typical data rate using multiple sessions of scp was ~200 KB/s. Using GridFTP we are now moving data at 6 MB/s, quite a significant boost in performance!"
- Brian Tieman and Francesco De Carlo, Advanced Photon Source, Argonne National Laboratory - Mar, 2008

"Using GridFTP, we moved about 1.5 TB of data from University of Wisconsin, Milwaukee to Hannover, Germany at a sustained rate of 80 MB/s. We were quite pleased with 80 MB/s."
- Scott Koranda, LIGO Project - June, 2008

"Getting large datasets to our GPFS filesystem can be greatly speeded up by using striped transfers with GridFTP. For example transferring a 14GB file from the UC/ANL TeraGrid site to the CI GPFS cluster using scp took almost 28 minutes at an average 8.6MB/s. That same file transferred via GridFTP took 6.5 minutes for an average of 37.3MB/s or a 4.3x increase over scp. And lastly that same file transferred via a striped GridFTP took just over 3 minutes for an average of 75.8MB/s or a 2.0x increase over non-striped GridFTP or an 8.8x increase over scp."
- Ti Leggett, Senior HPC systems administrator Computation Institute, University of Chicago - July, 2008

"Using GridFTP, we currently get up to 200 MB/s on the TeraGrid network between Texas Advanced Computing Center (TACC) and San Diego Supercomputing Center (SDSC)."
- Yifeng Cui, San Diego Supercomputer Center - July, 2008

"ChinaGrid's data service uses GridFTP as its underly data transfer tool. Its third-party data transfer capability enables us to use a centrally managed service to control all the data transfer tasks, simplifying our data service's design, configuration and management greatly. ChinaGrid applications such as BioInformatics Grid, Computational Fluid Dynamics Grid, Massive Information Processing Grid, Image processing Grid move huge volumes of data all over China using GridFTP. GridFTP's parallel data transfer and other optimizations to improve the speed enables the ChinaGrid applications to get better data transfer rates."
- Yongwei Wu, Tsinghua University, China - Jan 2009

"The availability of several striped data nodes and parallel streams provided by GridFTP allows us at LRZ to circumvent disk and network interface bottlenecks and to avoid long latencies between Grid computing centres. Moreover, the disk block size parameter of GridFTP has been useful to improve performance. We achieved over 300 MB/s data transfer speeds fairly easily inside Europe with the TCP-based GridFTP protocol."
- Helmut Heller, Leibniz Supercomputing Center (LRZ), Germany - Feb 2009

"The Dark Energy Survey Data Management (DESDM) team has relied on GridFTP as the underlying file transfer protocol for rapidly transferring large quantities of raw and reduced data between a collection of heterogeneous sites. The project has needed to transfer data between TeraGrid sites (NCSA and SDSC), between NCSA and FermiLab (a DOE site), from DESDM team dedicated servers to shared supercomputer platforms, from parallel file systems to tape archives. For all of these use cases, GridFTP has provided the flexible and high-performance solution required by the DESDM project."
- Greg Daues, National Center for Supercomputing Applications, USA - Mar 2009

"GridFTP has been a pivotal part of our application development, providing the fast, reliable and secure communication that underpins our applications for the large-scale service infrastructures we have created. Its use has been particularly significant in our work with the British Broadcasting Corporation (BBC) in prototyping next generation on-demand media infrastructures. In these applications GridFTP has been used to provide a media streaming service and has been embedded in set-top boxes to provide fast on-demand download of video content. GridFTP has proven more effective than the traditional proprietary tools that are more often used in a broadcast infrastructure. For our BBC applications, GridFTP has been used to move thousands of hours of video content between sites and to users."
- Terence Harmer, Belfast e-Science Center, UK - Mar 2009

"High performance file transfer has always been critical to the success of WestGrid. Right from the initial phase of WestGrid in 2003, some compute resources were more than 500 miles away from our primary storage server; now WestGrid spans larger distances. Due to this, appropriate tools are needed to enable high perfromance files transfers across WestGrid's private network over these distances. In addition, due to participation in world wide high energy physics experiments, large amounts data needs to be transferred between sites in US and Europe, and WestGrid systems. In all cases GridFTP has been the primary tool used to enable high performance file transfers we rely on. While we have explores other high performance file transfer tools over the past 5 years, the GridFTP implemention in the Globus toolkit is the only one that we have found t be fully robust and remains the primary file tranfer tool used within WestGrid."
- Rob Simmonds, University of Calgary, Canada - Apr 2009

"The data rates available with SSH authenticated globus-url-copy are roughly double (100MB/sec vs 50MB/sec) what we were achieving with other high speed data movement tools at a fraction of the system resource utilization. Having an SSH authenticated GridFTP has allowed us to quickly deploy globus-url-copy using our existing infrastructure."
- Chandin Wilson, National Oceanic and Atmospheric Administration - Sep, 2009

"We now have 2 beamlines routinely automating data movement using GridFTP as part of the acquisition workflow with plans for 2 more to begin working this way in the next 1-2 months. The ability to script data movement 3rd party is really working well for us. Not only are we moving data ~4 times faster than before, the ability to pipe it during acquisition rather than burst it all at the end has been a big boon!"
- Brian Tieman, Advanced Photon Source, Argonne National Laboratory - Feb, 2010