The GridFTP Protocol and Software

GridFTP is a high-performance, secure, reliable data transfer protocol optimized for high-bandwidth wide-area networks. The GridFTP protocol is based on FTP, the highly-popular Internet file transfer protocol. We have selected a set of protocol features and extensions defined already in IETF RFCs and added a few additional features to meet requirements from current data grid projects.

Our white paper, GridFTP: Universal Data Transfer for the Grid, describes the motivations behind this work in more detail. A subsequent update titled GridFTP Update, January 2002 provides the latest news regarding GridFTP, including current status and plans for future development.

Protocol Features

GridFTP provides the following protocol features.

  • GSI security on control and data channels
  • Multiple data channels for parallel transfers
  • Partial file transfers
  • Third-party (direct server-to-server) transfers
  • Authenticated data channels
  • Reusable data channels
  • Command pipelining

We have prepared a draft specification for the GridFTP protocol through the Global Grid Forum, and intend to submit a draft standard to the IETF for review and approval.

Implementation

Our implementation of the GridFTP protocol takes the form of two APIs and corresponding libraries: globus_ftp_control and globus_ftp_client. Our libraries use high-performance I/O and security services provided by the Globus Toolkit.

Besides supporting the protocol features described above, our APIs also include interfaces for adding software "plug-ins". These plug-ins allow developers to add customized reliability and fault tolerance, performance monitoring, and extended data processing.

In addition to our software libraries, we have also implemented an API/library (globus_gass_copy) and a command-line tool (globus-url-copy) that integrates GridFTP, HTTP, and local file I/O to enable secure transfers using any combination of these protocols.

Finally, we have adapted a popular FTP server package (Washington University's wu-ftpd) to support a majority of the GridFTP protocol features (GSI security, parallel transfer, third-party transfer, partial file transfer).

Availability

Our data grid software is currently available to the public as components of the Globus Toolkit 2.0 release. Prior to this release, the software was tested and evaluated for more than a year by several external project teams who are using our technologies to build data grids for their own use.