XGet
Introduction
XGet is a scalable file-transfer agent that was designed to transfer files across nodes in a cluster. One such use is to transfer kernel and initrd boot images upon node boot up. xget completes file transfers much faster than some of the existing standard file transfer utilities. xget is based on the 9P distributed resource sharing protocol originally developed as a part of the Plan 9 research operating system at the AT&T Bell Labs.The initially spawned xget server (typically running on the head node of the cluster) serves files to other xget clients across the cluster. The xget client connects to the master xget server and either starts downloading from the master, or is redirected to a secondary server. After the file has been downloaded on the client, the client can become a server and serve other clients. This creates an ad-hoc tree which is generated dynamically and can efficiently serve over thousands of nodes.
Operation
The first thing to do is to start xget on the node that will be acting as the master server. An example command that does this is:xget /tmp/dirtoserve
This will start xget as a master server and will serve all the files and subdirectories in the directory /tmp/dirtoserve.
To start a client, an example command is:
xget -n masterserver . /tmp/download
Here "masterserver" needs to be the ip or the hostname of the node running the master server. The '.' means to download all the files served by the master. The directory: /tmp/download is the destination where all the downloaded files will be placed. The '.' could be replaced with a specific directory or file. For example:
xget -n masterserver file1 /tmp/download
This command assumes that there is a file or directory named "file1" served by the master server. In this example, the server is serving all files in the directory /tmp/dirtoserve. So, for the client command to succeed, there needs to be a file or directory /tmp/dirtoserve/file1.
Download
xget (formerly called xbootfs) is a part of the XCPU cluster framework . Grab the latest tarball or binaries from the XCPU download page. The Perceus stateless clustering toolkit also uses xget (replacing NFS) as its default method for scalable VNFS transfer. XGET comes bundled with Perceus right from stage one of provisioning.Installation
Extract the XCPU tarball and use the target xget for make to build and install xget.make xget
