Structural Biology Center
Home About Us Facilities User Program Resources Research Links Contact Us
links-top_final.gif

Quick Links:

SBC:
APS:
ANL:
Home Pages:

Data Backups:

SBC staff recommend that you organize your data into sub-directories as you collect and process data at SBC.
In addition, we HIGHLY recommend that you backup ALL of the data from your SBC HOME DIRECTORY ($HOME) to a single disk or storage system. Plan on needing to backup as much as 250GB of data per day of scheduled beam time.
Backup Disc
If you must backup to separate disks, you should realize that the size (free space) of your backup disks could influence how you should organize data collection and data processing. It will simplify the data backup process if you create sub-directories under $HOME (for example, DATA1 and DATA2) that correspond to the separate backup disks that you intend to use, and then limit the amount of data that you place in each sub-directory to the available space on the corresponding backup disk.

Network Copies

 

Standard Linux Commands

 

One method of data backup supported at SBC is a network copy.

 

The network copy method is used to transfer data from an SBC Linux Workstation, to a computer at your home institute (typically running Linux), or to a laptop computer that you bring with you to SBC.

 

Copying data to a computer at your home institute using standard Linux commands such as “rsync” or “scp” requires, at a minimum, SSH access through your home institute firewall to reach your computer. If you run the data transfer commands from an SBC computer then your firewall must permit incoming ssh traffic from SBC.

 

If your laptop computer is running a Microsoft Windows operating system, you need to install client software onto it that can transfer files through ssh (such as FileZilla or winSCP).

 

Linux and Macintosh OS X have ssh clients built-in and can run an SBC-provided script called RemoteBackupScript.csh to perform the backup. Instructions for using this script are given in Using Remote Backup Script.

 

One linux computer is reserved strictly for data backup purposes, and serves both beamlines 19-ID and 19-BM:

rose.gp.sbc.aps.anl.gov

 

 

 

Globus Online

 

The easiest, fastest and most reliable way to copy data across the network - from SBC to data storage at your home institute - is to use the free data sharing service, Globus Online.

 

Globus Online provides an easy-to-use, web-based interface for transferring data between two Globus Online endpoints. This transfer can be up to two times faster than “rsync” or ”scp”.

 

SBC maintains beamline-specific Globus Online endpoints: sbc#19ID-DTN and sbc#19BM-DTN.

 

There are three requirements to meet in order to use Globus Online, to copy your data from the SBC endpoint:

  1. Register for a free Globus Online account.
  2. Set up a Globus Online endpoint on a computer at your home institute; this computer will be receiving your data.
  3. Both Globus Online endpoints will need to have valid, local accounts (login and password) on the computers managing the transfers.

Further documentation is available at: Installing GlobusConnect, and Using Globus Online at SBC.

 

External USB Disk

 

Pre-formated Disk

 

A different recommended method of data backup supported at SBC is to copy data to a user-provided USB disk that is directly attached to an SBC Linux Workstation.

 

SBC currently supports USB disks which are formatted for either Windows (NTFS or FAT32), or LINUX (ext3 or ext4).

 

You may then connect your disk directly to an SBC Linux Workstation and run an SBC script called HomerArchiver to perform the backup.

 

Alternatively, if your disk is not a compatible format, you may attach it to your own laptop and perform a network copy, as described above.

 

Your Computer

 

SBC RemoteBackupScript

 

The RemoteBackupScript.csh is intended to be run on your computer, either a laptop that you bring to the beamline, or one of the computers at your home institute which has been authorized by SBC as an NX Client computer (since NX requires ssh access).

 

Linux and Macintosh OS X have ssh client commands built-in, and can run the SBC-provided script.

 

RemoteBackupScript.csh uses the “rsync” command to perform the backup..