The problem: Backup interferes with normal operation
Backup is running at the same time as the normal operation of the production system and the system is already quite busy. Copy and transfer tools take as many resources as they can get from CPU, DMA and network. This usually interferes with the normal server operations and can even bring down the server (thank you DMA).
The solution: Throttling
Many of the tools have a throttling option, which can limit the bandwidth in network transfers and file system operations.
- Option -l limits the used bandwidth, specified in Kbit/s.
- Example for max 5 MByte/sec: scp -l 40000 myfile $user@$host:
- Option --bwlimit limits I/O bandwidth in KBytes per second.
- Example for max 5 MByte/sec: rsync --bwlimit 5000 myfile destfolder
- Setting "net:limit-rate" limits transfer rate on data connection in bytes per second.
- Example for max. 5 MByte/sec: echo "set net:limit-rate 5000000; put myfile" | lftp -u $user,$password $host
- cstream can limit throughput of pipes with option -t in bytes per second.
- Example for max 5 MByte/sec: tar cf - myfolder | cstream -t -5000k > myfolder.tar
- Use "-5000k" instead of "5000k" to never exceed the max. Otherwise cstream saves throughput it could not use for later, which may be more bursty.