rsync -P: keep partial files and show progress (--partial --progress)

rsync -P is shorthand for --partial --progress. --partial tells rsync to keep a partially transferred file when the transfer is interrupted, instead of deleting it, so the next run can continue from it. --progress prints a progress line for each file. Together they are the usual choice for large files over slow or unreliable links, where a dropped SSH connection would otherwise restart a multi-gigabyte file from zero.

Syntax

rsync -aP big.iso user@backup.example.org:/srv/isos/
rsync -a --partial --progress big.iso user@backup.example.org:/srv/isos/   # same thing
rsync -a --partial-dir=.rsync-partial --progress big.iso host:/srv/isos/

After an interruption (Ctrl+C, network drop, exit code 20 or 12), run the same command again. rsync finds the partial file at the destination and uses it as the basis file for the delta-transfer algorithm, so only the missing data travels.

The options involved

--partial vs --partial-dir

rsync normally writes each file to a temporary file in the destination directory and renames it into place when it is complete. When the transfer stops, plain --partial keeps the incomplete data under the final file name, so anything that reads the destination (a web server, a backup check) sees a truncated file that looks complete. --partial-dir=.rsync-partial puts the incomplete file in a hidden directory next to the destination instead; the next run uses it to resume and deletes it once it has served its purpose. For a relative directory name rsync also adds an exclude rule so partial directories on the sending side are not copied.

The manual warns that the partial directory must not be writable by other users: do not point it at /tmp. You can also set RSYNC_PARTIAL_DIR=.rsync-partial in the environment; then -P uses that directory automatically.

Reading the progress line

While a file is transferring, --progress shows bytes received, percentage, current rate and estimated time left. When the file finishes, the line is replaced by a summary with xfr#N (the Nth file transferred) and to-chk=X/Y (files still to check out of the total). During incremental recursion you see ir-chk instead of to-chk, meaning the total is not known yet and will grow.

Pitfalls

• Local copies do not resume. Between two local paths rsync uses --whole-file by default, so a partial file is not used as a basis. Add --no-whole-file when that matters.
• --inplace conflicts with --partial-dir: it writes straight into the destination file, and with it RSYNC_PARTIAL_DIR is ignored.
• -P does not retry on its own. Wrap the command in a loop, or rerun it, after a connection unexpectedly closed (code 12) error; the meaning of each exit status is in rsync exit codes.
• Progress output goes to standard output; in cron jobs or logs drop --progress (keep --partial) to avoid huge log files.

FAQ