/commands/bash/read-file-line-by-line
bash read a file line by line (while IFS= read -r line)
- tool
- Bash
- versions
- Bash 3.2+, mapfile/readarray: Bash 4.0+
- os
- linux, macos, windows
while IFS= read -r line; do echo "$line"; done < file.txt reads a file one line at a time and keeps each line exactly as written. IFS= stops read from trimming leading and trailing spaces, -r keeps backslashes literal, and < file.txt after done feeds the file to the loop without a subshell. Add || [ -n "$line" ] to also process a last line that has no newline.
The safe loop and what each part does
printf ' indented line\npath\\with\\backslashes\n\nlast line without newline' > input.txt
echo "--- IFS= read -r, plus the last line"
n=0
while IFS= read -r line || [ -n "$line" ]; do
n=$((n + 1))
printf '%d: [%s]\n' "$n" "$line"
done < input.txt
echo "lines read: $n"
echo "--- plain read (trims, eats backslashes, drops the last line)"
while read line; do printf '[%s]\n' "$line"; done < input.txt
echo "--- mapfile into an array (Bash 4+)"
mapfile -t lines < input.txt
echo "array has ${#lines[@]} elements, first: [${lines[0]}]"Output
--- IFS= read -r, plus the last line 1: [ indented line] 2: [path\with\backslashes] 3: [] 4: [last line without newline] lines read: 4 --- plain read (trims, eats backslashes, drops the last line) [indented line] [pathwithbackslashes] [] --- mapfile into an array (Bash 4+) array has 4 elements, first: [ indented line]
The plain read loop shows the three classic bugs at once: leading spaces trimmed, backslashes removed, and the last line lost because the file does not end with a newline.
Variants
| Code | Use for |
|---|---|
| while IFS= read -r line; do ...; done < file | Every line, unchanged |
| ... || [ -n "$line" ] | Also handle a last line without a newline |
| while IFS=, read -r name age; do | Split each line into fields (simple CSV without quoted commas) |
| while IFS= read -r line; do ...; done < <(command) | Read the output of a command, keeping variables after the loop |
| mapfile -t lines < file | Load all lines into an array (Bash 4+, also called readarray) |
| while IFS= read -r -u 3 line; do ...; done 3< file | Read from descriptor 3 so commands in the loop can still use stdin |
Pitfalls
- Commands that read stdin inside the loop.
ssh,ffmpegor anotherreadin the loop body consume the rest of the file, so the loop stops after one line. Redirect their input (ssh host cmd < /dev/null,ffmpeg -nostdin) or read the file from descriptor 3 as in the table. - Pipes create subshells.
cat file | while read ...works, but counters set inside are lost afterdone. - Windows line endings. Lines from a CRLF file end in
\r, which breaks comparisons and file names; strip it withline=${line%$'\r'}. - Unquoted
$line.echo $linecollapses spaces and expands wildcards; quote it. set -eandread.readreturns 1 at end of file; that is fine as a loop condition, but a barereadoutside a loop stops a script running with set -euo pipefail.
Confirm it worked
Print each line between brackets with its number, as in the example: leading spaces, empty lines and a lost last line become obvious. Compare the count with wc -l plus one if the file lacks a final newline.
FAQ
Why is the last line of my file skipped?
read returns a non-zero status when it hits end of file before a newline, so a last line without a trailing newline ends the loop before the body runs. Add || [ -n "$line" ] to the loop condition.
Why are variables set inside the loop empty afterwards?
With cat file | while read ..., the loop runs in a subshell, and its variables disappear when it ends. Redirect the file into the loop instead (done < file), or use shopt -s lastpipe.
Can I use for line in $(cat file)?
Not for lines. The output is split on every space, tab and newline, and each word is globbed, so you get words, not lines. Use while IFS= read -r line.