sed -z: replace newlines and match across lines (null-separated data)
- tool
- sed
- command
- sed
- flag
- -z, --null-data
- default
- off (input records are separated by newlines)
sed -z makes GNU sed split its input at NUL bytes instead of newlines. A normal text file has no NUL bytes, so the whole file becomes a single pattern space, and \n in a pattern matches the line breaks. That is why sed -z 's/\n/,/g' joins lines while the same command without -z does nothing. The second use is processing NUL-separated lists, such as the output of find -print0, safely even when names contain newlines.
Without and with -z
printf 'one\ntwo\nthree\n' > list.txt
sed 's/\n/,/g' list.txt
echo '--- with -z'
sed -z 's/\n/,/g' list.txt; echo
echo '--- trailing comma removed'
sed -z 's/\n/,/g; s/,$/\n/' list.txtOutput
one two three --- with -z one,two,three, --- trailing comma removed one,two,three
Without -z each line is processed alone and has no newline to replace. With -z the final newline is replaced too, which leaves a trailing comma; the second expression turns it back into a newline.
Replace text that spans lines
printf 'server {\n listen 80;\n}\n' > site.conf
sed -z 's/{\n listen 80;/{\n listen 8080;/' site.confOutput
server {
listen 8080;
}A pattern can now include the newline between two lines, which is impossible in normal line mode without the N command.
How -z changes the input and output
| Aspect | Default | With -z |
|---|---|---|
| Input record separator | newline | NUL byte |
| Output record terminator | newline | NUL (only for records that ended with one) |
| $ address | last line | last NUL-separated record |
| ^ and $ in a regex | start and end of the line | start and end of the record; use the M flag for each line |
| Line numbers (1, 2p) | lines | records |
NUL-separated file lists
mkdir -p docs
touch 'docs/a report.txt' docs/b.txt
find docs -name '*.txt' -print0 | sort -z | sed -z 's|^docs/|archive/|' | tr '\0' '\n'Output
archive/a report.txt archive/b.txt
Each file name is one record, so ^ anchors to the start of every name, and spaces in names are harmless. tr converts the NULs back to newlines only for display; in a real script pass the stream to xargs -0.
Pitfalls
- GNU only. BSD sed on macOS has no
-z; the portable equivalents aretrfor single characters or theNloop for multiline edits. - Anchors. With
-zon a text file,^matches only at the very start of the file and$only at its end. In a test with GNU sed 4.9, even theMmodifier did not make^match after embedded newlines in-zmode; match the newline itself instead, as ins/\n/\n> /g. - In-place edits.
sed -zirewrites the whole file in one go, which is fine, but make sure the expression does not join lines you meant to keep. - Memory. The whole file is held in memory as one record.
FAQ
Why does sed 's/\n/,/g' not replace newlines?
sed reads one line at a time and removes the trailing newline before running the script, so the pattern space never contains one. With -z the input is split at NUL bytes instead, so a normal text file is a single record and its newlines can be matched.
Does sed -z work on macOS?
No. -z (--null-data) is a GNU sed extension. On macOS install GNU sed (gsed), use the portable N-loop sed ':a;N;$!ba;s/\n/,/g', or tr '\n' ',' for simple replacements.
Is sed -z safe for very large files?
It loads each NUL-separated record into memory, and a text file without NUL bytes is one record. For multi-gigabyte files prefer tr, awk or a streaming approach.