Most of what a server tells you arrives as text: access logs, error logs, config files, command output, JSON from an API. When the Orbit API starts returning errors at 9 a.m., nobody opens a spreadsheet. You type a short chain of commands, each doing one small job, and thirty seconds later you know which IP is hammering the server, which endpoint is slow and how many 5xx errors happened in the last few minutes.
In this lesson you learn how text flows between commands, how to send it to files or throw it away, and the dozen tools that filter, count and reshape it. You finish by building real one-liners on an nginx access log. A container or WSL is enough for everything here; you do not need the VM.
Three streams: stdin, stdout and stderr
Every process starts with three open channels, each with a number called a file descriptor:
| Number | Name | Default | Used for |
|---|---|---|---|
| 0 | stdin | Your keyboard | Input the program reads |
| 1 | stdout | Your terminal | Normal output |
| 2 | stderr | Your terminal | Errors and warnings |
Both stdout and stderr land on your screen by default, so they look identical. They are not, and the difference shows the moment you redirect:
$ ls /etc/hostname /etc/nope > found.txt
ls: cannot access '/etc/nope': No such file or directory
$ cat found.txt
/etc/hostname
The good result went into the file; the error still reached your screen because it travels on stream 2. That separation is deliberate: a script can save its data and still show you what went wrong.
Redirection
Redirection tells the shell where a stream should go before the command starts. The command itself never knows.
| Syntax | Effect |
|---|---|
cmd > file | stdout to file, truncating it first |
cmd >> file | stdout appended to file |
cmd 2> file | stderr to file |
cmd > file 2>&1 | stdout to file, then stderr to wherever stdout points |
cmd &> file | Bash shorthand for both streams to one file |
cmd < file | file becomes stdin |
cmd 2> /dev/null | discard errors (/dev/null swallows everything written to it) |
2>&1 means “make stream 2 point where stream 1 points right now“, and the shell reads redirections from left to right. So these two lines differ:
$ ./deploy.sh > deploy.log 2>&1
$ ./deploy.sh 2>&1 > deploy.log
The first sends both streams to deploy.log. In the second, stderr was copied to the terminal before stdout moved to the file. When in doubt in Bash, write &> and the question disappears. In portable sh scripts, use the first form.
Warning: > truncates the file before the command even runs. sort notes.txt > notes.txt leaves you with an empty file. Write to a new file and rename it, or use sort -o notes.txt notes.txt, which is safe.
Pipes and tee
The pipe | connects the stdout of one command to the stdin of the next, as in ls /etc | wc -l. No temporary files, and all commands run at the same time, so a pipe can process a 10 GB log with almost no memory. Pipes carry only stdout; to send errors down the pipe too, write cmd 2>&1 | less (Bash also accepts cmd |& less).
tee copies its input to a file and to stdout, like a T-junction: ./orbit-healthcheck.sh | tee healthcheck.txt shows the result and keeps a copy. It also solves a classic trap. sudo echo "LOG_LEVEL=debug" >> /etc/orbit/orbit.env fails with “Permission denied”, because the redirection is done by your unprivileged shell, not by sudo. Let a root process write the file instead:
$ echo "LOG_LEVEL=debug" | sudo tee -a /etc/orbit/orbit.env > /dev/null
-a appends instead of overwriting, and > /dev/null hides the copy tee would print.
Set up a practice log
If you skipped ahead, or are in a fresh container, create the files used in the rest of this lesson. Orbit’s nginx uses the standard combined log format plus one extra field at the end: the request time in seconds ($request_time).
mkdir -p ~/lab && cd ~/lab
cat > access.log <<'EOF'
203.0.113.10 - - [03/Nov/2026:09:14:02 +0000] "GET /api/notes HTTP/1.1" 200 1532 "-" "Mozilla/5.0 (X11; Linux x86_64)" 0.042
203.0.113.10 - - [03/Nov/2026:09:14:03 +0000] "POST /api/notes HTTP/1.1" 201 211 "-" "Mozilla/5.0 (X11; Linux x86_64)" 0.118
198.51.100.7 - - [03/Nov/2026:09:14:05 +0000] "GET /api/notes/42 HTTP/1.1" 404 64 "-" "curl/8.14.1" 0.006
192.0.2.55 - - [03/Nov/2026:09:14:09 +0000] "GET /wp-login.php HTTP/1.1" 404 162 "-" "Go-http-client/1.1" 0.001
203.0.113.10 - - [03/Nov/2026:09:14:11 +0000] "GET /api/search?q=linux HTTP/1.1" 200 4810 "-" "Mozilla/5.0 (X11; Linux x86_64)" 1.874
198.51.100.7 - - [03/Nov/2026:09:14:12 +0000] "GET /api/notes HTTP/1.1" 502 157 "-" "curl/8.14.1" 3.001
192.0.2.55 - - [03/Nov/2026:09:14:15 +0000] "GET /.env HTTP/1.1" 404 162 "-" "Go-http-client/1.1" 0.001
203.0.113.10 - - [03/Nov/2026:09:14:20 +0000] "GET /api/notes HTTP/1.1" 200 1532 "-" "Mozilla/5.0 (X11; Linux x86_64)" 0.039
203.0.113.24 - - [03/Nov/2026:09:14:21 +0000] "GET / HTTP/1.1" 200 3120 "-" "Mozilla/5.0" 0.004
198.51.100.7 - - [03/Nov/2026:09:14:25 +0000] "GET /api/notes HTTP/1.1" 504 167 "-" "curl/8.14.1" 60.002
203.0.113.24 - - [03/Nov/2026:09:14:30 +0000] "DELETE /api/notes/7 HTTP/1.1" 204 0 "-" "Mozilla/5.0" 0.051
192.0.2.55 - - [03/Nov/2026:09:14:31 +0000] "GET /admin HTTP/1.1" 404 162 "-" "Go-http-client/1.1" 0.001
EOF
Split on spaces, the useful fields are: $1 client IP, $4 time, $6 method (with a leading quote), $7 path, $9 status, $10 bytes and the last field, request time. The user agent contains spaces, so its field count varies; that is why you will reach the request time as “the last field” rather than by number.
Reading files: cat, less, head and tail
cat fileprints a whole file. Fine for short files, useless for a 2 GB log.less filepages through it:/textsearches,njumps to the next match,Ggoes to the end,qquits.head -n 5 fileshows the first five lines,tail -n 20 filethe last twenty, andtail -n +2 fileeverything from line 2 (skipping a CSV header).
For live logs, tail -f keeps the file open and prints new lines as they arrive. There is a catch: logrotate renames access.log at night and nginx starts a new file. tail -f keeps watching the old, renamed file and goes silent. tail -F follows the name and reopens the new file, so sudo tail -F /var/log/nginx/access.log is the one to use on servers. For services that log to the journal instead of files, journalctl -u orbit-api -f is the equivalent; lesson 18 covers it.
Searching with grep
grep prints the lines that match a pattern. The flags you will use every week:
| Flag | Meaning | Example |
|---|---|---|
-i | Ignore case | grep -i error app.log |
-v | Invert: lines that do not match | grep -v '^#' nginx.conf |
-n | Show line numbers | grep -n listen nginx.conf |
-c | Count matching lines | grep -c '" 404 ' access.log |
-r | Search a directory recursively | grep -rn ORBIT_ /etc/orbit |
-E | Extended regex (+, ?, alternation) | grep -E '50[234]' access.log |
-o | Print only the matched part | grep -oE '^[0-9.]+' access.log |
-F | Fixed string, no regex | grep -F '[error]' error.log |
-A n, -B n, -C n | n lines after, before, or around each match | grep -C 3 Traceback api.log |
The -c example searches for '" 404 ' (quote, space, 404, space) rather than just 404, so a response that happens to be 404 bytes long is not counted; on the practice log it prints 4. Alternation in action:
$ grep -E 'wp-login|\.env|/admin' access.log | cut -d' ' -f1,7
192.0.2.55 /wp-login.php
192.0.2.55 /.env
192.0.2.55 /admin
This finds a bot probing for well-known files; | inside the quotes is the regex “or”, not a pipe, and \. means a literal dot.
Tip: egrep and fgrep are obsolete and recent GNU grep prints a warning when you use them. Write grep -E and grep -F. For searching large code trees, rg (ripgrep) is a fast modern alternative, but grep is on every server.
Cutting, sorting and counting
cut
cut extracts columns: -d sets the delimiter (one character) and -f picks fields, so cut -d: -f1,7 /etc/passwd prints every account with its login shell. It treats every single delimiter as a boundary, so two spaces in a row create an empty field. For whitespace-aligned output such as ps or df, use awk or squeeze the spaces first with tr -s ' '.
sort, uniq and wc
sort orders lines alphabetically by default. -n sorts numerically (10 after 9), -r reverses, -k 2 sorts by the second whitespace-separated field (-k 2,2n: only that field, numerically), -h understands human sizes like 512K and 1.5G (try du -sh /var/log/* | sort -h) and -u drops duplicates.
uniq collapses adjacent identical lines, and uniq -c prefixes each with a count. Adjacent is the key word: always sort before uniq. Finally, wc -l counts lines (-w words, -c bytes), the end of countless “how many?” pipelines.
tr
tr translates, squeezes or deletes single characters from stdin. tr 'a-z' 'A-Z' upper-cases text, tr -s ' ' squeezes runs of spaces into one, and tr -d '\r' < win.txt > unix.txt strips the carriage returns Windows editors add, a frequent cause of “bad interpreter” errors in scripts.
Editing streams with sed
sed (stream editor) changes text as it flows past. Its most common command is substitution: sed 's/80/8080/' replaces the first match on each line, and s/80/8080/g every match. It can also select lines by number or pattern:
| Command | What it does |
|---|---|
sed -n '5,10p' file | Print only lines 5 to 10 (-n turns off automatic printing) |
sed -n '$p' file | Print the last line |
sed '/^#/d' file | Delete comment lines |
sed -n '/09:14:10/,/09:14:25/p' access.log | Print from the first line matching one pattern to the next matching the other |
To change a file in place, use -i. On a config file, always add a suffix so sed keeps a backup:
$ sudo sed -i.bak 's/^LOG_LEVEL=info$/LOG_LEVEL=debug/' /etc/orbit/orbit.env
$ diff /etc/orbit/orbit.env.bak /etc/orbit/orbit.env
< LOG_LEVEL=info
> LOG_LEVEL=debug
Preview first by running the command without -i, and check afterwards with diff (output shortened here). Anchors (^ start, $ end of line) keep the change from hitting lines you did not mean.
Warning: sed -i behaves differently on macOS (BSD sed), which requires an argument: sed -i '' 's/a/b/' file. Scripts that run on both should use sed -i.bak and delete the backup afterwards.
Processing columns with awk
awk reads input line by line, splits each line into fields ($1, $2 and so on; $NF is the last field and NF the number of fields), and runs pattern { action } rules on it. With no pattern, the action runs on every line; with no action, matching lines are printed.
$ awk '$9 >= 500' access.log | cut -d' ' -f1,7,9
198.51.100.7 /api/notes 502
198.51.100.7 /api/notes 504
$ awk '{bytes += $10} END {print bytes}' access.log
12079
$ awk '{t += $NF; n++} END {printf "%.3f s average over %d requests\n", t/n, n}' access.log
5.428 s average over 12 requests
The first prints matching lines (a pattern with no action); the last two use a variable and the special END block, which runs after the final line: that is how awk sums and averages. Note how one 60-second timeout drags the average to 5.4 seconds while most requests took milliseconds. Averages hide outliers; the slowest-requests list below shows them.
Use -F to change the separator, for example awk -F: '$3 >= 1000 {print $1}' /etc/passwd lists human accounts, a preview of lesson 7.
Tip: Ubuntu installs mawk as awk, Rocky Linux uses GNU gawk, and Alpine uses BusyBox awk. Everything in this lesson works in all three; gawk-only extras such as strftime do not.
Turning output into arguments with xargs
Some commands do not read stdin; they want file names as arguments (rm, chmod, sed -i). xargs reads items from stdin and appends them to a command:
$ grep -rlZ 'orbit-db-01' /etc/orbit | xargs -0 -r echo sudo sed -i.bak 's/orbit-db-01/orbit-db-02/'
sudo sed -i.bak s/orbit-db-01/orbit-db-02/ /etc/orbit/orbit.env
grep -l lists matching file names and -Z ends each with a null byte; xargs -0 splits on those bytes, so names with spaces survive. -r runs nothing if the input is empty. The echo in front is a dry run that prints the command instead of running it; remove it when the output looks right. -n 1 passes one argument per command and -P 4 runs four in parallel.
JSON logs with jq
Modern apps, including Orbit’s API, often log one JSON object per line. Install the processor with sudo apt install jq, then create a small sample:
cat > ~/lab/api.log <<'EOF'
{"ts":"2026-11-03T09:14:12Z","level":"error","path":"/api/notes","status":502,"ms":3001,"msg":"redis timeout"}
{"ts":"2026-11-03T09:14:20Z","level":"info","path":"/api/notes","status":200,"ms":39,"msg":"ok"}
{"ts":"2026-11-03T09:14:25Z","level":"error","path":"/api/notes","status":504,"ms":60002,"msg":"upstream timeout"}
EOF
$ jq -r 'select(.level == "error") | "\(.ts) \(.status) \(.msg)"' ~/lab/api.log
2026-11-03T09:14:12Z 502 redis timeout
2026-11-03T09:14:25Z 504 upstream timeout
select() filters objects and -r prints raw strings without quotes; jq -c 'select(.ms > 1000) | {path, ms}' would print compact one-line JSON objects instead. Never parse JSON with grep and sed: field order and spacing can change at any release, and jq does not care.
One-liners for the Orbit access log
Put the pieces together. On the server, replace access.log with /var/log/nginx/access.log and prefix sudo.
# Top client IPs
$ awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -n 3
4 203.0.113.10
3 198.51.100.7
3 192.0.2.55
# Requests per status code
$ awk '{print $9}' access.log | sort | uniq -c | sort -rn
4 404
4 200
1 504
1 502
1 204
1 201
# Three slowest requests: seconds, path, status
$ awk '{print $NF, $7, $9}' access.log | sort -rn | head -n 3
60.002 /api/notes 504
3.001 /api/notes 502
1.874 /api/search?q=linux 200
# How many 5xx in the last 1000 requests
$ tail -n 1000 access.log | awk '$9 >= 500 {n++} END {print n+0}'
2
Read each from left to right: pick a column, sort so duplicates sit together, count, sort by the count, keep the top. That pattern answers almost any “which, how many, how slow” question. n+0 prints 0 instead of an empty line when nothing matched.
Practice lab
It is Monday morning and the Orbit on-call channel reports “the API feels slow and some calls fail”. Use ~/lab/access.log (or a real nginx log on your lab VM) to write a short incident note:
- Count how many requests failed with a 5xx status and list their paths.
- Find the three slowest requests and the client that made each one.
- Find which IP sent the most 404s and what paths it asked for.
- Save the answers to
~/lab/incident.txtwhile also seeing them on screen, then print only themsgof error entries in~/lab/api.log.
Show a solution
awk '$9 >= 500 {print $9, $7}' access.log lists the two failures (502 and 504 on /api/notes). awk '{print $NF, $1, $7}' access.log | sort -rn | head -n 3 shows the slow requests and their clients; the 60-second one is a 504 timeout. awk '$9 == 404 {print $1}' access.log | sort | uniq -c | sort -rn | head -n 1 names 192.0.2.55, and awk '$1 == "192.0.2.55" {print $7}' access.log shows it probing /wp-login.php, /.env and /admin: a scanner, not a user. Wrap any of these as { cmd1; cmd2; } | tee ~/lab/incident.txt to see and save the output together. Finally, jq -r 'select(.level == "error") | .msg' ~/lab/api.log prints the two error messages.
Common mistakes
- Using uniq without sort. It only merges neighbours, so counts come out split and wrong.
- Redirecting into the file you are reading.
cmd file > fileempties the file beforecmdreads it. - Combining sudo with a redirect. The redirection runs as you, not as root. Use
sudo teeorsudo tee -a. - Following rotated logs with tail -f. It silently follows the old file. Use
tail -F.
Exercise
You add a line to Orbit’s config with sudo echo “FEATURE_SEARCH=on” >> /etc/orbit/orbit.env and get “Permission denied”, even though sudo worked a second ago. What is going on?
Show answer
Answer: C. Redirections are set up by the shell that parses the line, and that shell runs as you. sudo only elevates the echo. Piping into sudo tee -a makes a root process open the file for appending.
Exercise
A nightly job runs ./orbit-backup.sh 2>&1 > /var/log/orbit/backup.log, yet the error messages never appear in the log file. What is the fix?
Show answer
Answer: B. Redirections apply left to right. 2>&1 first copies stderr to where stdout points at that moment (the terminal or cron’s mail), and only then is stdout moved to the file. Redirect stdout first, then duplicate stderr onto it.
Exercise
You count requests per IP with awk ‘{print $1}’ access.log | uniq -c and the same IP appears on dozens of separate lines with small counts. Why?
Show answer
Answer: D. uniq compares each line with the previous one only. Sorting groups identical values together so uniq -c can count each one once; add sort -rn afterwards to rank them.
Key takeaways
- Every process has stdin (0), stdout (1) and stderr (2); redirect them with
>,>>,2>,2>&1(order matters) or&>, and discard with/dev/null. - Pipes connect small tools into a stream;
teesaves a copy, andsudo teeis how you write root-owned files. grepfinds lines,cutandawkpick columns,sort,uniq -candwccount,trandsedtransform.- The pattern
awk | sort | uniq -c | sort -rn | headanswers most “who, what, how many” questions about a log. - Use
tail -Ffor rotated logs,sed -i.bakfor safe edits,xargs -0for file lists andjqfor JSON.
FAQ
What is the difference between > and >> in Linux?
> creates the file or truncates it to zero bytes before writing, so its old content is gone. >> creates the file if needed and appends to the end. Use >> for logs and anything you want to keep; in Bash, set -o noclobber makes > refuse to overwrite an existing file as a safety net.
What does 2>&1 mean?
It duplicates file descriptor 2 (stderr) onto whatever file descriptor 1 (stdout) currently points to. Combined with a redirect, as in cmd > out.log 2>&1, both normal output and errors end up in the same file. The order matters because the shell processes redirections from left to right.
Should I use grep, sed or awk?
Use grep to find lines, sed to change text in lines (substitutions, deleting or printing ranges) and awk when you think in columns or need arithmetic such as sums, averages and comparisons. They overlap, and a pipeline often uses all three. When the logic grows beyond one or two lines, a short Python script is clearer.
How do I follow a log file in real time?
Use tail -F /path/to/file, which keeps printing new lines and survives log rotation. Combine it with grep, for example tail -F access.log | grep --line-buffered '" 5', so filtered lines appear immediately. For systemd services, journalctl -u name -f does the same for the journal.
How do I search for text in all files in a directory?
Run grep -rn 'pattern' /path. -r recurses into subdirectories and -n shows line numbers; add -i to ignore case, -l to list only file names, and --include='*.conf' to limit the search to some file types. Send errors about unreadable files away with 2>/dev/null.