How to use the awk command in Linux, with examples
How the awk command reads text line by line, splits it into fields like $1, picks lines with patterns, sums columns with END and changes separators with -F.
awk, explained by cats. Kitty runs a tuna shop. Every order goes into one long file. Who ordered, which fish, how many. Today she only wants the names. awk reads a file one line at a time. It cuts each line into fields, wherever there's a space.
Dollar one is the first field. Dollar two is the second. So Kitty types awk, print dollar one. Every name. Nothing else. Now only the big orders. She puts a pattern in front: dollar three greater than ten. awk runs the action only on lines that match.
How many fish did she sell? She adds each dollar three to a sum, then prints it at the end. A hundred and fifty-six fish. Her other file uses commas? Dash F comma, and awk cuts at the commas instead. That's awk. A pattern and an action, for every line. And no, it's not short for awkward.
awk is a Linux command that reads text one line at a time, splits each line into fields, and runs a small program on every line that matches a pattern you give it.
It earns its place when your data is already in columns: a log file, a list of orders, the output of ls -l or ps. Questions that would otherwise mean opening a spreadsheet, such as 'which rows have more than ten?' or 'what's the total?', become one line in the terminal.
Lines, fields and $1
Take a file of orders, one per line: who ordered, which fish and how many.
awk calls each line a record and each piece of it a field. By default a field ends wherever there is whitespace, and a run of spaces or tabs counts as one break, so the uneven gaps above don't matter. Spaces at the start or end of a line are ignored too.
Inside the program, $1 is the first field, $2 the second and $3 the third. $0 is the whole line. Printing the first field of every line gives you the names and nothing else:
awk, explained by cats. Kitty runs a tuna shop. Every order goes into one long file. Who ordered, which fish, how many. Today she only wants the names. awk reads a file one line at a time. It cuts each line into fields, wherever there's a space.
Dollar one is the first field. Dollar two is the second. So Kitty types awk, print dollar one. Every name. Nothing else. Now only the big orders. She puts a pattern in front: dollar three greater than ten. awk runs the action only on lines that match.
How many fish did she sell? She adds each dollar three to a sum, then prints it at the end. A hundred and fifty-six fish. Her other file uses commas? Dash F comma, and awk cuts at the commas instead. That's awk. A pattern and an action, for every line. And no, it's not short for awkward.
awk also keeps a few counters for you. NF is the number of fields on the current line, so $NF is always the last field, however many there are. NR is the line number, counting from 1. awk '{print NR, $1}' numbers the names.
Pattern and action
Every awk program is made of rules, and every rule has the same shape: a pattern, then an action in braces.
pattern { action }
For each line, awk checks the pattern. If it is true, the action runs; if not, awk moves to the next line. To list only the big orders, put a comparison in front:
awk '$3 > 10 {print $1}' orders.txt
garfieldtigermochi
Either half can be left out, and both shortcuts are common:
With no pattern, every line matches. That is why {print $1} printed all seven names.
With no action, awk prints the whole line. awk '$3 > 10' orders.txt prints the three big orders in full.
A pattern can be any test. $2 == "tuna" checks a field for an exact value. /tuna/ matches any line containing 'tuna' anywhere, the way grep would. && and || combine tests: $2 == "tuna" && $3 > 10.
Adding things up with END
Two special patterns run outside the line-by-line loop. BEGIN runs once before the first line is read, which suits printing a header. END runs once after the last line, which suits printing a result.
This adds up the number of fish sold:
awk '{sum += $3} END {print sum}' orders.txt
62
The first rule has no pattern, so it runs on every line and adds the third field to sum. The END rule prints the total once the file is finished. On Kitty's longer file in the video, the same command prints 156.
You never declared sum. In awk a variable that hasn't been set yet is an empty string, and it counts as 0 in arithmetic. A text field you add up counts as 0 too, so a header line such as name fish count adds nothing rather than causing an error.
Totals per group
awk also has arrays whose keys can be any text. Use the fish as the key and you get a total for each kind in one pass:
awk '{t[$2] += $3} END {for (f in t) print f, t[f]}' orders.txt
sardine 4tuna 54salmon 4
for (f in t) visits every key, but in no fixed order. Pipe the output through sort when the order matters.
Other separators: -F
Not every file uses spaces. -F sets the field separator, so -F, splits each line at the commas instead:
awk -F, '{print $1}' today.csv
Here it is running on a comma-separated file of today's orders:
From the coding.kitty video
Everything else works the same once the separator changes, so awk -F, '$3 > 10 {print $1}' today.csv finds the big orders in the comma file. Use -F'\t' for tab-separated files, and -F: for files like /etc/passwd, whose fields are separated by colons.
A separator longer than one character is treated as a regular expression, so -F'[,;]' splits at either a comma or a semicolon.
When the data has commas in it
-F, splits at every comma, including ones inside a quoted value. A line like "Kitty, the cat",tuna,3 has three columns to a spreadsheet but four fields to awk -F,. Recent versions of GNU awk and of the original awk have a --csv option that understands quoted fields. Without it, for messy CSV, a CSV-aware tool or a short script in a language with a CSV library is safer.
awk, grep, sed and cut
These tools overlap, and picking the smallest one that does the job keeps a command readable:
Tool
Best at
grep
Finding the lines that match
sed
Find-and-replace inside lines
cut
Taking columns at a single-character separator
awk
Columns plus tests, sums and formatting
cut -d' ' treats every single space as a separator, so it trips over the uneven gaps in the orders file, where awk doesn't. Once you need a condition on one column and a value from another, awk is usually shorter than chaining the others together.
When a job grows past a few rules, with several files, error handling or lots of state, a script in Python or another general-purpose language is easier to read and test than a long awk one-liner.
Common mistakes
Double quotes around the program. In awk "{print $1}", the shell replaces $1 with its own first argument, usually nothing, before awk sees it. The program becomes {print }, which prints every whole line. Wrap awk programs in single quotes.
Leaving out the comma in print. print $1, $3 puts a space between the fields. print $1 $3 joins them with nothing in between: salem4.
Comparing with a quoted number. $3 > "10" compares as text, character by character, so 4 counts as bigger than 10. Write $3 > 10 with no quotes to compare numbers.
Expecting -F, to understand CSV. It splits at every comma, quoted or not.
Where it comes from
awk was written at Bell Labs in 1977 by Alfred Aho, Peter Weinberger and Brian Kernighan, and the name is their initials. It is part of the POSIX standard, so every Linux and macOS system has one. Most Linux distributions use GNU awk (gawk) or mawk, and macOS ships the original 'one true awk'. Simple programs like the ones on this page run the same on all of them. On Windows, WSL gives you a Linux shell with awk in it, and if you need to look at a file before working on it, cat prints it to the terminal.
Key takeaways
awk reads input one line at a time and splits each line into fields: $1, $2 and so on, with $0 for the whole line.
A program is a list of pattern { action } rules. No pattern runs on every line; no action prints the line.
END runs after the last line, which makes totals and counts a one-liner.
-F changes the separator, such as -F, for comma-separated files.
Wrap the program in single quotes so the shell leaves $1 alone.