awk
A small programming language for text that splits each line into fields and runs actions on the lines that match a condition.
In more detail
Fields are $1, $2 and so on, a built-in variable counts the lines read so far, and -F sets the field separator, as in awk -F, for comma-separated data. A program is a series of condition { action } pairs, plus an optional block that runs after the last line, which makes totals easy. Unlike cut, its default separator treats any run of spaces as one.
Examples
awk -F, 'NR > 1 && $3 > 100 { print $1, $2, $3 }' sales.csvawk -F, 'NR > 1 { total += $3 } END { print total }' sales.csvawk '{print $1}'awk 'END {print NR}' # count linesA one-line awk program can count failed passwords per source address in an auth log: match lines containing "Failed password", pick the field holding the address, and add to a counter.Related terms
- cut command — The cut command, which extracts columns from each line, either by character position or by fields split on a delimiter.
- sed — The stream editor, which applies editing commands such as find-and-replace to each line of text passing through it.
- pipe — The | operator, which connects one command's standard output to the next command's standard input so they run together.
- shasum — A command-line tool that prints the SHA checksum of files, with -a 256 selecting SHA-256.
- grep — The command that prints the lines of its input that match a pattern.
- auth.log — The Linux log file that records logins, SSH sessions, sudo use and other authentication events.
Taught in
- Single Quotes, Double Quotes, Backslash — Bash Scripting & Command Line Automation
- Reshaping Text: cut, sort, uniq, tr, sed, awk — Bash Scripting & Command Line Automation
- Passing Data Between Jobs: Artifacts — Continuous Integration & Continuous Delivery Pipelines
- Speeding Up Builds: Caching Dependencies — Continuous Integration & Continuous Delivery Pipelines
- The Two A.M. Page — CompTIA Cloud+ (CV0-004) Exam Prep
- Operations Sprint: Twelve Servers, Two Hours — CompTIA Cloud+ (CV0-004) Exam Prep
- Event Correlation Rules, Thresholds & Anomaly Pipelines — Security Operations & Threat Hunting (SOC)
- The Pager Goes Off At 2 A.M.: A Three-Question Triage — CompTIA Linux+ (XK0-006) Exam Prep
- The Upgrade That Broke Nginx: Packages, Repositories And Pins On Both Families — CompTIA Linux+ (XK0-006) Exam Prep
- The Pager Command That Opened A Root Shell: sudo, su And Least Privilege — CompTIA Linux+ (XK0-006) Exam Prep
- When awk Runs Out: Python Data Types, Modules And Virtual Environments For Admins — CompTIA Linux+ (XK0-006) Exam Prep
- It Is Slow, Says Everyone: CPU, Memory, Disk And Network Bottlenecks — CompTIA Linux+ (XK0-006) Exam Prep
- Network Troubleshooting: The Ladder From Link To Port — Linux Foundation Certified System Administrator (LFCS) Exam Prep
- OpenSSH: Keys, Client Aliases And A Hardened Server — Linux Foundation Certified System Administrator (LFCS) Exam Prep
- Bridges And Bonds: Two Links, One Interface — Linux Foundation Certified System Administrator (LFCS) Exam Prep
- Performance Triage: Processor, Memory Or Disk? — Linux Foundation Certified System Administrator (LFCS) Exam Prep
- Constraints: Limits, Control Groups And Listeners — Linux Foundation Certified System Administrator (LFCS) Exam Prep
- Disk Full At Three In The Morning — Linux Foundation Certified System Administrator (LFCS) Exam Prep
- Runlevels, Targets And A Polite Shutdown — LPIC-1 Linux Administrator (101-500 and 102-500) Exam Prep