awk

A practical reference for AWK, the classic Unix pattern scanning and text processing language, with concise examples for filtering, transforming, and analyzing structured text

---

Print hello world

awk 'BEGIN { print "hello, world" }'

Print hello world with multi-line

awk '
    BEGIN {
        print "hello, world"
    }
'

Print all matches of a string (alias)

awk '/alias/' ~/.bashrc

Remove lines with string alias

awk '!/alias/' ~/.bashrc

Print field start with string (e.g Linux)

awk '$1 ~ /^alias/' ~/.bashrc

Remove last column

awk 'NF{NF-=1};1' ~/.bashrc

Remove duplicate entries in a file without sorting

awk '!seen[$0]++'

Break combine column data into rows

awk '{split($2,a,",");for(i in a)print $1"\t"a[i]}' ~/.bashrc

Reverse column order

awk '{print $2, $1}' ~/.bashrc

Split and do for loop

awk '{split($2, a,",");for (i in a) print $1"\t"a[i]}' ~/.bashrc

Print all lines before nth occurrence of a string (e.g stop print lines when ‘alias’ appears 7 times)

awk -v N=7 '{print}/alias/&& --N<=0 {exit}' ~/.bashrc    

Give number to every row

awk '{printf("%s\t%s\n",NR,$0)}' ~/.bashrc

Print filename and last line of all files in directory

ls|xargs -n1 -I file awk '{s=$0};END{print FILENAME,s}' ~/.bashrc

Add string to the beginning of a column (e.g add testing to column $3)

awk 'BEGIN{OFS="\t"}$2"testing"$2' ~/.bashrc

Print line number and number of characters on each line

awk '{print NR,length($0);}' ~/.bashrc

Show uptime in minutes

awk '{print int($1/3600)"h", int(($1%3600)/60)"m"}' /proc/uptime

Display system load averages from /proc/loadavg

awk '{print $1, $2, $3}' /proc/loadavg

Print duplicate lines using awk

awk '++seen[$0] == 2 { print }'

Print entire file contents with awk

awk '{ print $0 }' /proc/cmdline

Print first field from /proc/cmdline using awk

awk '{ print $1 }' /proc/cmdline

Print eighth field from /proc/cmdline using awk

awk '{ print $8 }' /proc/cmdline

Print lines containing the exact word root

awk '/\<root\>/' /etc/passwd

Print every line that includes ro from /etc/passwd

awk '/ro/ { print $0 }' /etc/passwd

Print every line that is shorter then 80 characters from /etc/passwd

awk 'length($0) < 80' /etc/passwd

Print every line that is longer than 80 characters

awk 'length($0) > 80' `/etc/passwd`

Print the length of the longest input line from /etc/passwd

awk '{ if (length($0) > max) max = length($0) }
 END { print max }' /etc/passwd

Print the length of the longest line in /etc/passwd

cat /etc/passwd| awk '{ if (x < length($0)) x = length($0) }
               END { print "maximum line length is " x }'

Print every line that has at least one field

• This is an easy way to delete blank lines from a file (or rather, to create a new file similar to the old file but from which the blank lines have been removed)

awk 'NF > 0' data

Print seven random numbers from 0 to 100, inclusive

awk 'BEGIN { for (i = 1; i <= 7; i++)
             print int(101 * rand()) }'

Print the total number of bytes used by files

ls -l files | awk '{ x += $5 }
               END { print "total bytes: " x }'

Print the total number of kilobytes used by files

ls -l files | awk '{ x += $5 }
   END { print "total K-bytes:", x / 1024 }'

Print a sorted list of the login names of all users

awk -F: '{ print $1 }' /etc/passwd | sort

Count the lines in a file

awk 'END { print NR }' data

Print the even-numbered lines in the data file

awk 'NR % 2 == 0' /etc/passwd

Print the odd-numbered lines in the data file

awk 'NR % 2 == 1' /etc/passwd

Print each line that contains 10 in /etc/passwd

awk '/10/ { print }' /etc/passwd

Print each line that contains 1000 in /etc/passwd

awk '/1000/ { print }' /etc/passwd

Print all files that has been modified in Jan (january month)

ls -l | awk '$0 == "Jan" { print }'

Print lines containing the exact word 1000

awk '/\<1000\>/' /etc/passwd

Extract the Username Field from /etc/passwd Using awk

sed 1q /etc/passwd | awk '{ FS = ":" ; print $1 }'

• Print all files and folders that has been created/modified in January month

awk field	Description
$1	file permissions
$2	number of hard links
$3	owner
$4	group
$5	file size (bytes)
$6	month (Jan, Feb, …)
$7	day
$8	time or year
$9	filename

```bash
ls -l | awk '$6 == "Jan" { print }'
```

Print entries between the bin and news users in /etc/passwd

awk '/^bin:/,/^news:/' /etc/passwd

Just print "Dont Panic

awk 'BEGIN { print "Don\47t Panic!" }'

The following one-liner prints a list of usernames from /etc/passwd

awk -F":" '{print $1 }' /etc/passwd

Mixing single and double quotes is difficult. You have to resort to shell quoting tricks, like this

awk 'BEGIN { print "Here is a single quote <'"'"'>" }'

The field separator can also be set in a BEGIN function block

awk 'BEGIN { FS=":" } {print $1 }' /etc/passwd

Every user whose shell is not /sbin/nologin can be printed by preceding the block with a pattern match:

awk 'BEGIN { FS=":" } ! /\/sbin\/nologin/ {print $1 }' /etc/passwd

Print entire line

awk '{print $0}' /some/file

Convert to uppercase and print line 2

awk 'NR==2 {print toupper($0)}' /etc/os-release

Convert to lowercase and print line 2

awk 'NR==2 {print tolower($0)}' /etc/os-release

Find all lines that begins on Example and delete one line below

awk '/^Example:/ { getline; next } 1' file.txt

Print current cpu usage for all cores

awk -F": " '/cpu MHz */ { print "Processor (or core) running speed is: " $2 }' /proc/cpuinfo

Get cpu running speed

awk '{ print "Processor running speed is:", $1/1000, "MHz" }' /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq

Another way to get cpu running speed in mhz

awk '{ print "Processor running speed is:", $1/1000, "MHz" }' /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq

Get average processor speed

awk '{ sum += $1; n++ }
    END { printf "Average processor running speed is: %.2f MHz\n", sum/n/1000 }' \
    /sys/devices/system/cpu/cpu*/cpufreq/scaling_cur_freq

Print cpu base speed in GHz

awk -F'@ ' '
    /model name/ { base = $2; exit }
    END { print "Processor base speed is:", base }
    ' /proc/cpuinfo

Print cpu temperature by awk script

for f in /sys/class/hwmon/hwmon*/temp*_input; do
  base="${f%_input}"
  label_file="${base}_label"

  label=$(cat "$label_file" 2>/dev/null || echo "$(basename "$base")")
  awk -v label="$label" '{printf "%-12s %.1f°C\n", label ":", $1/1000}' "$f"
done

Print uptime in hours

awk '{ printf "%dh\n", int($1/3600) }' /proc/uptime

Print date and timesmpta

awk '{t=int($1);printf "%d days %02d:%02d:%02d\n",t/86400,(t%86400)/3600,(t%3600)/60,t%60}' /proc/uptime

Print uptime from uptime to timestamp

awk '{
  t = int($1)
  d = t / 86400
  t %= 86400
  h = t / 3600
  t %= 3600
  m = t / 60
  s = t % 60
  printf "%d days %02d:%02d:%02d\n", d, h, m, s
}' /proc/uptime

Print uptime from uptime to timestamp

awk '{printf "%d days %02d:%02d:%02d\n",
int($1/86400),
int($1%86400/3600),
int($1%3600/60),
int($1%60)}' /proc/uptime

Column subtraction

cat file| awk -F '\t' 'BEGIN {SUM=0}{SUM+=$3-$2}END{print SUM}'

Print duplicates side by side

awk '
{
    lines[$0] = lines[$0] ? lines[$0] RS NR ":" $0 : NR ":" $0
}
END {
    for (l in lines)
        if (split(lines[l], a, RS) > 1)
            print a[1] "\t|\t" a[2]
}
' $SOMEPATH/filename.md | column -ts $'\t'

A shorter way to print duplicates side by side

awk '
++seen[$0]==1 { first[$0]=$0 }
seen[$0]==2 { print first[$0] "\t|\t" $0 }
' filename.md | column -ts $'\t'

Generate fake mac addresses from bluetoothctl devices to hide our real mac addresses online

bluetoothctl devices | awk '
BEGIN { srand() }
{
    printf "Device %02X:%02X:%02X:%02X:%02X:%02X ",
        rand()*256, rand()*256, rand()*256,
        rand()*256, rand()*256, rand()*256
    for (i=3;i<=NF;i++) printf "%s%s", $i, (i<NF?" ":"")
    print ""
}'
Device 7C:47:EE:54:54:9C WH-1000XM5
Device C5:36:E2:64:07:9B Samsung S24 Ultra
Device 07:D4:CC:6C:78:1D [Samsung] Soundbar
Device 65:B4:9C:B5:A4:B2 Jabra Evolve 65
Device B2:31:10:74:CA:12 Moto G50
Device 17:CB:00:7B:74:72 WH-1000XM4
Device 76:84:29:BA:28:1C Galaxy Watch5 Pro (RMCF)

• Above awk script shuffle the mac addresses so we get new ones everytime we run the awk script

Running above command again

bluetoothctl devices | awk '
BEGIN { srand() }
{
    printf "Device %02X:%02X:%02X:%02X:%02X:%02X ",
        rand()*256, rand()*256, rand()*256,
        rand()*256, rand()*256, rand()*256
    for (i=3;i<=NF;i++) printf "%s%s", $i, (i<NF?" ":"")
    print ""
}'
Device 13:6D:0A:A0:89:57 WH-1000XM5
Device BA:5C:77:89:5C:43 Samsung S24 Ultra
Device 7B:1B:91:C6:72:64 [Samsung] Soundbar
Device 7F:C8:02:25:79:60 Jabra Evolve 65
Device C3:D2:4D:42:B4:73 Moto G50
Device 4E:25:FC:33:2B:B2 WH-1000XM4
Device CA:28:C8:B4:D0:06 Galaxy Watch5 Pro (RMCF)

Setup markdowns easy

Example: We got commandehre | description here then it will add our syntax

cat input.txt | awk -F' *\\| *' '
{
printf "!!! Example \"%s\"\n\n    ```bash\n    gsettings %s\n    ```\n\n", $2, $1
}
'

If we got "Text1 | Text2 | It will add it to bash syntax with !!! Example

awk -F'|' '
{
    id=$1
    desc=$2
    gsub(/^[ \t]+|[ \t]+$/, "", id)
    gsub(/^[ \t]+|[ \t]+$/, "", desc)

    printf "!!! Example \"%s\"\n\n", desc
    printf "    ```bash\n"
    printf "    kcmshell6 %s\n", id
    printf "    ```\n\n"
}
' filename.txt

AWK Check Memory

"Print RAM in bytes"

awk '/MemTotal/ {print $2}' /proc/meminfo

"Convert all values in /proc/meminfo to megabytes"

awk '$3==""kB""{$2=$2/1024;$3=""MB""} 1' /proc/meminfo | column -t

"This version converts to gigabytes"

awk '$3==""kB""{$2=$2/1024^2;$3=""GB""} 1' /proc/meminfo | column -t

"The math behind the conversion"

The value in /proc/meminfo (for MemTotal) is always given in kB (kilobytes).

kB → MB: divide by 1024.

Example:

1 048 576 kB / 1024 = 1024 MB

MB → GB: divide by 1024 again.

Example:

1024 MB / 1024 = 1 GB

Conclusion:
To convert directly from kB (what /proc/meminfo gives you) to GB, divide by:

\[ 1024 \times 1024 = 1\,048\,576 \]

Unit	AWK Calculation	Description
kB	$2	Raw data from the kernel
MB	$2 / 1024	First step (Kilobytes to Megabytes)
GB	$2 / 1024 / 1024	Second step (Megabytes to Gigabytes)
TB	$2 / 1024 / 1024 / 1024	Third step (Gigabytes to Terabytes)

How to write this in your code

If you want to calculate terabytes (which would be impressive on your HP EliteDesk!), the AWK command looks like this:

(Here we use %.3f to show three decimals. Otherwise, a machine with 24 GB RAM would display something like 0.0 TB because the value is so small relative to a terabyte.)

Why do we divide multiple times?

.-> Think of it like a staircase ---------------------------------------------
|
| To go up one step (to MB), divide by 1024 once.
| To go up two steps (to GB), divide by 1024 twice.
| To go up three steps (to TB), divide by 1024 three times.
´------------------------------------------------------------------------------

---

"If you want to calculate how much free RAM you have in percent, use this line"

---

"Print in Bytes"

awk '/MemTotal/ {printf(""%.0f b\n"", $2 * 1024)}' /proc/meminfo

"Print in Kilobytes"

awk '/MemTotal/ {printf(""%.2f kb\n"", $2)}' /proc/meminfo

"Print in Megabytes"

awk '/MemTotal/ {printf(""%.1f MB\n"", $2 / 1024)}' /proc/meminfo

"Print in Gigabytes"

awk '/MemTotal/ {printf(""%.1f GB\n"", $2 / 1024 / 1024)}' /proc/meminfo

"Print in Terabytes"

awk '/MemTotal/ {printf(""%.3f TB\n"", $2 / 1024 / 1024 / 1024)}' /proc/meminfo

"Memory in Percentage"

awk '/MemTotal/ {t=$2} /MemAvailable/ {f=$2} \
    END { \
        printf(""/dev/mem: %.1fGB ("", t / 1024 / 1024); \
        printf(""%.1f%%)\n"", (f/t)*100); \
    }' /proc/meminfo

Resource(s)

• https://tldp.org/LDP/abs/html/
• https://www.gnu.org/software/gawk/manual/gawk.html
• https://www.gnu.org/software/gawk/manual/html_node/Very-Simple.html