A man casting a shadow over a wall covered in project nerdy text.

When I was learning Unix in the early nineties I struggled to understand and remember the syntax of commandline tools. To help myself learn, each time I discovered a new command, I’d record each new success so I had a known good starting point for the next time.

Initially I kept these notes in ~/etc/ so they were easy to grep. Later they moved to a wiki page where they continued to flourish, and then finally to my WordPress site where they languished … until (hopefully) now!

Over the years I've kept the habit because it continues to be useful.

101 Recently Updated Gists

		nmap --script broadcast-dhcp-discover --script-args clientid=heyzeus

Broadcast a DHCP client request with the client ID of "heyzeus" and see all responses.

		nmap --script broadcast-dhcp-discover --script-args mac=01:23:45:67:89:ab

Broadcast a DHCP client request with a MAC address and see responses from all DHCP servers.

		# rclone mount <myHidrive>:/users/<myHidrive>/mounted /mnt/<myHidrive> --vfs-cache-mode writes --allow-other --transfers=1 --buffer-size=0 --use-mmap --hidrive-chunk-size=5M --hidrive-upload-cutoff=5M --hidrive-upload-concurrency=1 --daemon

Tweak rclone to use as little memory as possible.

		# curl -X POST -H 'Content-type: application/json' --data '{"text":"Hello, World!"}' http://localhost:5173/webhook/thing/other/whazzit

How to test sending a webhook with cURL.

		# sshuttle --dns -r 0/0

Forward all traffic, including DNS, over an ssh tunnel.

		# [ -z "$TMUX"  ] && { tmux attach || exec tmux new-session && exit;}


Add to `.zshrc` to automatically start a tmux session on login.  If there is an existing session, it will attach, otherwise it'll spawn a new tmux session.

		# sh -c "$(curl -fsLS" -- init --apply adamshand --ssh --purge-binary

Bootstrap Chezmoi in new ${HOME}.  Download binary, install dotfiles from Github user adamshand using SSH authentication, and then delete downloaded binary.

		# python3 --version | awk -F'[ .]' '{print $2"."$3}'

Set awk field delimiter to use a character class (be either <space> or <period>).

		# op group list | awk '!/ID/ {print $0}' > ~/tmp/groups.txt 
# for group in $(cat ~/tmp/groups.txt | awk '!/ID/ {print $1}' ); do grep $group ~/tmp/groups.txt | awk '{$1=""; print "## GROUP: "$0}'; op vault list --group $group | tail -n +2; echo; done > ~/tmp/1p-groups2vaults.txt


Generate a list of all 1Password groups and the vaults which they are mapped to.

		# awk '{$1=""; print $0}' /tmp/file.txt
# awk '{for (i=2; i<=NF; i++) printf $i " "; print ""}' /tmp/file.txt

Two different ways to use awk to print the entire line except the first field.  The first option is simpler but leaves a leading space.

		# git clean -fdx

Remove all files which aren't managed by Git.

		# docker service update captain-captain --force

Restart Caprover

		# docker service logs srv-captain--APP_NAME --since 60m --follow

View Caprover logs

		User-agent: GPTBot
Disallow: /

Add to your websites /robots.txt to disable OpenAI's GPT bot from scraping your site.

		# gists () { links -dump -width 512 | egrep -v "^ $" | egrep --color=auto -A2 "^ # .$*.*$"; }

Add to your ~/.zshrc and then you can search these gists from the shell with: gists <keyword>

		macos# brew leaves
List Homebrew packages which were installed manually (as opposed to pulled in as a dependency of something else).

		macos# sudo tmutil listlocalsnapshots /
macos# sudo tmutil deletelocalsnapshots /
List all local snapshots and then delete them.

		macos# brew cleanup -s
Delete cached files in ~/Library/Caches/Homebrew/

		# cat blah.log | xpipe -n 1000 -J % /bin/sh -c "gzip > %.gz"
Split blah.log into multiple gzip’d files each 1000 lines long.

		# wget --restrict-file-names=windows -k --adjust-extension --span-hosts --convert-links --backup-converted --page-requisites
Download an archiveable copy of a website using wget.

		# sqlite3 :memory: -cmd '.import -csv taxi.csv taxi' 'SELECT passenger_count, COUNT(*), AVG(total_amount) FROM taxi GROUP BY passenger_count'
Load a CSV into an in memory SQLite database and then do a SQL query on the data (details).

		# shot-scraper '' --width 800 --height 600 --retina
Take a screenshot of a website (details).

		linux# Press escape when Grub menu appears.
grub> ls
grub> set root=(hd0,msdos5)
grub> chainloader /efi/boot/grubx64.efi
grub> boot
Boot from USB stick via Grub (details).

		# dog -S TXT
Look up TXT records for using DNS-over-TLS from the DNS server

		# docker exec -it -u www-data nextcloud php occ user:resetpassword adam
Change password for user adam from the commandline on a Dockerised NextCloud installation.

		# wget -q -O - | jq -r '.[].name'
List all tags (versions) available for a Docker image at

		# ssh-keyscan -D

Generate SSHFP DNS entries for a host remotely.  See also sshfp2cf for automatically creating SSHFP records on Cloudflare.

		linux# lsblk -f
Get UUID for disk partition to add to /etc/fstab.

		# diff --brief --recursive /backup/music /media/music
Recursively compare two folders and see differences in a format useful to humans.

		# bw get item 937b567a-a8a4-4ede-9fc1-adf000c17a4b |jq '.login | .password'
Use the BitWarden commandline to retrive an item and then jq to only print the password. More information at

		# ssh -o KexAlgorithms=+diffie-hellman-group1-sha1 admin@
How to SSH to a Cisco switch which only supports old key exchange algorithms. The error you get without is: no matching key exchange method found. Their offer: diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1

		macos# networkQuality
New tool built into macOS 12 (Monterey) to test quality of internet connection (some details at #speedtest

		# ( echo -e "one
three"; echo -e "two
four" 1>&2 ) 2> >(egrep "o|e")
How to only grep stderr without redirecting all of stderr to stdout. Can use “| tee 2> >(grep …)” on a command which doesn’t support redirection.

		# git init && git remote add origin && git fetch && git checkout -t origin/main -f
Clone a git repository into a non-empty directory (note that this overwrites exiting files).

		zsh# if (( ${+commands[]} )); then echo found; else echo not found; fi
Zsh trick for checking to see if a command exists without spawning a shell.

		linux# umount /data; e2fsck -f /dev/xvda3; resize2fs -b /dev/xvda3; resize2fs /dev/xvda3; e2fsck -f /dev/xvda3; mount /data
Resize the filesystem /data (/dev/xvda3) to a larger than 32bit filesystem. Note the first resize2fs only converts to a 64 bits, the second actually resizes the filesystem.

		# curl -iLo -
Print headers and contents of to standard out, but follow and show any redirects.

		# docker exec -it -u www-data nextcloud-app php console.php files:scan --all
As the user www-data run the command php console.php files:scan –all on the container “nextcloud-app”.

		# swaks -tls --server --to --from --auth login --auth-user --header-X-Test "test email"
Use swaks (available via Homebrew) to send a TLS encrypted email via to authenticating as via the AUTH LOGIN mechanism.

		macos# netstat -nr -f inet
Print all the IPv4 routes

		# dateseq 2010-01-01 2010-01-10 --skip sat,sun
Print the list of dates between 2010-01-01 and 2010-01-10 skipping weekends. Just part of the amazing dateutils package.

		# ansible -Kb all -m shell -a 'ssh-keyscan >> /root/.ssh/known_hosts'
On every host defined in the Ansible inventory file, append’s SSH host key to root’s known_hosts file.

		bash# export TIMEFORMAT="%3lR"; time find /etc > /dev/null
Use the Bash builtin time to see how long a process takes to run. Output is formatted with the TIMEFORMAT variable (3 decimal places, long format, Elapse time). Details here.

		# export TIME="%E"
shell; /usr/bin/time find /opt/ > /dev/null
Use the time command to see how long a process takes to run. Output is formatted by the TIME variable (long format elapsed time). Details here.

		linux# ufw allow proto tcp from any to any port 80,443,8080:8090 comment 'web app'
Update UFW firewall to allow all TCP traffic to ports 80, 443 and 8080-8090 inclusive and adds a comment for the rule.

		macos# sudo dscacheutil -flushcache; sudo killall -HUP mDNSResponder
Flush DNS cache on Catalina.

		# sshpass -psecretpass rsync --progress test.ogg
Provide ssh password on commandline so you script commands like rsync.

		# docker ps | awk 'BEGIN { IGNORECASE=1 } /maria|mongo|mysql|postgis|postgres/ {print $NF}'
Case-insensitive searching with gawk by setting builtin variable.

		# docker ps | awk 'tolower($0) ~ /maria|mongo|mysql|postgis|postgres/ {print $NF}')
Case-insensitive searching with awk by forcing $0 to lowercase.

		# ansible -Kb webservers -m shell -a "dpkg -l cronic | grep '^ii'"
Run a shell command as root on all servers in the webservers group.

		# curl "
Submit DNS queries and get JSON answer back.

		# awk -F: '{print $NF}' /etc/passwd
Print the last, colon-delimited field of /etc/passwd. This works because NF is the number of fields (eg. 7 for /etc/passwd) and so this is the same as print $7.

		# openssl s_client -connect -starttls smtp
Connect to and initiate an SMTP STARTTLS session. More OpenSSL examples here.

		# cat /root/.ssh/ | ssh 'dd of=.ssh/authorized_keys oflag=append conv=notrunc'
If you can’t use ssh-copy-id to upload your ssh key to a remote host you might be able to use this.

		# sftp -o
Use sftp to connect to using as a jump host.

		# ansible-playbook --ssh-extra-args "-o" -l -K backups.yml
Run ansible-playbook with as an SSH jump host.

		# ssh-copy-id -o
Copy ssh key to leia via the jump host gateway.

		# sudo certbot certonly --dns-cloudflare --dns-cloudflare-credentials /path/to/cloudflare.ini -d --preferred-challenges dns-01
Create an SSL certificate for mera using Cloudflare DNS challenge. Format of cloudflare.ini is described here. Requires installing certbot and python3-certbot-dns-cloudflare packages in Ubuntu.

		# checkdmarc
Python tool (domainaware/checkdmarc)for looking up DMARC and SPF record and showing any warnings.

		# docker-compose -f docker-compose.yml -f volumes-cached.yml up
Build a composite docker compose file by overlaying volumes-cached.yml over the top of docker-compose.yml.

		bash# (set -- one two three 4 5 6 7; for i in $@; do echo $@; shift; done)
How to simulate bash commandline arguments.

		# git clone; git checkout stable-4548
How to clone a repo and then take it back to a specific branch.

		linux# awk '/32 host/ { print f } {f=$2}' /proc/net/fib_trie
How to look up the IP address on a Linux host which doesn’t have any binaries to look it up for you (eg. a minimal docker host without ifconfig or ip).

		bash# FILENAME="/tmp/spack.txt"; echo $(dirname $FILENAME); echo $(basename $FILENAME); TMPNAME=$(basename $FILENAME) && echo ${TMPNAME%.*}; echo ${FILENAME##*.}
How to break out parts of an absolute file path in Bash.

		macos# defaults write DSDontWriteNetworkStores -bool true; defaults write DSDontWriteUSBStores -bool true
Disable .DS_Store and AppleDouble files on network and USB devices.

		macos# tmutil listlocalsnapshots /; tmutil deletelocalsnapshots; /System/Library/Filesystems/apfs.fs/Contents/Resources/mount_apfs -s / /mnt
List all APFS snapshots on local disk; delete a snapshot to free up space and mount a snapshot to recover files.

		debian# aptitude purge ?config-files
Will delete all packages which have only been partially removed (eg. removed but not purged). Replace purge with search to get a list of partially removed packages.

		linux# systemctl enable avahi-alias
Turn on a systemd service at book (I think, need to read systemd docs …).

		linux# wpa_passphrase spacknet secret > /tmp/wpa_supplicant.conf; wpa_supplicant -B -c /tmp/wpa_supplicant.conf -i wlp5s0; dhclient wlp5s0
Bring up a wireless wpa2 network from the command line.

		# netstat -lntu
Show all currently open TCP and UDP ports.

		macos# dns-sd -B
Browse all mDNS/Bonjour/Zeroconf host names and services. Use -Z to get it in zone file format.

		# openssl req -x509 -out bitwarden.local.crt -keyout bitwarden.local.key -newkey rsa:2048 -nodes -sha256 -subj '/CN=bitwarden.local' -extensions EXT -config <( printf "[dn]nCN=bitwarden.localn[req]ndistinguished_name = dnn[EXT]nsubjectAltName=DNS:bitwarden.localnkeyUsage=digitalSignaturenextendedKeyUsage=serverAuth")
Oneliner to generate a self-signed SSL certificate.

		debian# apt-file search --regexp 'bin/netstat$'
Search packages for any file which matches the regex. Can also do a non-regex search as apt-file search bin/netstat. Nice because no longer require

		linux# systemctl set-default
Disable X11 starting on boot with GDM3 / systemd. Use get-default to read current setting. To restore X11 use

		# clamscan -r ~/Downloads/_scanme/
Use ClamAV to recursively scan ~/Downloads/_scanme for infected files.

		# docker cp 01c7b7a07733:/var/lib/mysql.tbz /tmp
Copy /var/lib/mysql.tbz from the container to /tmp.

		# docker-compose down; docker-compose up -d; docker-compose logs -f
Use docker-compose to stop the container, start the container and then show the logs (remember docker-compose has to be run from the directory with the docker-compose.yml file.

		debian# apt list --upgradable |grep "$(lsb_release -cs)-security"
Get list of outstanding security updates

		# docker container rm gifted_villani
Delete a stopped container (remember that stopped containers will only show up with a ls -a).

		# docker container prune
Delete all containers which aren’t running

		# docker run --name zabbix-appliance -t -p 10051:10051 -p 81:80 -d zabbix/zabbix-appliance:latest
Download (if necessary) the zabbix/zabbix-appliance:latest image and name it zabbix-appliance locally. Map container ports 10051/80 to 1051/81 respectively (so that external port 81 routes to internal container port 80)

		# docker exec -i -t zabbix-appliance /bin/bash
Ppen a shell on a running container

		# docker exec zabbix-appliance /usr/bin/mysql -h localhost -u zabbix -pzabbix zabbix -e "show tables;"
Open a shell on the container zabbix-appliance, and use mysql in the container to connect to containers MySQL and run the show tables; command.

		# docker exec -i -t zabbix-appliance /usr/bin/mysql -h localhost -u zabbix -pzabbix zabbix
Open an interactive MySQL shell on the container

		# docker exec zabbix-appliance /usr/bin/mysqldump --add-drop-table -u zabbix --password=zabbix zabbix > /tmp/zabbix.sql; docker exec -i zabbix-appliance /usr/bin/mysql -u zabbix --password=zabbix zabbix < /tmp/zabbix.sql
Dump and restore MySQL (drop tables means you don’t have to delete the db or tables before restoring)

		# docker update --restart=always zabbix
Change settings on an existing container (eg can’t use docker run on an existing container)

		# snmpwalk -v2c -c public localhost
Talk to the SNMP daemon on macOS 10.14.

		# sftp -q /vol/backups/

Silently download / from and save it /vol/backups. Good for cron.

		# docker cp /usr/bin/telnet.netkit zabbix-appliance_zabbix_1:/tmp
Copy the telnet to a Docker instance which doesn’t have it for debugging.

		debian# grep security /etc/apt/sources.list > /tmp/security.list; sudo apt-get -s upgrade -oDir::Etc::Sourcelist=/tmp/security.list
Create a temporary sources.list file which only contains security related updates. Then run apt-get in simulation mode to show what those updates are. Run without the -s to actually install security updates.

		# ansible-playbook -K -l stuff.yml
Only run the playbook stuff.yml on the host and prompt for a sudo password before running.

		# ansible all -m setup
Gather facts from all configured hosts (replace all with a hostname to run against a single host).

		# ansible all -m shell -a 'echo $TERM'
Get the value of $TERM from all hosts.

		# hostname -I
A way to get the IP address of a host which doesn’t have ifconfig or ip installed.

		# ps --no-headers -o "rss,cmd" -C php-fpm | awk '{ sum+=$1 } END { printf ("%d%sn", sum/NR/1024,"Mb") }'
Print the total MB used of php-fpm processes.

		# git remote add origin; git push -u origin master
Connects a local repository to a remote one and pushes contents of local repo.

		# git remote set-url origin
Changes the remote repo that the local repo is connected to.

		# curl
Download adamshand’s ssh public key from Github.

		# curl
Download adamshand’s ssh public key from Github in JSON.

		macos# lsbom -fls /private/var/db/receipts/org.insecure.nmap.*bom
Show all files installed as part of a package.

		macos# log show -predicate '(processImagePath contains "configd") && (eventMessage contains "en0: SSID pekapeka")' -style compact -last 1d | awk '{print $1,$2,$6,$8}' | sed 's/:[0-9][0-9].[0-9][0-9][0-9]//' | uniq
Uses the new log subsystem to print a tidy list of the connections to the “pekapeka” WiFi network in the last day.