/til

Today-I-Learned, the long way. The feed shows the most recent ones mixed in; this page collects all of them, by year, so I can grep my own brain later.

Filed under: linux · go · postgres · networking · perf.

2026

apr 11netstat -i is faster than ip -s link on boxes with 2k+ interfaces. The kernel API path is shorter.
apr 02Go's sync.Map is slower than map + RWMutex unless you have read-mostly workloads with disjoint keys.
mar 28postgres -D ... --single gives you a Postgres REPL that doesn't need the server running. Good for surgery on a corrupted cluster.
mar 14kubectl debug node/... -it --image=busybox drops you on the host without SSH. (Or with it, depending on the cluster.)
feb 27The Linux TCP accept queue silently drops SYNs when full. Watch ListenOverflows in nstat -a.
feb 11go test -run Foo/bar runs subtests. I've been typing full test names for five years like a chump.
jan 30systemd's ProtectSystem=strict + ReadWritePaths= is 80% of what people reach for Docker for. The other 20% is the network namespace.
jan 16tcpdump can decode TLS ClientHello SNI directly with -vvv and a grep. No Wireshark needed for the simple case.
jan 04ss -i shows congestion control algorithm and current cwnd. I don't need to read /proc for this.

2025

dec 19EXPLAIN (ANALYZE, BUFFERS, SETTINGS) is strictly better than plain EXPLAIN ANALYZE. Change my mind.
nov 21GODEBUG=schedtrace=1000 is the cheapest possible way to see if your Go program is goroutine-starved.
oct 30Postgres' pg_stat_io (v16+) is the single best thing to happen to capacity-planning a database in a decade.
sep 18bpftrace -l 'tracepoint:sched:*' shows every sched tracepoint by name. Better than re-reading /sys/kernel/tracing/events/.
aug 09journalctl _PID=1234 --grep="foo" exists. I had been piping to grep for six years.
jul 22Go's runtime/trace can be opened in go tool trace for one (1) goroutine at a time, and that's exactly the right granularity for chasing a stall.
jun 14iptables-nft and iptables-legacy are different binaries with the same name. update-alternatives picks at install time. This explains a lot of Tuesday afternoons.
may 03perf stat -e migrations tells you how often the scheduler is moving your process between CPUs. On Kubernetes, the answer is "more than you'd like".
apr 12Linux ships an in-kernel TLS implementation (kTLS). It's been there since 4.13. Most people I work with don't know.
mar 27Go's errors.Join (1.20+) is the right answer for "I have N errors and want to return all of them". I keep reaching for fmt.Errorf out of habit.
feb 14SQLite's PRAGMA synchronous=NORMAL in WAL mode is durability-equivalent to FULL for almost every reasonable failure model. I had been wrong about this since 2018.
jan 23chrt -p $$ shows your current shell's scheduling policy. Almost always SCHED_OTHER, but the day it isn't, you'll be glad you checked.

2024

nov 11nft monitor trace shows every nftables rule a packet hits. Took me ten minutes to find. Will save me ten hours next time.
oct 02Postgres VACUUM (FREEZE) on a partition can be cheap, almost trivial, if the partition is read-only.
aug 28git replace exists for editing history without rewriting it. I had been re-cloning to fix this for a decade.
jun 17setpriv --bounding-set can drop Linux capabilities for a child process without writing C. Better than I assumed.
apr 30blkparse + blktrace is the disk-IO equivalent of strace. Use it once, never go back.
feb 19Go panics in deferred functions can be recovered by the outer recover. I had assumed otherwise. (I was 8 years into writing Go.)

The feed has a TIL filter chip if you want to scroll the freshest ones. This page is the longer view.

Last edited: this month. New ones land here within a day of the feed.