Monthly Archives: November 2015

SSHFP records: A different way to check host keys

The JHU ACM has many machines that users might want to SSH into. The JHU ACM also occasionally reinstalls those machines for various reasons, which changes their SSH keys. And I’m usually not near the office anymore to verify for myself that the keys I’m seeing are correct.

The ACM happens to do DNSSEC on their domain, though, and I run a validating resolver, so I can use SSHFP records as a trusted source of SSH fingerprints.

Using SSHFP records

On my end I added this line to ~/.ssh/config:

VerifyHostKeyDNS yes

And ssh checks the key it sees against SSHFP records (falling back to known_hosts if there aren’t any), giving the usual loud warning about any discrepancies it may come across.

(It even knows the difference between a DNSSEC-authenticated response and the other kind, and it will still ask you whether the key is OK if the matching SSHFP record is unauthenticated and the key isn’t in known_hosts. You can make it ask even on authenticated SSHFP records by setting VerifyHostKeyDNS to ask instead.)

That’s all well and good… what about the other end?

ssh-keygen can generate SSHFP records for a given key file when run with the -r option. We wrapped it in a script, which you can run on the machine in question to get a zonefile snippet (and which only looks at public key files, and so doesn’t need to be run as root):

# Generate a zone file snippet for SSHFP records for the host this is run on.

set -e

# This is in increasing order of algorithm ID in the SSHFP records.
for algo in rsa dsa ecdsa ed25519; do
  if test -f "$keyfile"; then
    if ! ssh-keygen -r "`hostname`" -f "$keyfile" | awk '
      # Do not bother with SHA1 fingerprints - only process SHA256 ones.
      # Reformat the lines so the whitespace matches common zone file layout.
      if ($5 == "2")
        print $1 (length($1) < 8 ? "\t" : "") "\t" $2 "\t" $3 "\t" $4, $5, $6
    }' | grep SSHFP  # the grep is just to check whether there was any output
      if test x"$algo" = xed25519; then
        echo "; placeholder for SSHFP record for `hostname` ed25519 key"
        # Complain on stderr about ed25519 SSHFP records not yet being supported
        # so the output of this script can be sensibly directed into a zone file.
        cat >&2 <<"EOF"

There is an ed25519 key, but ssh-keygen did not turn it into an SSHFP
record. It is probably not a new enough version to support doing so -
if this is the case, remember to regenerate the records once it is.
        echo "unable to generate SSHFP record for $keyfile" >&2
        # ssh-keygen puts some error messages on stdout (shame!), so re-run
        # the failing invocation to get that output and throw away stderr.
        ssh-keygen -r "`hostname`" -f "$keyfile" 2>/dev/null
        exit 1

(Remove the if ($5 == "2") if you also want SHA1 SSHFPs, which older versions of ssh might need. The ugly conditionals involving ed25519 are because at the time the script was written, Debian stable’s OpenSSH didn’t support SSHFP for ed25519, Debian testing’s did, we had machines running both, and the OpenSSH tools could stand to do much better when it comes to error exit codes and to what goes to stdout and what goes to stderr. The script is also available as /afs/

Since we would occasionally forget to update the SSHFP records when reinstalling a machine (or when the key changed for some other reason), I wrote a Nagios plugin for checking SSHFP records, which the ACM now uses. It checks that all key types offered by the server have SSHFP records, that every SSHFP record goes to a key type the server offers, and that every SSHFP record is correct.

The b(ack)log

I’ve been highly remiss on posting anything here.

Which is terrible, because over the last few years, I have very regularly had some experience well worth a blog post, whether it’s to do with sysadminly things, coding, scripting to make my life easier, otherwise improving how I interact with my daily-driver laptop, engineering something interesting, reverse-engineering something interesting, other technical things, or even other non-technical things.

So I want to share some of that.

I will do my best to regularly post about something I did in that timespan – at the very least, weekly. I have built up a long list to pick from and flesh out, which should keep me going for a long while yet.

Here’s a high-level overview of what else has been going on with me:

  • I’m now all finished with my master’s degree in computer science from Johns Hopkins.
  • The JHU ACM moved to a new building and rebuilt its systems from the ground up, during which I was on their sysadmin team (and had been for years before that, and continue to be) and was part of implementing any number of interesting bits of functionality.
  • I spun up my own Kerberos realm and AFS cell to play around with (since the ACM uses those too) and have put them to good use. (Expect some posts about making particular services use GSSAPI authentication and interact more nicely with having their backing stores in AFS.)
  • I went to DebConf 14 (and used an Amtrak USA Rail Pass to expand my trip there into a month-long adventure across the continental United States and back, spending time in seven extra cities).
  • I gave a talk at FOSSCON 2013 about the challenges I experienced with the FoFiX project; I went to FOSSCON 2015 too (missed 2014 because of the DebConf trip).
  • I’ve taken up speedrunning (completing video games as fast as possible) and Twitch streaming. (Expect some posts about glitches and quirks in games and game hardware, and about the technical side of streaming from Unix-like systems.)
  • I finally got a new laptop in winter 2014 and dealt with a number of things for the first time while setting it up, such as configuring disk encryption and handling UEFI.
  • I went to AGDQ 2015 and SGDQ 2015 (and couched two runs at SGDQ, mainly to provide technical background on glitches).
  • I finally got a new VPS in summer 2015 and am moving my services over. (I challenged myself to move each service with as close to zero downtime as possible; this was much easier for some services than others. More details in future posts.)

As I said, I will try to write about no less than one thing every week; there are all kinds of stories (technical and not) built up over this long blog gap. See you then!

(And let me know in the comments if there’s anything in particular that you’d like me to try to prioritize writing about.)