Not much technical writings here - just my opinion on some tech-related topics.
I do a lot of lab setups on my internal network, which means that there is a lot of tearing down and rebuilding machines. That’s pretty easy to do with a deployment server, but I often re-use the same hostname. SSH doesn’t like that. This is because the server fingerprint changes when I re-build a VM with the same hostname. If this was a production server, and the fingerprint randomly changes, then yes, this is an issue to look into. But in a lab environment…who cares?
The Problem
Here is what happens when I ssh into a box for the first time. For this example, I provisioned an Ubuntu machine called dev-tinker.
[andrew@a11-deploy ~]$ ssh dev-tinker The authenticity of host 'dev-tinker (192.168.4.123)' can't be established. ECDSA key fingerprint is 7f:26:68:e8:9e:ef:cf:08:98:cd:b3:fb:b2:f4:a3:2b. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'dev-tinker,192.168.4.123' (ECDSA) to the list of known hosts. andrew@dev-tinker's password: Welcome to Ubuntu 12.04.4 LTS (GNU/Linux 3.2.0-58-generic x86_64) * Documentation: https://help.ubuntu.com/ [andrew@dev-tinker ~]$
So what happened here? SSH doesn’t recognize dev-tinker because I am connecting to it for the first time, so it wants to verify with me that I am connecting to the correct machine. Since any machine can spoof their hostname, it is verifying with the ECDSA key fingerprint. This is generated with the SSH server encryption keys. (Note that this could be an RSA or DSA fingerprint, depending on the server keys that are installed.) What you are supposed to do is verify this is in fact the fingerprint of the server you intend on connecting to.
But how do you do that? In an ultra-secure network, the key fingerprints are probably documented somewhere. Otherwise, log into the machine in question via other means (direct keyboard access, etc), and run this command.
[andrew@dev-tinker ~]$ ssh-keygen -lf /etc/ssh/ssh_host_ecdsa_key.pub 256 7f:26:68:e8:9e:ef:cf:08:98:cd:b3:fb:b2:f4:a3:2b root@dev-tinker (ECDSA)
Looks like I’m OK because the fingerprint matches the one provided to me in the connection prompt. So I type in yes, and continue with my password. Boom, I’m in.
Now every other time I login to that machine, all I provide is my password.
[andrew@a11-deploy ~]$ ssh dev-tinker andrew@dev-tinker's password: Welcome to Ubuntu 12.04.4 LTS (GNU/Linux 3.2.0-58-generic x86_64) * Documentation: https://help.ubuntu.com/ Last login: Mon Jan 27 18:06:29 2014 from a11-deploy [andrew@dev-tinker ~]$
Now let’s say I tore down dev-tinker and rebuilt it with a fresh OS. Let’s see what happens when I connect to the new machine.
[andrew@a11-deploy ~]$ ssh dev-tinker @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed. The fingerprint for the ECDSA key sent by the remote host is a4:dd:91:dc:e1:24:46:df:0a:47:54:bf:fc:57:c0:ed. Please contact your system administrator. Add correct host key in /home/andrew/.ssh/known_hosts to get rid of this message. Offending ECDSA key in /home/andrew/.ssh/known_hosts:1 remove with: ssh-keygen -f "/home/andrew/.ssh/known_hosts" -R dev-tinker ECDSA host key for dev-tinker has changed and you have requested strict checking. Host key verification failed. [andrew@a11-deploy ~]$
Oh noes!!!! (play some audible breach alarm for added effect) This time, I see a seemingly serious error message, and I find that I am back at the local bash prompt. The login attempt failed. Remember that before the remote machine was re-imaged, the fingerprint was 7f:26:68:e8:9e:ef:cf:08:98:cd:b3:fb:b2:f4:a3:2b, and it is now a4:dd:91:dc:e1:24:46:df:0a:47:54:bf:fc:57:c0:ed.
So I know that there is nothing to be worried about. Let’s follow the error message’s advice and remove the old fingerprint.
[andrew@a11-deploy ~]$ ssh-keygen -f "/home/andrew/.ssh/known_hosts" -R dev-tinker /home/andrew/.ssh/known_hosts updated. Original contents retained as /home/andrew/.ssh/known_hosts.old
The above error message doesn’t say this, but we need to remove the fingerprint record for the IP address, too.
[andrew@a11-deploy ~]$ ssh-keygen -f "/home/andrew/.ssh/known_hosts" -R 192.168.4.123 /home/andrew/.ssh/known_hosts updated. Original contents retained as /home/andrew/.ssh/known_hosts.old
Now connect to the machine again.
[andrew@a11-deploy ~]$ ssh dev-tinker The authenticity of host 'dev-tinker (192.168.4.123)' can't be established. ECDSA key fingerprint is a4:dd:91:dc:e1:24:46:df:0a:47:54:bf:fc:57:c0:ed. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'dev-tinker' (ECDSA) to the list of known hosts. andrew@dev-tinker's password: Welcome to Ubuntu 12.04.4 LTS (GNU/Linux 3.2.0-58-generic x86_64) * Documentation: https://help.ubuntu.com/ Last login: Mon Jan 27 18:11:06 2014 from a11-deploy
Looks like we are back in business.
Have SSH Chill Out With the Minor Offenses
So basically SSH needs to calm down. There is no attack. There is nobody hijacking my connection. Take off the tin foil hat and calm down. In all seriousness though, it’s good that the default configuration errors on the side of caution. But in our case, we are going to change a few settings on the client machine to make this a little less strict. This is the host Linux machine that you are SSH’ing from.
We will make the changes in your local SSH config file.
[user]$ vim ~/.ssh/config
Take a look at the below configuration.
Host *
CheckHostIP no
StrictHostKeyChecking no
UserKnownHostsFile /dev/null
The CheckHostIP and StrictHostKeyChecking lines are pretty self-explanatory. Refer to the ssh_config manual page if you need further information.
The UserKnownHostsFile option probably needs more explanation. This is sort of a hack. There is no way to prevent SSH from saving fingerprints. They way around that is to tell SSH to save the host fingerprints to /dev/null. This is basically the same as discarding the information.
So with this configuration, you will see this warning:
Warning: Permanently added 'dev-tinker' (ECDSA) to the list of known hosts.
That’s because SSH is saving the fingerprint to /dev/null each time, which is not an actual file. So when SSH attempts to read this data back, it will come up with nothing and assume it’s a new machine again. It’s also giving a warning instead of the yes/no prompt because of the StrictHostKeyChecking setting.
If you were to place these lines in ~/.ssh/config, you would effectively disable host key checking. Is that what you want to do? Great, you’re done! Add these lines to the file, and save.
…but there could be a problem with this configuration. If you connect out to production boxes, you probably would want to keep the default security settings enabled. I’m assuming that a man-in-the-middle attack on your lab or home lab is pretty rare, so let us only apply this new configuration to SSH connections on your local subnet.
Take a look at this new ~/.ssh/config file.
Host dev-*
CheckHostIP no
StrictHostKeyChecking no
UserKnownHostsFile /dev/null
All I changed was the Host line, and I am limiting the connection settings to all machines that start with “dev-“. Your mileage for this option will vary. For me, all my development boxes have that hostname prefix. So any VM that starts with dev likely will not last long, so I might as well not save the key when connecting. Again, that’s my personal preference. Yours will vary.
I should also note that some other blogs out there report that you could add something like Host 192.168.1.* to the config to make it easier for all machines on your LAN…but who connects to their machines by IP address?
Final Warning
So all of this is my opinion. Lowering security in this way may end very badly for you. Please make sure that you limit the hosts this applies to as much as possible. Read man pages and research what each setting does that you are changing.