Making SSH and SCP work for automated deployments via in GitLab or any other CI/CD system
So you want to copy your build artifacts to some server but ssh complains about host key verification and missing credentials?
This is really about automated passing of credentials and host key verification.
User and Password: use public keys
On the server, create a new key pair. It’s pretty important that you don’t use your personal key pair since we’re going to share the private key with GitLab. Don’t do that with your personal private key — it is really your identity file.
ssh-keygen -f id_rsa.yourserver
Make sure you leave all passphrases empty. Just hit return when asked to enter a phrase.
This will create a id_rsa.yourserver
and id_rsa.yourserver.pub
. That’s your fancy key pair. Now add the public key id_rsa.yourserver.pub
to you server’s authorized keys file:
cat id_rsa.yourserver >> ~/.ssh/authorized_keys
Inspect the new private key via cat id_rsa.yourserver
— — -BEGIN RSA PRIVATE KEY — — -
MIIEpAIBAAKCAQEAufBB3Xm52WYtSxjZKpSInqoEejk1ZP7QOzYvdOzxpOvZVPwF
V2WriNDd6mCex6d4ULxZnxswb8ZjXnCiUDVtnHADA6AH6y3UvMUqfzxs5QElgQKG
NlI/ . . I REMOVED TWENTY LINES FOR BREVITY . . . . . . . . .HzG
zhbJE9Kv41bpADRMG+HmGU9CpHbN4JkY1gNKJBxrrxTkeR5zkBGv7exSTM5+9Bun
pxX5cmczky2/BBovOur3soBM4QkQPwl26E9k0WLrT88YjhO1z0mMJmfDPCgZgh87
iPuAmQKBgQCIAk1QXQC+ . AND THIS KEY IS NOT A REAL KEY . . aIaUC
lwaj0yhdJPTWsHqWc0Rhhbuk7oEjn8NriyduPLU/0d+iA9bQx9ETi+kCF+A3sYY+
qIUq8j28hIDBJGUZldQxhhHIXwICG47/GzLEw7zop0szez++gOCZTw==
— — -END RSA PRIVATE KEY — — -
and add the contents that should look something like above (just way bigger) to your GitLab CI/CD variables. It’s within your project under settings, then CD/CD. Expand the Variables section and set
- type: File
- Key:
IDENTITY
- Value: the private key above (your key, not the gibberish above)
Then in your .gitlab-ci.yml
you can reference that identity file as $IDENTITY
. For example as parameter to ssh
ssh -i $IDENTITY you@yourserver.com "systemctl restart your.service"
Unfortunately, only. This is not enough:
Host key verification
ssh
wants to make sure that you are connecting to the right server, so the server will present a fingerprint of its host key. Only if that key is known to ssh
it will let you connect to that server
On your developer machine ssh
will show you the fingerprint on screen and ask you if you really want to connect to that server. You will have to type yes
usually. Then, the fingerprint will be added to your ~/.ssh/known_hosts
file so ssh
doesn’t need to ask again for that server.
Unfortunately, typing isn’t an option in automated scenarios like your CI/CD runner environments. You can either
- disable host key verification (for security reasons, don’t!)
- set upSSHFP records in your DNS
- or simply ship a
known_hosts
file in your repository
Crafting the known_hosts
file
The simplest solution is to inspect the verbose (the -v
option) output of ssh -v you@some_server.com
when connecting to the server. Look for
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:4F7nGBwJqx97Er3XXXXXXXXX/sS4MGYNvrwYYRqBqL8
debug1: Host ‘10.0.2.100’ is known and matches the ECDSA host key.
debug1: Found key in /home/mart/.ssh/known_hosts:2
Yay, this known_hosts:2
from the last line above tells you that you want to copy line number 2. Extract that to a file and name it conveniently.
sed -n ‘2,+1p’ ~/.ssh/known_hosts > known_hosts
Now, check if that works:
ssh -v -o "UserKnownHostsFile=./known_hosts" you@some_server.com
If you can login, great! Add that known_hosts
file to your repo and you can then copy stuff in GitLab’s CI/CD runner.
scp \
-o "UserKnownHostsFile=./known_hosts" \
-i $IDENTITY some_build \
you@some_server.com:/some/target
And it works!
Final words
There’s much more to be automated here :-) And yes, this is more of a general documentation for myself on how to do stuff. Because once it runs I usually don’t touch it that often and then forget about how I managed to get it to work in the first place.