May 18, 2012

Windows 7 IKEv2 with StrongSwan Certificate Generation Guide

Windows 7 supports IPSec IKEv2 with machine certificate authentication. Using StrongSwan on Linux for server, this is a good solution for Road Warrior remote access.

The problem with Windows 7 IKEv2 client is that it does not provide any log for trouble-shooting at all. So if it does not like something in your setup, it simply throws an error number and a very vague error message.  Error 13806 is one of them.

I spent a few days finally got my Windows 7 running IKEv2 machine authentication with my strongswan server. Below are a few tips for Error 13806:

1. Error 13806 is because the machine certificate or CA certificate  on the Windows 7 has problems. This is NOT about the Server's certificate. So focus on fixing the Windows 7 certificates. The error message does not really tell you this clearly.

2. There are a few variables in your client certificate that are important, whether you use OpenSSL or StrongSwan ipsec pki to generate your certs:


  1. The Common Name (CN) in the subject of your certificate
  2. The Subject Alternative Name (SAN) 
  3. The Key Usage
  4. The Extended Key Usage, where you can specify "clientAuth", "serverAuth", "1.3.6.1.5.5.8.2.2", either one or a combination of them.


Here is what works and what does not work in my tests (For Windows 7 client only, not for StrongSwan server):

  • You do not need to set Key Usage and Extended Key Usage(EKU). If you set them, some combinations may cause trouble. see more below.
  •  You do not need to set SAN. If you really want to set it, it has to be exactly the same as the CN. Otherwise you will get Error 13806.
  •  Set CN to either a full DNS name such as "win7.mycompany.local", or just a string name "win7". Either one works.
  •  If you set Key Usage to "nonRepudiation, digitalSignature, keyEncipherment", and Extended Key Usage to "1.3.6.1.5.5.8.2.2,serverAuth,clientAuth", you MUST set your CN to the long form of DNS name such as "win7.mycompany.local". Using the short form "win7" will fail and cause error 13806.

The name win7.mycompany.local does not have to be in your DNS server.

What works:

  • CN="win7", no SAN, No Keyusage, No EKU.
  • CN="win7", SAN="win7", No Keyusage, No EKU.
  • CN="win7.mycompany.local", SAN="win7.mycompany.local", No Keyusage, No EKU.
  • CN="win7.mycompany.local", SAN="win7.mycompany.local",keyUsage = nonRepudiation,   digitalSignature, keyEncipherment,  extendedKeyUsage = 1.3.6.1.5.5.8.2.2,serverAuth,clientAuth

What does not work (Error 13806):

  • CN="win7", SAN="something else", No Keyusage, No EKU.
  • CN="win7", SAN="win7",keyUsage = nonRepudiation,   digitalSignature, keyEncipherment,  extendedKeyUsage = 1.3.6.1.5.5.8.2.2,serverAuth,clientAuth

Your server certificate should have the following:
  •   CN 
  •   SAN, which should be exactly the same as your CN
  •   keyUsage = nonRepudiation,   digitalSignature, keyEncipherment,  
  •   extendedKeyUsage = 1.3.6.1.5.5.8.2.2,serverAuth
Your Windows 7 VPN connection host name should exactly the same as the Server's CN. And that CN name should be DNS resolvable. The easiest way to do that in a test environment is to add the CN name to your local computer's host file, located at "c:\windows\system32\drivers\etc\hosts"

Of course you should read StrongSwan's Wiki page on Windows 7. Very helpful, although can be a little bit more detailed on the client side. (http://wiki.strongswan.org/projects/strongswan/wiki/Windows7)

Andreas at StrongSwan also pointed out to me that although ECDSA has been supported by Windows since Windows Vista, it only works in IKEv1. IKEv2 in Windows 7 and Windows 2008 R2 does not support ECDSA. I have also personally verified that this is true.

One more important note:

Please make sure all your certificates has the PKIX recommended "Authority Key Identifier" in it. Otherwise, Windows 7 will give an Error 13806.


If you use OpenSSL to generate your certs, make sure the following line is in your openssl.conf file:

authorityKeyIdentifier=keyid,issuer

"Subject Key Identifier" seems to be optional for Windows 7.

Another useful post on this subject: http://forums.opensuse.org/english/get-technical-help-here/network-internet/435097-strongswan-opensuse-11-2-quick-setup.html

More Update:
Following above-written rules, I was able to generate certificates using StrongSwan ipsec pki command for both Windows 7 and StrongSwan, and the IKEv2 connection was established.

Here are the scripts:

First, Generate the CA:



bits=2048
ipsec pki --gen --type rsa --size $bits --outform pem > ca.key
ipsec pki --self --flag serverAuth --in ca.key --type rsa --digest sha1 --dn "C=CH, O=strongSwan, CN=pkiCA" --ca > caCert.der

Next, generate two certs, one for Server and one for Win7

bits=2048
for user in server win7; do
    ipsec pki --gen --type rsa --size $bits --outform pem > $user.key
    flags="";
    [ "$user" = "server" ] && flags="--flag serverAuth";
    ipsec pki --pub --in $user.key --type rsa | ipsec pki --issue --cacert caCert.der --cakey ca.key --digest sha1 \
        --dn "C=CH, O=strongSwan, CN=$user.mycompany.local" --san "$user.mycompany.local" $flags --outform pem > $user.cert
done;

Next convert Win7 cert to P12 format:

# Usage: ./convert-to-p12.sh Username
user=$1
openssl pkcs12 -export -inkey $user.key -in $user.cert -name "$user" -certfile caCert.der -caname "vpnserver7" -out $user.p12

then run ./convert-to-p12.sh win7

Next convert Server key to DER format so that Strongswan can take it:

openssl rsa -in $1.key  -out $1.der -outform DER

where $1 is "server"


17 comments:

  1. When running the first script:
    bits=2048
    ipsec pki --gen --type rsa --size $bits --outform pem > ca.key
    ipsec pki --self --flag serverAuth --in ca.key --type rsa --digest sha1 --dn "C=CH, O=strongSwan, CN=pkiCA" --ca > caCert.der

    I get:
    unrecognized option '--flag'
    Error: invalid --self option
    strongSwan 4.4.0 PKI tool

    However, 2 certs do get created but maybe created without the --flag option

    ReplyDelete
  2. I am using Debian Stable, and the ipsec pki version is 4.4.1 PKI tool. And it seems to work fine.


    strongSwan 4.4.1 PKI tool
    usage:
    pki --self [--in file] [--type rsa|ecdsa]
    --dn distinguished-name [--san subjectAltName]+
    [--lifetime days] [--serial hex] [--ca] [--ocsp uri]+
    [--flag serverAuth|clientAuth|ocspSigning]+
    [--digest md5|sha1|sha224|sha256|sha384|sha512] [--outform der|pem]
    --help (-h) show usage information
    --in (-i) private key input file, default: stdin
    --type (-t) type of input key, default: rsa
    --dn (-d) subject and issuer distinguished name
    --san (-a) subjectAltName to include in certificate
    --lifetime (-l) days the certificate is valid, default: 1095
    --serial (-s) serial number in hex, default: random
    --ca (-b) include CA basicConstraint, default: no
    --pathlen (-p) set path length constraint
    --flag (-e) include extendedKeyUsage flag
    --ocsp (-o) OCSP AuthorityInfoAccess URI to include
    --digest (-g) digest for signature creation, default: sha1
    --outform (-f) encoding of generated cert, default: der
    --debug (-v) set debug level, default: 1
    --options (-+) read command line options from file

    ReplyDelete
    Replies
    1. You need the "serverAuth" flag for certificates used by the StrongSwan in server mode.

      Delete
  3. I exactly runned your commands and the following files were made :

    caCert.der
    ca.key
    server.cert
    server.key
    win7.cert
    win7.key
    win7.p12

    Then set the leftcert=server.cert in ipsec.conf file and ": RSA server.key" in ipsec.secrets file and restart the ipsec.
    Also I've installed caCert.der and win7.p12 using the guide of strongswan windows 7 certificate import...

    But I still receive the 13086 error. Please help .

    ReplyDelete
  4. Thanks for posting the findings.
    I have question about commercial CA which will be able to provide or sign the cert with "extendedKeyUsage = 1.3.6.1.5.5.8.2.2,serverAuth". Contacted Entrust support apparently they don't do the "extendedKeyUsage = 1.3.6.1.5.5.8.2.2". Do you have experience with commercial CA which will provide this EKU property or use private CA?

    ReplyDelete
  5. Hi there, I also wrestled with this Error 13806 problem for several hours. Your notes were a big help in narrowing down my problem even though my problem turned out to be quite different.

    My problem was that the private key was not being imported properly, despite showing up as "You have a private key that corresponds to this certificate", it wasn't actually able to open the private key (Error 0x80090011) for use. This turned out to be because the code I was using to import the PKCS#12 file was missing the X509KeyStorageFlags.PersistKeySet flag.

    Along the way I also discovered that the relevant diagnostic information was hiding in the Windows Event Viewer, under Windows Logs\Security. I'm not used to working with Windows so finding the logs was half the battle for me, but it was much more helpful than the vague error 13806 dialog.

    ReplyDelete
  6. This didn't work. openssl would not combine the keys into a p12 format, it kept saying it couldn't load the certificates. Could you post the code without the scripts...thanks

    ReplyDelete
  7. i figured it out...you created a der CA certificate. Once I converted it to PER, the process works. Thanks!

    ReplyDelete
  8. Do you already have the code error 809 with the IKEv2 vpn on windows 7??
    I'm not in the same situation like us because i want to establish an IPsec tunnel between two Windows 7 gateways.

    I've heard that i need to allow ports or NAT and i do that but i still have this error!

    Maybe someone knows how to fix that?

    Thanks

    ReplyDelete
  9. Hi, i try this and it works fine when i generate the certificates using ipsec pki,
    but when i try with openssl, it fails.....

    On the client :
    CN=passA.mycompany.local

    in the openssl.cnf file i with a session for extensions with :
    SubjectAltName=DNS:passA.mycompany.local,
    keyUsage = nonRepudiation, digitalSignature, keyEncipherment,
    extendedKeyUsage = 1.3.6.1.5.5.8.2.2,serverAuth,clientAuth

    On the server :

    CN = passB.mycompany.local

    openssl.cnf :

    SubjectAltName = passB.mycompany.local
    keyUsage = nonRepudiation, digitalSignature, keyEncipherment
    extendedKeyUsage = 1.3.6.1.5.5.8.2.2,serverAuth


    So what i'm doing wrong?
    Can you have a the command you with openssl? or the configurations you made in the openssl.cnf file?

    Thanks

    ReplyDelete
  10. hi

    am stuck in this:
    ipsec pki --gen --type rsa --size $bits --outform pem > ca.key
    no response is coming after it can you advice

    ReplyDelete
  11. Sorry for commenting on such an old entry of your blog, but did you happen to have to update this config for a windows 10 system in the meantime ? I'm currently stuck with a strongswan setup that has worked for years with windows 7 clients, but win10 is giving me error 13806 (and all the discussion threads I could find either die out unresolved or link to your blog entry)..
    Thanks
    Martin

    ReplyDelete
  12. sorry I have not tried this on a Windows 10 computer.

    ReplyDelete
    Replies
    1. Thank you for the quick response. It appears this was just a blunder on my part:
      The existing setup was old enough to still be using 1024-bit rsa keys (which Microsoft apparently no longer accepts* ), and when I generated a new CA I managed to overlook the "basicConstraints=CA:FALSE" in openssl.conf and made
      me a dud CA certificate.
      * according to technet cc751157 which I saw cited in this context, it seems that
      support for sha1 digest algorithm is being phased out in favor of sha2 as well, so this could become another pitfall in the near future.

      Delete
    2. Hello, do you mean your old configuration is always working on win10 ? (I am stucking with this setup that has worked on win7 and win8.1, but Win10 is giving me a strange error 0x80090308 Invalid Token, but google reports that this message can be reported instead of the Error 13806..)

      Delete
    3. Hello, do you mean your old configuration is always working on win10 ? I'm stuck with a strange error (Invalid Token 0x80090308) but Google reports that this error can be displayed instead of the Error 13806. But the configuration given in this article worked well on win 7 and win8.1, and not on win10, so i would like to know if there is any change to make on the certificates for win10 or in the strongswan config file ? Thank you.

      Delete
  13. I have been caught in this problem for a very long time, Thanks a lot for your splendid blog! Good article on this topic are rare, I followed your step by step guide, Finally my problem fixed!!! Thanks again.

    ReplyDelete