Windows 7 IKEv2 with StrongSwan Certificate Generation Guide

Windows 7 supports IPSec IKEv2 with machine certificate authentication. Using StrongSwan on Linux for server, this is a good solution for Road Warrior remote access.

The problem with Windows 7 IKEv2 client is that it does not provide any log for trouble-shooting at all. So if it does not like something in your setup, it simply throws an error number and a very vague error message.  Error 13806 is one of them.

I spent a few days finally got my Windows 7 running IKEv2 machine authentication with my strongswan server. Below are a few tips for Error 13806:

1. Error 13806 is because the machine certificate or CA certificate  on the Windows 7 has problems. This is NOT about the Server's certificate. So focus on fixing the Windows 7 certificates. The error message does not really tell you this clearly.

2. There are a few variables in your client certificate that are important, whether you use OpenSSL or StrongSwan ipsec pki to generate your certs:

1. The Common Name (CN) in the subject of your certificate
2. The Subject Alternative Name (SAN) 
3. The Key Usage
4. The Extended Key Usage, where you can specify "clientAuth", "serverAuth", "1.3.6.1.5.5.8.2.2", either one or a combination of them.

Here is what works and what does not work in my tests (For Windows 7 client only, not for StrongSwan server):

• You do not need to set Key Usage and Extended Key Usage(EKU). If you set them, some combinations may cause trouble. see more below.
•  You do not need to set SAN. If you really want to set it, it has to be exactly the same as the CN. Otherwise you will get Error 13806.
•  Set CN to either a full DNS name such as "win7.mycompany.local", or just a string name "win7". Either one works.
•  If you set Key Usage to "nonRepudiation, digitalSignature, keyEncipherment", and Extended Key Usage to "1.3.6.1.5.5.8.2.2,serverAuth,clientAuth", you MUST set your CN to the long form of DNS name such as "win7.mycompany.local". Using the short form "win7" will fail and cause error 13806.

The name win7.mycompany.local does not have to be in your DNS server.

What works:

• CN="win7", no SAN, No Keyusage, No EKU.
• CN="win7", SAN="win7", No Keyusage, No EKU.
• CN="win7.mycompany.local", SAN="win7.mycompany.local", No Keyusage, No EKU.
• CN="win7.mycompany.local", SAN="win7.mycompany.local",keyUsage = nonRepudiation,   digitalSignature, keyEncipherment,  extendedKeyUsage = 1.3.6.1.5.5.8.2.2,serverAuth,clientAuth

What does not work (Error 13806):

• CN="win7", SAN="something else", No Keyusage, No EKU.
• CN="win7", SAN="win7",keyUsage = nonRepudiation,   digitalSignature, keyEncipherment,  extendedKeyUsage = 1.3.6.1.5.5.8.2.2,serverAuth,clientAuth

Your server certificate should have the following:

•   CN 
•   SAN, which should be exactly the same as your CN
•   keyUsage = nonRepudiation,   digitalSignature, keyEncipherment,  
•   extendedKeyUsage = 1.3.6.1.5.5.8.2.2,serverAuth

Your Windows 7 VPN connection host name should exactly the same as the Server's CN. And that CN name should be DNS resolvable. The easiest way to do that in a test environment is to add the CN name to your local computer's host file, located at "c:\windows\system32\drivers\etc\hosts"

Of course you should read StrongSwan's Wiki page on Windows 7. Very helpful, although can be a little bit more detailed on the client side. (http://wiki.strongswan.org/projects/strongswan/wiki/Windows7)

Andreas at StrongSwan also pointed out to me that although ECDSA has been supported by Windows since Windows Vista, it only works in IKEv1. IKEv2 in Windows 7 and Windows 2008 R2 does not support ECDSA. I have also personally verified that this is true.

One more important note:

Please make sure all your certificates has the PKIX recommended "Authority Key Identifier" in it. Otherwise, Windows 7 will give an Error 13806.


If you use OpenSSL to generate your certs, make sure the following line is in your openssl.conf file:

authorityKeyIdentifier=keyid,issuer

"Subject Key Identifier" seems to be optional for Windows 7.

Another useful post on this subject: http://forums.opensuse.org/english/get-technical-help-here/network-internet/435097-strongswan-opensuse-11-2-quick-setup.html

More Update:
Following above-written rules, I was able to generate certificates using StrongSwan ipsec pki command for both Windows 7 and StrongSwan, and the IKEv2 connection was established.

Here are the scripts:

First, Generate the CA:



bits=2048
ipsec pki --gen --type rsa --size $bits --outform pem > ca.key
ipsec pki --self --flag serverAuth --in ca.key --type rsa --digest sha1 --dn "C=CH, O=strongSwan, CN=pkiCA" --ca > caCert.der

Next, generate two certs, one for Server and one for Win7

bits=2048

for user in server win7; do

    ipsec pki --gen --type rsa --size $bits --outform pem > $user.key

    flags="";

    [ "$user" = "server" ] && flags="--flag serverAuth";

    ipsec pki --pub --in $user.key --type rsa | ipsec pki --issue --cacert caCert.der --cakey ca.key --digest sha1 \

        --dn "C=CH, O=strongSwan, CN=$user.mycompany.local" --san "$user.mycompany.local" $flags --outform pem > $user.cert

done;

Next convert Win7 cert to P12 format:

# Usage: ./convert-to-p12.sh Username

user=$1

openssl pkcs12 -export -inkey $user.key -in $user.cert -name "$user" -certfile caCert.der -caname "vpnserver7" -out $user.p12

then run ./convert-to-p12.sh win7

Next convert Server key to DER format so that Strongswan can take it:

openssl rsa -in $1.key  -out $1.der -outform DER

where $1 is "server"

[Solved] apt-get “is to be installed” errors in Debian

Fortunately, you can force the version in apt-get:

sudo apt-get install deluge-common=1.3.0-0ubuntu1

Source: http://www.absolutelytech.com/2010/10/17/solved-apt-get-is-to-be-installed-errors-in-ubuntu/

use vimdiff for hg diff

put the following to your ~/.hgrc file

[extensions]
hgext.extdiff =

[extdiff]
cmd.vimdiff =

[alias]
vi  = vimdiff
vim = vimdiff

ubuntu update issue

If your ubuntu is failing to update/install software, make sure you have the ubuntu update repository set in your apt source.list file:

###### Ubuntu Main Repos

deb http://us.archive.ubuntu.com/ubuntu/ lucid main restricted universe multiverse 

###### Ubuntu Update Repos

deb http://us.archive.ubuntu.com/ubuntu/ lucid-security main restricted universe multiverse 

deb http://us.archive.ubuntu.com/ubuntu/ lucid-updates main restricted universe multiverse 

FINDDUPE: Duplicate file detector and eliminator

FINDDUPE: Duplicate file detector and eliminator

This tools is cool. Especially the hard-linking feature:

Freeing hard drive space
Sometimes its intentional to have certain media in multiple places. By running finddupe, and hard linking the identical files, you can keep the files in multiple places, while only having one physical copy on the hard drive.

 

IE and IPv6 address

To use IPv6 address in your IE8/IE9 address bar, you have to add a square bracket to the beginning and the end of the IP address. Your IPv6 URL will look something like this:

http://[2001:1:1::02]

Change WireShark Dispaly Time to GMT/UTC or other timezone

On Windows:

1. Open a CMD window

2. set TZ=GMT or TZ=GMT10 or set TZ=GMT-5 (whatever you want your timezone to be)

3. lunch wireshark by: "C:\Program Files\Wireshark\wireshark.exe"

On Linux:

TZ=GMT wireshark

rsync using a different ssh port

# rsync -avz -e "ssh -p $portNumber" user@remoteip:/path/to/files/ /local/path/

Disable Windows Temporary IPv6 Address

In a "cmd" window with administrative rights, run the following commands:

netsh interface ipv6 set privacy state=disabled store=active
netsh interface ipv6 set privacy state=disabled store=persistent
netsh interface ipv6 set global randomizeidentifiers=disabled store=active
netsh interface ipv6 set global randomizeidentifiers=disabled store=persistent

Then Restart your machine.

Shrew Soft VPN client group auth key auth-mutual-psk

If you are provided with a shrew-soft Windows VPN client from your company, and you would like to run a VPN client on Linux. You can either use the Shrew VPN client Linux version or another open source VPN client "vpnc".

To run vpnc, you will need to know the group password. If you export your Shrew VPN profile which is a plain text file, you will see a line in the file that starts with:


   b:auth-mutual-psk:


This is the group password, encoded with BASE64 format. So just copy the value and paste it to any web-based BASE64 decoder such as the one http://www.opinionatedgeek.com/dotnet/tools/Base64Decode/ (This link happens to be on the top when I searched and it works fine), then you get the group password in plain-text.

Running User Mode Linux as normal non-root user

User Mode Linux (UML) is running linux over linux. The guest linux OS runs as a regular process on the host CPU. UML has been around for a while, the famous windows-based "coLinux" is inspired by it.

An unique feature of UML is that you can run it on a powerful Linux server as a regular user. You do not need any sort of root permission to run it. In fact, this is the only feasible solution for a non-root user (other than QEMU running in emulation mode which is much much slower). UML in fast.

To get UML running, you need two things: the kernel and the file system. UML provides the kernel and the compilation is straightforward.

To get file system, you need to have root access on a Linux machine (does not have to be your final HOST machine) and do the following:

(root@host)# apt-get install debootstrap

(root@host)# cd /tmp
(root@host)# dd if=/dev/zero of=debian.bin bs=1M count=1 seek=4096
(root@host)# mkfs.ext3 debian.bin

(root@host)# mkdir -p /mnt; mount -o loop debian.bin /mnt
(root@host)# debootstrap squeeze /mnt

If you are in the year after 2012, Replace "squeeze" with whatever the latest debian stable version name is. 

(root@host)# chroot /mnt 
(chroot@host)# mkdir /dev/ubd
(chroot@host)# cd /dev/ubd
(chroot@host)# for i in 0 1 2 3 4 5 6 7; do mknod $i b 98 $[ $i * 16 ]; done
(chroot@host)# cat > /etc/fstab << EOF
/dev/ubd/0      /        xfs    defaults 0 0
/dev/ubd/1      none     swap   defaults 0 0
none            /proc    proc   defaults 0 0
sys             /sys     sysfs  defaults 0 0
none            /dev/pts devpts defaults 0 0
EOF


(chroot@host)# echo uml0 > /etc/hostname
(chroot@host)# exit
(root@host)# rm -f /mnt/root/.bash_history
(root@host)# umount /mnt



Now the file debian.bin has the latest debian base installation. This is your File System. 

Now you need to download and compile Slirp from http://slirp.sourceforge.net/. Slirp is a cool hack that allows UML limited access to the network by 
tunneling it over regular UDP and TCP connections. This is really the 
key to making UML useful as a regular user. The other (better) methods 
for accessing the network need root access. Be sure to apply the latest 
patch, as otherwise it does not work. It is critical to edit config.h and uncomment the line that says #define FULL_BOLT. This will make Slirp go as fast as it can. Otherwise, slirp will only go 115kbps. Trying to do "apt-get" over that is painful. 

Copy the slirp program you build to /home/YOU/bin/

./linux ubd0=debian6.bin con=pts con0=fd:0,fd:1 eth0=slirp,,/home/user/bin/slirp

I also add "single rw" to get linux boot into single mode and get me started. Once everything is running, I take out "single rw" so that all the daemons are started correctly.

I also have a file ~/.slirprc to direct TCP connections so that I can ssh into the UML. 

#cat ~/.slirprc
redir 2200 22

StrongSwan Configuration Guide

Recently I got a chance to study strongswan and its configurations. This document intends to record the findings, in the hope to help myself in the future and to help others too.

Strongswan is open-source IPSec/VPN software. It was based on FreeSwan, whose development is now stopped. Another descendent of FreeSwan is “OpenSwan”. I have no experience with OpenSwan, and therefore will be focusing on StrongSwan in this document.

By the way, I found the authors of StrongSwan (Andrea and Martini) very much helpful. The know strongswan inside out and was able to explain things really well in many of the mailinglist posts and also in the wiki documentation. This is a great strength of strongswan.

StrongSwan’s core VPN behavior is largely controlled by the configuration file /etc/ipsec.conf. There are many possible lines there you can put in this file. Some lines are extremely important, and a good understanding of what they mean is critical to the successful establishment of the VPN tunnels.

There are a few types of VPN Connections:

1. Host to Host
2. Net to Net
3. Host to Net

Host to Host is fairly rare, and many of the things discussed here also apply to apply to it. So I will focus on “net-net” and “host-net”.

The following settings are tested with StrongSwan 4.5.3 and 4.6.1.

1.      Common Configuration

Common configuration lines in /etc/ipsec.conf

config setup

        strictcrlpolicy=no

        charonstart=yes

        plutostart=no

conn %default

        ikelifetime=60m

        keylife=20m

        rekeymargin=3m

        keyingtries=1

        keyexchange=ikev2

        esp=aes256gcm16,aes128gcm16!

        mobike=yes

        leftikeport=4500

        rightikeport=4500

Explanations:

“strictcrlpolicy” indicates whether CRL is mandatory or not. If CRL is not mandatory, put no. Otherwise, put yes.

“Charon” is the IKEv2 daemon, and “Pluto” is the IKEv1 daemon. In this document, we are only using “IKEv2” and will focus on IKEv2 options only.

“Mobike” stands for Mobile IKE. This is for the case where the public IP of the device may change. If Mobike is enabled, strongswan may float its communication port from UDP port 500 to UDP port 4500 and start telling the Linux kernel to use UDP encapsulation for ESP packets.  It is a good thing to enable if there is a chance that your device’s public IP may change.

“leftikepor” and “rightikeport” tells strongswan to always use UDP port 4500, from the very beginning of IKEv2 message exchange.

2.      Net to Net

2.1   VPN Server

A working net-net VPN SERVER configuration file

Conn myvpn

    left=%defaultroute

    leftcert=/etc/certs/vpn.cert

    leftsubnet=192.168.17.0/24

    leftfirewall=yes

    right=%any

    rightsubnet=0.0.0.0/0

auto=add

Left means “my side”. Right means “my peer’s side”.  You could switch it the other way, but this is how most people use it and is a common convention. Unless one really wants to be different and asking for troubles, it is strongly suggested that this common convention be followed.

“left” is your IP address. This can be set to “%defaultroute” where the system will figure out the value based on the “right” IP address.

“leftcert” is the certificate file of the left. The file path can be an absolute path (starting with /) or a relative path, in which case, the system will look for certs under /etc/ipsec.d/certs/

“leftsubnet”: This is “the” critical line that tells strongswan you want a subnet tunnel instead of a host tunnel. Without this line, the strongswan will try to make a host-only tunnel.  This is the subnet on “your” device’s side. Your peers will only be able to talk to IPs in this subnet.

“leftfirewall”: optional. Tells strongswan to automatically insert firewall rules (iptables rules) when a connection is up or down.

“right” is the peer’s address. For server, this can be “%any”.

“rightsubnet”: similar to “leftsubnet”, this line is critical to indicating that you want to connect to a “subnet”, not just a remote host. Unless “leftsubnet”, you can put “0.0.0.0/0” indicating that you just accept the subnet that the peer defines.

“auto=add” means when you run “ipsec start”, the ipsec daemon just listens, not initiate a connection. “auto=start” means that when you run “ipsec start”, the ipsec daemon will actually try to initiate a call. So “auto=add” is good for servers, and “auto=start” is good for clients. “auto=start” is equivalent to “auto=add” plus “ipsec up MYCONNECTION”

We skipped “leftid=”. leftid by default is derived from the leftcert certificate file, using the Distinguished Name, in the format of “C=XX, O=XXX, CN=XXX, …”. There are any forms of ID that can be used by the leftid/rightid field, but in this document we chose to use this format. See the end of this document for more detailed description of leftid/rightid field.

A working net-net VPN CLIENT configuration file

Net-net is pretty much symmetrical. You can run the above same configuration file on client side and it will work. I chose to one more configuration line:

rightid="C=CH, O=strongSwan, CN=server"

This tells the client to check the server’s certificate ID and make sure it matches this ID. This is just to be safe so that I know I did not connect to some other server. Keep this mind this is after the server certificate is being authenticated by the CA certificate.

3.      Host to Net

3.1   VPN Server

A working host-net VPN SERVER configuration file

conn server

leftcert=server.cert

leftfirewall=yes

right=%any

rightsourceip=192.168.22.0/24

auto=add

Notice that we simply removed “leftsubnet” and “rightsubnet” from the net-net VPN SERVER configuration, and here we have a host-net VPN SERVER.

We added one more line “rightsourceip”. This enables the server to “automatically assign a virtual IP address to the connecting peer”.

Note that in IKEv1 (Pluto) "rightsourceip" can be used to specify the internal side IP address. For IKEv2 (Charon) this is done automatically by the charon daemon and rightsourceip takes up the new meaning of requesting a virtual IP address. If the server does not have "rightsourceip" configured but client has "leftsourceip=x.x.x.x" configured, the tunnel establishment will fail because the server is rejecting the request for a virtual IP address.

3.2  VPN Client

A working host-net VPN CLIENT configuration file

left=%defaultroute

leftcert=client.cert

leftsourceip=%config

leftfirewall=yes

right=192.168.5.1

rightid="C=CH, O=strongSwan, CN=server"

auto=start

The only special line here is “leftsourceip”, which tells the client to obtain a virtual IP address from the VPN Server.

4.       leftid and rightid, what to use?

The ID by which a peer is identifying itself during IKE main mode can by any of the ID types IPV4_ADDR, FQDN, USER_FQDN or DER_ASN1_DN. If one of the first three ID types is used, then the accompanying X.509 certificate of the peer must contain a matching subjectAltName field of the type ipAddress (IP:), dnsName (DNS:) or rfc822Name (email:), respectively. With the fourth type DER_ASN1_DN, the identifier must completely match the subject field of the peer's certificate. One of the two possible representations of a Distinguished Name (DN) is the LDAP-type format

     rightid="C=CH,O=Linux strongSwan, CN=sun.strongswan.org"

Additional whitespace can be added everywhere as desired since it will be automatically eliminated by the X.509 parser. An exception is the single whitespace between individual words , like e.g. in Linux strongSwan, which is preserved by the parser.

The Relative Distinguished Names (RDNs) can alternatively be separated by a slash ( '/')  instead of a comma (',')

rightid="/C=CH/O=Linux strongSwan/CN=sun.strongswan.org"

This is the representation extracted from the certificate by the OpenSSL command line option

openssl x509 -in sunCert.pem -noout –subject

The following X.501 RDNs are supported by strongSwan

DC	Domain Component
C	Country
ST	State or province
L	Locality or town
O	Organisation
OU	Organisational Unit
CN	Common Name
ND	Name Distinguisher, used with CN
N	Name
G	Given name
S	Surname
I	Initials
T	Personal title
E	E-mail
Email	E-mail
emailAddress	E-mail
SN	Serial number
serialNumber	Serial number
D	Description
UID	User ID
ID	X.500 Unique Identifier
TCGID	[Siemens] Trust Center Global ID
unstructuredName	Unstructured Name
UN	Unstructured Name
employeeNumber	Employee Number
EN	Employee Number

5.      Recommended Documentation

1. http://www.strongswan.org/docs/readme.htm I’ve found this readme file very helpful, although it may be a little old. Information presented here still applies.  It is a good starting point, and gives you a good base understanding of everything. This should be the first read, and then you can move on to other documentation such as the wiki.
2. Strong Swan WiKi, which contains “lots of” information. Here is a guide to get you started:
1. IpsecConf http://wiki.strongswan.org/projects/strongswan/wiki/IpsecConf
2. ConnSection http://wiki.strongswan.org/projects/strongswan/wiki/ConnSection
3. ConfigurationExamples: http://wiki.strongswan.org/projects/strongswan/wiki/UserDocumentation#Configuration-Examples

ubuntu kvm KSM high cpu usage

Source: http://www.interphero.com/?p=219

If you use KVM virtualization, under Ubuntu 10.04 Server LTS, with more than one virtual host running at a time, you may have noticed ksmd eating a lot of CPU cycles.  This behavior was not present under 8.04 Server LTS.  Apparently, the kmsd functionality was enabled by default in 10.04.  KSM – Kernel Samepage Merging – merges memory pages between virtual hosts to save space.  Unfortunately, for some, it also utilizes a lot of CPU resources to perform this function.  For my use case, I don’t care about the amount of RAM occupied by my virtual hosts, indeed, I installed far more RAM than I need in my server so that I don’t have to worry about a lack of RAM. 

If you would like to disable ksmd, edit your /etc/default/qemu-kvm file as follows:

# To disable qemu-kvm’s page merging feature, set KSM_ENABLED=0 and
# sudo restart qemu-kvm
KSM_ENABLED=0
#SLEEP_MILLISECS=2000

Then, as the comments in the file state, restart qemu-kvm. (stop your VM, then "sudo stop qemu-kvm", then "sudo start qemu-kvm")

Net-snmp debug token list

http://net-snmp.sourceforge.net/wiki/index.php/Debug_tokens_5.4.2.1

For example: register_mib is an useful token for debugging SNMP AGENTX subagents.

crontab for non-root user

If you could not get your crontab file to work with non-root user, and you have made sure the crond is running and your crontab syntax is correct, check this:

Make sure there is an empty line as last line in your crontab file.

This fixed my problem.

Samba shared drive on Ubuntu/Debian

This samba configuration file configures the samba to be a group shared drive (x: drive). All users access are guest.

File: /etc/samba/smb.conf
----------------------------------


[global]
workgroup = DOCS
netbios name = DOCS_SRV2
security = share
name resolve order = wins lmhosts bcast host
wins server = 10.10.0.10
wins proxy = yes
guest account = nobody


[X]
        comment = Guest Share
        path = /data/x
        browseable = yes
        read only = no
        guest ok = yes
        guest account = nobody
        guest only = yes

cvs server setup in debian / ubuntu

 Setup CVS server on Debian / Ubuntu

1. Install CVS client and server:
apt-get install cvs cvsd

With the default installation, the root for cvsd is at /var/lib/cvsd.The default repositories demo and myrepos are defined in the configuration file /etc/cvsd/cvsd.conf.

By default, cvsd is installed in a chroot environment. To make it use the regular / environment, change this in cvsd.conf:

      RootJail none

Also, you may want to change the Listen statement to the following so that it does not listening on IPv6 sockets.

      Listen 0.0.0.0 2401

2. Start the CVS server:

        sudo /etc/init.d/cvsd start


3. Create myrepos and CVSROOT directories with following command (Note: the absolute path is needed to initialize the repose):

sudo cvs -d /var/lib/cvsd/myrepos init
sudo chown -R cvsd:cvsd /var/lib/cvsd/myrepos

The basic structures of cvsd and the repository $CVSROOT are ready.

4. Add user to the group cvsd and create the password for users.

From System->Administration->Users and Groups. Click Unlock button and enter the password, then click Manage Groups button. Highlight cvsd then click Properties and select the username from the Group Members list.

Create the user password for CVS login as shown below:

cvsd-passwd /var/lib/cvsd/myrepos username

5. Login to cvs server

cvs -d :pserver:username@localhost:/myrepos login

However, if the $CVSROOT is defined just simply enter "cvs login" instead. Add following line in ~/.bashrc is the easiest way to add it to the environment.

export CVSROOT=:pserver:$USER@localhost:/myrepos

If log in has no problem, CVS should be working fine.

Adapted from this post: http://pwong-tipsandtricks.blogspot.com/2009/09/setup-cvs-server-on-ubuntu.html

Windows SSH client with automatic reconnect capability

Putty does not automatic reconnect. There are a few that do:

1. MyEnTunnel  , Your background ssh tunnel software that runs on Windows.
2. Tunnelier  (Free for individual use)
3.  Putty-Tray 

What do you use for this purpose?

compiling / building colinux

I have been using colinux for the last 6 years under Windows XP and then Windows Vista. It has always worked great.  However, I have never tried to build coLinux from scratch. Today, due to the reason that I need a kernel module that is not in the module list provided by the stock colinux build image, I had to go through this exercise.

Overall it is a fairly straightforward process. But I need to make the notes so that later on other people can find it useful.

It is better to have a fast machine, a fast internet connection, and over 1GB of free disk space.

My host machine is Ubuntu 10.04 running in X86-64bit mode. The final image of colinux kernel is 32-bit. The build process automatically does that for you.

1. Download the latest snapshot release source tar ball from www.colinux.org/snapshots/
2. untar it, cd to devel-coLinux...
3. ./configure
4. make
    This will download gcc,bin-utils,mingw32, kernel source, etc
5. You will find you kernel (vmlinux) and modules tar.gz file in the build directory,which should be displayed on the console when the build started.


--To rebuild the kernel with your modified config file--
1. go to the kernel build directory, where your vmlinux file is located.
2. backup your current config file:  cp .config good_config
3. change the config file: make ARCH=i386 menuconfig
4. go to your devel-coLinux-xxx directory, do "make kernel"

linux dynamic library executable and initialization

-----------
C file
-------------




/* initialization when .so is loaded */
static int (*libc_flock)(int fd, int operation)=NULL;
int flock(int fd, int operation){
    printf("my flock called.  fd=%d operation=%d\n",fd,operation);


    if (!handle){
        handle=dlopen("/lib/libc.so.6",RTLD_LAZY);
    }
    if (!libc_flock){
        libc_flock=dlsym(handle,"flock");
        if (libc_flock==NULL)
            return NULL;
    }
    return libc_flock(fd,operation);
}


__attribute__((constructor)) static void mylib_init(void){
    printf("============>\nmylib is initialized\n================>\n");
}




/* this part of code make your .so executable */
const char my_interp[] __attribute__((section(".interp"))) = "/lib/ld-linux.so.2";


void my_main(int argc, char **argv) {
    int i;


    printf("Called as:");
    for (i = 0; i < argc; i++)
        printf(" %s", argv[i]);
    printf("\n");


    exit(0);
}

--------------

Makefile:

----------------

libmylib.so: mylib.c

        $(CC) -shared -Wl,-soname,$@ -fPIC -O2 -s -o $@ $^ -ldl  -Wl,-e,my_main

To read chinese in gvim in Windows

set guifont=NSimSun:h12:cGB2312

C/C++ dbug library

There are many debug libraries available for C/C++.  Recently I started looking into the "dbug" library created by Fred Fish in 1988 and released to public. You can download the library at source forge .

The library has just one C file: dbug.c and one header file: dbug.h (another header file dbug_long.h has more comments can be used in place of dbug.h)

The concept is that you can turn debug on/off on the fly; enable per-function tracing; and other stuff such as profiling which usually you don't need.

To use it:

First, you initialize it by doing this:

  DBUG_PUSH ("d:t:O:L:");

Then in every function, you begin your function code with:

  DBUG_ENTER ("my_function_name");

End your function with either

   DBUG_VOID_RETURN;

or 

   DBUG_RETURN (0);

if you return integer 0.

To use printf to debug something, use this macro:

   DBUG_PRINT ("info", ("Returned value: %d", ret));

where "info" is your keyword/category of debugs. In DBUG_PUSH, you can use "d,keyword1,keyword2..." to turn on/off a list of categories that you want to debug.

Note that there is a file "example.c" in the root directory of the downloaded package, which shows you how to use the library.

Reference for the controlling string in dbug_push:
this is copied from MySQL document Appendix C

The debug control string is a sequence of colon separated fields as follows:
      <field_1>:<field_2>:<field_N>
Each field consists of a mandatory flag character followed by an optional "," and comma separated list of modifiers:
       flag[,modifier,modifier,...,modifier]

The currently recognized flag characters are:

d       Enable output from DBUG_ macros for
        for the current state.  May be followed
        by a list of keywords which selects output
        only for the DBUG macros with that keyword.
        A null list of keywords implies output for
        all macros.
D       Delay after each debugger output line.
        The argument is the number of tenths of seconds
        to delay, subject to machine capabilities.
        I.E.  -#D,20 is delay two seconds.
f       Limit debugging and/or tracing, and profiling to the
        list of named functions.  Note that a null list will
        disable all functions.  The appropriate "d" or "t"
        flags must still be given, this flag only limits their
        actions if they are enabled.
F       Identify the source file name for each
        line of debug or trace output.
i       Identify the process with the pid for each line of
        debug or trace output.
g       Enable profiling.  Create a file called 'dbugmon.out'
        containing information that can be used to profile
        the program.  May be followed by a list of keywords
        that select profiling only for the functions in that
        list.  A null list implies that all functions are
        considered.
L       Identify the source file line number for
        each line of debug or trace output.
n       Print the current function nesting depth for
        each line of debug or trace output.
N       Number each line of dbug output.
o       Redirect the debugger output stream to the
        specified file.  The default output is stderr.
O       As O but the file is really flushed between each
        write. When needed the file is closed and reopened
        between each write.
p       Limit debugger actions to specified processes.
        A process must be identified with the
        DBUG_PROCESS macro and match one in the list
        for debugger actions to occur.
P       Print the current process name for each
        line of debug or trace output.
r       When pushing a new state, do not inherit
        the previous state's function nesting level.
        Useful when the output is to start at the
        left margin.
S       Do function _sanity(_file_,_line_) at each
        debugged function until _sanity() returns
        something that differs from 0.
        (Mostly used with safemalloc)
t       Enable function call/exit trace lines.
        May be followed by a list (containing only
        one modifier) giving a numeric maximum
        trace level, beyond which no output will
        occur for either debugging or tracing
        macros.  The default is a compile time
        option.

Some examples of debug control strings which might appear on a shell command line (the "-#" is typically used to introduce a control string to an application program) are:

                -#d:t
                -#d:f,main,subr1:F:L:t,20
                -#d,input,output,files:n

For convenience, any leading "-#" is stripped off.

vim paste and indent

If you use "dd" to cut a line in vim and want to use "p" to paste it to another place. The original indent will remain. To make the intent fit with the destination indent, use ]p.

List unused C/C++ functions

You have a C/C++ project. It is more than a few files, including some libraries. Now you want to know whether you can get rid of / comment out some of the functions that are never used.

There is no well-known FREE and OPEN SOURCE tools that can do this for you as of 2011, but a not-very-well-known tool called "callcatcher" created by Caolan does the job beautifully.

First, to download the callcatcher, go to the official website: http://www.skynet.ie/~caolan/Packages/callcatcher.html

It is written in Python. So you may want to install it by becoming a root or use sudo.

To use the tool, you need to change ALL your make files to prepend "callcatcher" to your "gcc/g++" command. For example, in your original Makefile, you have

CC=gcc
AR=ar

Now change it to

CC=callcatcher gcc
AR=callarchive ar

When you are done compiling, to see which functions is not used, do:


callanalyse MY-EXE-FILE

Another caveat is that you cannot compile multiple times at the same time, like this


$(CC) -o main main.c lib1.c lib2.c  (This is bad for callcatcher)

Instead you need to change it to something like this, so that you only compile one file at a time, then link it:

%.o:%.c

   $(CC) $(CFLAGS) -c $^

main: $(OBJS)

    $(CC) -o $@ $(OBJS)

Setup your own (small / tiny) Certificate Authority (CA)

For more than 100 certificates, you could use: NewPKI, OpenCA, IDX PKI which
are opensource projects.

A smaller solution is TinyCA (http://tinyca.sm-zone.net/). Note that this is just a GUI front-end to openssl.

From Wikipedia:

EJBCA (java-based)
OpenCA (the most popular one)
OpenSSL, which is really an SSL/TLS library, but comes with tools allowing its use as a simple certificate authority.
gnoMint
DogTag (This looks really good)
XCA

simple telnet server by using socat

On server side:

./socat exec:'bash -li',pty,stderr,setsid  tcp-listen:8999,reuseaddr

On client side:

./socat tcp-connect:127.0.0.1:8999 file:`tty`,raw,echo=0

C, C++ source code tools to help you be productive

1. Source code beautifier: helps you with coding style compliance
  indent
  uncrustify

2. Static anylysis
  pc-lint/flexelint (needs $$)
  splint
  cpplint.py (google code)

3. bug finder
  cppcheck
  flawfinder
  uno

gitignore diff_exclude

A list of files usually should be ignored by git and diff when you generate a patch:


*.o
*.out
*.a
*.so
*.dll
*.swp
*.la
*.lo
*.swp
.gitignore
.depend
tags
*~
*.d
*.1
*.Po
*.Plo
*.log
*.status
*.dd

CVS

git cheat sheet

http://cheat.errtheblog.com/s/git

Makefile Automatic Variables

Here is a table of automatic variables:

$@

The file name of the target of the rule. If the target is an archive member, then ‘$@’ is the name of the archive file. In a pattern rule that has multiple targets (see Introduction to Pattern Rules), ‘$@’ is the name of whichever target caused the rule's recipe to be run.

$%

The target member name, when the target is an archive member. See Archives. For example, if the target is foo.a(bar.o) then ‘$%’ is bar.o and ‘$@’ is foo.a. ‘$%’ is empty when the target is not an archive member.

$<

The name of the first prerequisite. If the target got its recipe from an implicit rule, this will be the first prerequisite added by the implicit rule (see Implicit Rules).

$?

The names of all the prerequisites that are newer than the target, with spaces between them. For prerequisites which are archive members, only the named member is used (see Archives).

$^

The names of all the prerequisites, with spaces between them. For prerequisites which are archive members, only the named member is used (see Archives). A target has only one prerequisite on each other file it depends on, no matter how many times each file is listed as a prerequisite. So if you list a prerequisite more than once for a target, the value of $^ contains just one copy of the name. This list does not contain any of the order-only prerequisites; for those see the ‘$|’ variable, below.

$+

This is like ‘$^’, but prerequisites listed more than once are duplicated in the order they were listed in the makefile. This is primarily useful for use in linking commands where it is meaningful to repeat library file names in a particular order.

$|

The names of all the order-only prerequisites, with spaces between them.

$*

The stem with which an implicit rule matches (see How Patterns Match). If the target is dir/a.foo.b and the target pattern is a.%.b then the stem is dir/foo. The stem is useful for constructing names of related files. In a static pattern rule, the stem is part of the file name that matched the ‘%’ in the target pattern. In an explicit rule, there is no stem; so ‘$*’ cannot be determined in that way. Instead, if the target name ends with a recognized suffix (see Old-Fashioned Suffix Rules), ‘$*’ is set to the target name minus the suffix. For example, if the target name is ‘foo.c’, then ‘$*’ is set to ‘foo’, since ‘.c’ is a suffix. GNU make does this bizarre thing only for compatibility with other implementations of make. You should generally avoid using ‘$*’ except in implicit rules or static pattern rules. If the target name in an explicit rule does not end with a recognized suffix, ‘$*’ is set to the empty string for that rule.

‘$?’ is useful even in explicit rules when you wish to operate on only the prerequisites that have changed. For example, suppose that an archive named lib is supposed to contain copies of several object files. This rule copies just the changed object files into the archive:

lib: foo.o bar.o lose.o win.o
             ar r lib $?

Of the variables listed above, four have values that are single file names, and three have values that are lists of file names. These seven have variants that get just the file's directory name or just the file name within the directory. The variant variables' names are formed by appending ‘D’ or ‘F’, respectively. These variants are semi-obsolete in GNU make since the functions dir and notdir can be used to get a similar effect (see Functions for File Names). Note, however, that the ‘D’ variants all omit the trailing slash which always appears in the output of the dir function. Here is a table of the variants:

‘$(@D)’

The directory part of the file name of the target, with the trailing slash removed. If the value of ‘$@’ is dir/foo.o then ‘$(@D)’ is dir. This value is . if ‘$@’ does not contain a slash.

‘$(@F)’

The file-within-directory part of the file name of the target. If the value of ‘$@’ is dir/foo.o then ‘$(@F)’ is foo.o. ‘$(@F)’ is equivalent to ‘$(notdir $@)’.

‘$(*D)’

‘$(*F)’

The directory part and the file-within-directory part of the stem; dir and foo in this example.

‘$(%D)’

‘$(%F)’

The directory part and the file-within-directory part of the target archive member name. This makes sense only for archive member targets of the form archive(member) and is useful only when member may contain a directory name. (See Archive Members as Targets.)

‘$(<D)’

‘$(<F)’

The directory part and the file-within-directory part of the first prerequisite.

‘$(^D)’

‘$(^F)’

Lists of the directory parts and the file-within-directory parts of all prerequisites.

‘$(+D)’

‘$(+F)’

Lists of the directory parts and the file-within-directory parts of all prerequisites, including multiple instances of duplicated prerequisites.

‘$(?D)’

‘$(?F)’

Lists of the directory parts and the file-within-directory parts of all prerequisites that are newer than the target.

Note that we use a special stylistic convention when we talk about these automatic variables; we write “the value of ‘$<’”, rather than “the variable <” as we would write for ordinary variables such as objects and CFLAGS. We think this convention looks more natural in this special case. Please do not assume it has a deep significance; ‘$<’ refers to the variable named < just as ‘$(CFLAGS)’ refers to the variable named CFLAGS. You could just as well use ‘$(<)’ in place of ‘$<’.

vim global command

The :global command is your friend - learn it well. It lets you run arbitrary :ex commands on every line that matches a regex. It abbreviates to :g.
To delete all lines that match "George Bush":

:g/George Bush/ d

The command that follows can have its own address/range prefix, which will be relative to the matched line. So to delete the 5th line after George Bush:

:g/George Bush/ .+5 d

To delete the DEBUG log entries:

:g/DEBUG/ .,+10 d

If you knew the stack trace was variable length but always ended at a blank line (or other regex):

:g/DEBUG/ .,/^$/ d

You can also execute a command on every line that does NOT match with :g!. e.g. to replace "Bush" with "Obama" on every line that does not contain the word "sucks":

 :g!/sucks/ s/Bush/Obama/

The default command is to print the line to the message window. e.g. to list every line marked TODO:

 :g/TODO

This is also useful for checking the regex matches the lines you expect before you do something destructive.
You can chain multiple commands using "|". e.g. to change Bush to Obama AND George to Barack on every line that does not contain "sucks":

 :g!/sucks/ s/Bush/Obama/g | s/George/Barack/g

vim global command

The :global command is your friend - learn it well. It lets you run arbitrary :ex commands on every line that matches a regex. It abbreviates to :g.
To delete all lines that match "George Bush":

:g/George Bush/ d

The command that follows can have its own address/range prefix, which will be relative to the matched line. So to delete the 5th line after George Bush:

:g/George Bush/ .+5 d

To delete the DEBUG log entries:

:g/DEBUG/ .,+10 d

If you knew the stack trace was variable length but always ended at a blank line (or other regex):

:g/DEBUG/ .,/^$/ d

You can also execute a command on every line that does NOT match with :g!. e.g. to replace "Bush" with "Obama" on every line that does not contain the word "sucks":

 :g!/sucks/ s/Bush/Obama/

The default command is to print the line to the message window. e.g. to list every line marked TODO:

 :g/TODO

This is also useful for checking the regex matches the lines you expect before you do something destructive.
You can chain multiple commands using "|". e.g. to change Bush to Obama AND George to Barack on every line that does not contain "sucks":

 :g!/sucks/ s/Bush/Obama/g | s/George/Barack/g

gcc default include path

this is how to find out the default include path:

method 1:

`gcc -print-prog-name=cc1` -v

method 2:

You can create a file that attempts to include a bogus system header.
If you run gcc in verbose mode on such a source, it will list all the system include locations as it looks for the bogus header.
$ echo "#include <bogus.h> int main(){}" > t.c; gcc -v t.c; rm t.c
[..]
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/lib/gcc/i686-apple-darwin9/4.0.1/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
[..]
t.c:1:32: error: bogus.h: No such file or directory

FREE PHP IDE editors

1. aptana Studio
2. codeLobster
3. Komodo edit

Ubnt airos upgrade firmware

http://www.ubnt.com/wiki/Firmware_Recovery


hold reset pin while power up the device, then


tftp -i 192.168.1.20 put XS2.ar2316.v3.4-rc.4351.090504.2146.bin

virtualbox vboxheadless high cpu usage

If you are running RHEL / CentOS in virtualbox, and find that virtualbox uses high cpu even if the virtual machine is idling, it may be caused by the HZ=1000 timer interrupt issue in your linux kernel. The solution:

add divider=10 to the CentOS kernel parms in /boot/grub/grub.conf

Widows XP Remote Desktop IPv6

Windows XP Remote Desktop service does not listen on IPv6 addresses. Use the follow trick to fix it:

This command will do the trick.
netsh interface portproxy add v6tov4 listenport=3389 connectaddress=127.0.0.1 connectport=3389

After using it you'll be able to connect using a recent version of Remote Desktop Client over IPv6 to a WinXP/Win2k3 box.