Journey of Life

IE and IPv6 address

2012-02-01T09:41:00.000-08:00

To use IPv6 address in your IE8/IE9 address bar, you have to add a square bracket to the beginning and the end of the IP address. Your IPv6 URL will look something like this:

http://[2001:1:1::02]

Change WireShark Dispaly Time to GMT/UTC or other timezone

2012-01-24T16:24:00.000-08:00

On Windows:

1. Open a CMD window

2. set TZ=GMT or TZ=GMT10 or set TZ=GMT-5 (whatever you want your timezone to be)

3. lunch wireshark by: "C:\Program Files\Wireshark\wireshark.exe"

On Linux:

TZ=GMT wireshark

rsync using a different ssh port

2012-01-13T10:21:00.001-08:00

# rsync -avz -e "ssh -p $portNumber" user@remoteip:/path/to/files/ /local/path/

Disable Windows Temporary IPv6 Address

2011-12-30T15:50:00.001-08:00

In a "cmd" window with administrative rights, run the following commands:

netsh interface ipv6 set privacy state=disabled store=active
netsh interface ipv6 set privacy state=disabled store=persistent
netsh interface ipv6 set global randomizeidentifiers=disabled store=active
netsh interface ipv6 set global randomizeidentifiers=disabled store=persistent

Then Restart your machine.

Shrew Soft VPN client group auth key auth-mutual-psk

2011-12-29T09:16:00.000-08:00

If you are provided with a shrew-soft Windows VPN client from your company, and you would like to run a VPN client on Linux. You can either use the Shrew VPN client Linux version or another open source VPN client "vpnc".

To run vpnc, you will need to know the group password. If you export your Shrew VPN profile which is a plain text file, you will see a line in the file that starts with:

b:auth-mutual-psk:

This is the group password, encoded with BASE64 format. So just copy the value and paste it to any web-based BASE64 decoder such as the one http://www.opinionatedgeek.com/dotnet/tools/Base64Decode/ (This link happens to be on the top when I searched and it works fine), then you get the group password in plain-text.

Running User Mode Linux as normal non-root user

2011-12-28T16:44:00.000-08:00

User Mode Linux (UML) is running linux over linux. The guest linux OS runs as a regular process on the host CPU. UML has been around for a while, the famous windows-based "coLinux" is inspired by it.

An unique feature of UML is that you can run it on a powerful Linux server as a regular user. You do not need any sort of root permission to run it. In fact, this is the only feasible solution for a non-root user (other than QEMU running in emulation mode which is much much slower). UML in fast.

To get UML running, you need two things: the kernel and the file system. UML provides the kernel and the compilation is straightforward.

To get file system, you need to have root access on a Linux machine (does not have to be your final HOST machine) and do the following:

(root@host)# apt-get install debootstrap

(root@host)# cd /tmp
(root@host)# dd if=/dev/zero of=debian.bin bs=1M count=1 seek=4096
(root@host)# mkfs.ext3 debian.bin
(root@host)# mkdir -p /mnt; mount -o loop debian.bin /mnt
(root@host)# debootstrap squeeze /mnt

If you are in the year after 2012, Replace "squeeze" with whatever the latest debian stable version name is. 

(root@host)# chroot /mnt 
(chroot@host)# mkdir /dev/ubd
(chroot@host)# cd /dev/ubd
(chroot@host)# for i in 0 1 2 3 4 5 6 7; do mknod $i b 98 $[ $i * 16 ]; done
(chroot@host)# cat > /etc/fstab << EOF
/dev/ubd/0      /        xfs    defaults 0 0
/dev/ubd/1      none     swap   defaults 0 0
none            /proc    proc   defaults 0 0
sys             /sys     sysfs  defaults 0 0
none            /dev/pts devpts defaults 0 0
EOF


(chroot@host)# echo uml0 > /etc/hostname
(chroot@host)# exit
(root@host)# rm -f /mnt/root/.bash_history
(root@host)# umount /mnt

Now the file debian.bin has the latest debian base installation. This is your File System. 

Now you need to download and compile Slirp from http://slirp.sourceforge.net/. Slirp is a cool hack that allows UML limited access to the network by 
tunneling it over regular UDP and TCP connections. This is really the key to making UML useful as a regular user. The other (better) methods for accessing the network need root access. Be sure to apply the latest patch, as otherwise it does not work. It is critical to edit config.h and uncomment the line that says #define FULL_BOLT. This will make Slirp go as fast as it can. Otherwise, slirp will only go 115kbps. Trying to do "apt-get" over that is painful. 

Copy the slirp program you build to /home/YOU/bin/

./linux ubd0=debian6.bin con=pts con0=fd:0,fd:1 eth0=slirp,,/home/user/bin/slirp

I also add "single rw" to get linux boot into single mode and get me started. Once everything is running, I take out "single rw" so that all the daemons are started correctly.

I also have a file ~/.slirprc to direct TCP connections so that I can ssh into the UML. 

#cat ~/.slirprc
redir 2200 22

StrongSwan Configuration Guide

2011-12-23T12:08:00.001-08:00

Recently I got a chance to study strongswan and itsconfigurations. This document intends to record the findings, in the hope tohelp myself in the future and to help others too.

Strongswan is open-source IPSec/VPN software. It was basedon FreeSwan, whose development is now stopped. Another descendent of FreeSwanis “OpenSwan”. I have no experience with OpenSwan, and therefore will befocusing on StrongSwan in this document.

By the way, I found the authors of StrongSwan (Andrea andMartini) very much helpful. The know strongswan inside out and was able toexplain things really well in many of the mailinglist posts and also in thewiki documentation. This is a great strength of strongswan.

StrongSwan’s core VPN behavior is largely controlled by theconfiguration file /etc/ipsec.conf. There are many possible lines there you canput in this file. Some lines are extremely important, and a good understandingof what they mean is critical to the successful establishment of the VPNtunnels.

There are a few types of VPN Connections:

Host to Host
Net to Net
Host to Net

Host to Host is fairly rare, and many of the things discussedhere also apply to apply to it. So I will focus on “net-net” and “host-net”.

The following settings are tested with StrongSwan 4.5.3 and4.6.1.

1. Common Configuration

Common configuration lines in /etc/ipsec.conf

config setup

strictcrlpolicy=no

charonstart=yes

plutostart=no

conn %default

ikelifetime=60m

keylife=20m

rekeymargin=3m

keyingtries=1

keyexchange=ikev2

esp=aes256gcm16,aes128gcm16!

mobike=yes

leftikeport=4500

rightikeport=4500

Explanations:

“strictcrlpolicy” indicates whether CRL is mandatory or not.If CRL is not mandatory, put no. Otherwise, put yes.

“Charon” is the IKEv2 daemon, and “Pluto” is the IKEv1daemon. In this document, we are only using “IKEv2” and will focus on IKEv2options only.

“Mobike” stands for Mobile IKE. This is for the case wherethe public IP of the device may change. If Mobike is enabled, strongswan mayfloat its communication port from UDP port 500 to UDP port 4500 and starttelling the Linux kernel to use UDP encapsulation for ESP packets. It is a good thing to enable if there is achance that your device’s public IP may change.

“leftikepor” and “rightikeport” tells strongswan to alwaysuse UDP port 4500, from the very beginning of IKEv2 message exchange.

2. Net to Net

2.1 VPN Server

A working net-net VPN SERVER configuration file

Connmyvpn

left=%defaultroute

leftcert=/etc/certs/vpn.cert

leftsubnet=192.168.17.0/24

leftfirewall=yes

right=%any

rightsubnet=0.0.0.0/0

auto=add

Left means “my side”. Right means “my peer’s side”. You could switch it the other way, but thisis how most people use it and is a common convention. Unless one really wantsto be different and asking for troubles, it is strongly suggested that thiscommon convention be followed.

“left” is your IP address. This can be set to“%defaultroute” where the system will figure out the value based on the “right”IP address.

“leftcert” is the certificate file of the left. The filepath can be an absolute path (starting with /) or a relative path, in whichcase, the system will look for certs under /etc/ipsec.d/certs/

“leftsubnet”: This is “the” critical line that tellsstrongswan you want a subnet tunnel instead of a host tunnel. Without thisline, the strongswan will try to make a host-only tunnel. This is the subnet on “your” device’s side.Your peers will only be able to talk to IPs in this subnet.

“leftfirewall”: optional. Tells strongswan to automaticallyinsert firewall rules (iptables rules) when a connection is up or down.

“right” is the peer’s address. For server, this can be“%any”.

“rightsubnet”: similar to “leftsubnet”, this line iscritical to indicating that you want to connect to a “subnet”, not just aremote host. Unless “leftsubnet”, you can put “0.0.0.0/0” indicating that youjust accept the subnet that the peer defines.

“auto=add” means when you run “ipsec start”, the ipsecdaemon just listens, not initiate a connection. “auto=start” means that whenyou run “ipsec start”, the ipsec daemon will actually try to initiate a call.So “auto=add” is good for servers, and “auto=start” is good for clients.“auto=start” is equivalent to “auto=add” plus “ipsec up MYCONNECTION”

We skipped “leftid=”. leftid by default is derived from theleftcert certificate file, using the Distinguished Name, in the format of“C=XX, O=XXX, CN=XXX, …”. There are any forms of ID that can be used by theleftid/rightid field, but in this document we chose to use this format. See theend of this document for more detailed description of leftid/rightid field.

A working net-net VPN CLIENT configuration file

Net-net is pretty much symmetrical. You can run the abovesame configuration file on client side and it will work. I chose to one moreconfiguration line:

rightid="C=CH, O=strongSwan, CN=server"

This tells the client to check the server’s certificate IDand make sure it matches this ID. This is just to be safe so that I know I didnot connect to some other server. Keep this mind this is after the servercertificate is being authenticated by the CA certificate.

3. Host to Net

3.1 VPN Server

A working host-net VPN SERVER configuration file

conn server

leftcert=server.cert

leftfirewall=yes

right=%any

rightsourceip=192.168.22.0/24

auto=add

Notice that we simply removed “leftsubnet” and “rightsubnet”from the net-net VPN SERVER configuration, and here we have a host-net VPNSERVER.

We added one more line “rightsourceip”. This enables theserver to “automatically assign a virtual IP address to the connecting peer”.

Note that in IKEv1 (Pluto) "rightsourceip" can be used to specify the internal side IP address. For IKEv2 (Charon) this is done automatically by the charon daemon and rightsourceip takes up the new meaning of requesting a virtual IP address. If the server does not have "rightsourceip" configured but client has "leftsourceip=x.x.x.x" configured, the tunnel establishment will fail because the server is rejecting the request for a virtual IP address.

3.2 VPN Client

A working host-net VPN CLIENT configuration file

left=%defaultroute

leftcert=client.cert

leftsourceip=%config

leftfirewall=yes

right=192.168.5.1

rightid="C=CH, O=strongSwan, CN=server"

auto=start

The only special line here is “leftsourceip”, which tellsthe client to obtain a virtual IP address from the VPN Server.

4. leftid and rightid, what to use?

The ID by which a peer is identifying itself during IKE mainmode can by any of the ID types IPV4_ADDR, FQDN, USER_FQDNor DER_ASN1_DN. If one of the first three ID types is used, then theaccompanying X.509 certificate of the peer must contain a matching subjectAltNamefield of the type ipAddress (IP:), dnsName (DNS:)or rfc822Name (email:), respectively. With the fourth type DER_ASN1_DN,the identifier must completely match the subject field of the peer'scertificate. One of the two possible representations of a DistinguishedName (DN) is the LDAP-type format

rightid="C=CH,O=Linux strongSwan,CN=sun.strongswan.org"

Additional whitespace can be added everywhere as desiredsince it will be automatically eliminated by the X.509 parser. An exception isthe single whitespace between individual words , like e.g. in LinuxstrongSwan, which is preserved by the parser.

The Relative Distinguished Names (RDNs) can alternatively beseparated by a slash ( '/') instead of a comma (',')

rightid="/C=CH/O=LinuxstrongSwan/CN=sun.strongswan.org"

This is the representation extracted from the certificate bythe OpenSSL command line option

openssl x509 -in sunCert.pem -noout –subject

The following X.501 RDNs are supported by strongSwan

DC	Domain Component
C	Country
ST	State or province
L	Locality or town
O	Organisation
OU	Organisational Unit
CN	Common Name
ND	Name Distinguisher, used with CN
N	Name
G	Given name
S	Surname
I	Initials
T	Personal title
E	E-mail
Email	E-mail
emailAddress	E-mail
SN	Serial number
serialNumber	Serial number
D	Description
UID	User ID
ID	X.500 Unique Identifier
TCGID	[Siemens] Trust Center Global ID
unstructuredName	Unstructured Name
UN	Unstructured Name
employeeNumber	Employee Number
EN	Employee Number

5. Recommended Documentation

http://www.strongswan.org/docs/readme.htm I’ve found this readme file very helpful, although it may be a little old. Information presented here still applies. It is a good starting point, and gives you a good base understanding of everything. This should be the first read, and then you can move on to other documentation such as the wiki.
Strong Swan WiKi, which contains “lots of” information. Here is a guide to get you started:

IpsecConf http://wiki.strongswan.org/projects/strongswan/wiki/IpsecConf
ConnSection http://wiki.strongswan.org/projects/strongswan/wiki/ConnSection
ConfigurationExamples: http://wiki.strongswan.org/projects/strongswan/wiki/UserDocumentation#Configuration-Examples

ubuntu kvm KSM high cpu usage

2011-12-15T10:57:00.000-08:00

Source: http://www.interphero.com/?p=219

If you use KVM virtualization, under Ubuntu 10.04 Server LTS, with more than one virtual host running at a time, you may have noticed ksmd eating a lot of CPU cycles. This behavior was not present under 8.04 Server LTS. Apparently, the kmsd functionality was enabled by default in 10.04. KSM – Kernel Samepage Merging – merges memory pages between virtual hosts to save space. Unfortunately, for some, it also utilizes a lot of CPU resources to perform this function. For my use case, I don’t care about the amount of RAM occupied by my virtual hosts, indeed, I installed far more RAM than I need in my server so that I don’t have to worry about a lack of RAM.

If you would like to disable ksmd, edit your /etc/default/qemu-kvm file as follows:

# To disable qemu-kvm’s page merging feature, set KSM_ENABLED=0 and
# sudo restart qemu-kvm
KSM_ENABLED=0
#SLEEP_MILLISECS=2000

Then, as the comments in the file state, restart qemu-kvm. (stop your VM, then "sudo stop qemu-kvm", then "sudo start qemu-kvm")

Net-snmp debug token list

2011-12-13T06:56:00.001-08:00

http://net-snmp.sourceforge.net/wiki/index.php/Debug_tokens_5.4.2.1

For example: register_mib is an useful token for debugging SNMP AGENTX subagents.

crontab for non-root user

2011-12-07T17:11:00.001-08:00

If you could not get your crontab file to work with non-root user, and you have made sure the crond is running and your crontab syntax is correct, check this:

Make sure there is an empty line as last line in your crontab file.

This fixed my problem.

Samba shared drive on Ubuntu/Debian

2011-11-15T17:38:00.001-08:00

This samba configuration file configures the samba to be a group shared drive (x: drive). All users access are guest.

File: /etc/samba/smb.conf
----------------------------------

[global]
workgroup = DOCS
netbios name = DOCS_SRV2
security = share
name resolve order = wins lmhosts bcast host
wins server = 10.10.0.10
wins proxy = yes
guest account = nobody

[X]
comment = Guest Share
path = /data/x
browseable = yes
read only = no
guest ok = yes
guest account = nobody
guest only = yes

cvs server setup in debian / ubuntu

2011-11-09T13:51:00.000-08:00

Setup CVS server on Debian / Ubuntu

1. Install CVS client and server:
apt-get install cvs cvsd

With the default installation, the root for cvsd is at /var/lib/cvsd.The default repositories demo and myrepos are defined in the configuration file /etc/cvsd/cvsd.conf.

By default, cvsd is installed in a chroot environment. To make it use the regular / environment, change this in cvsd.conf:

RootJail none

Also, you may want to change the Listen statement to the following so that it does not listening on IPv6 sockets.

Listen 0.0.0.0 2401

2. Start the CVS server:

sudo /etc/init.d/cvsd start

3. Create myrepos and CVSROOT directories with following command (Note: the absolute path is needed to initialize the repose):

sudo cvs -d /var/lib/cvsd/myrepos init
sudo chown -R cvsd:cvsd /var/lib/cvsd/myrepos

The basic structures of cvsd and the repository $CVSROOT are ready.

4. Add user to the group cvsd and create the password for users.

From System->Administration->Users and Groups. Click Unlock button and enter the password, then click Manage Groups button. Highlight cvsd then click Properties and select the username from the Group Members list.

Create the user password for CVS login as shown below:

cvsd-passwd /var/lib/cvsd/myrepos username

5. Login to cvs server

cvs -d :pserver:username@localhost:/myrepos login

However, if the $CVSROOT is defined just simply enter "cvs login" instead. Add following line in ~/.bashrc is the easiest way to add it to the environment.

export CVSROOT=:pserver:$USER@localhost:/myrepos

If log in has no problem, CVS should be working fine.

Adapted from this post: http://pwong-tipsandtricks.blogspot.com/2009/09/setup-cvs-server-on-ubuntu.html

Windows SSH client with automatic reconnect capability

2011-11-03T11:39:00.000-07:00

Putty does not automatic reconnect. There are a few that do:

1. MyEnTunnel , Your background ssh tunnel software that runs on Windows.
2. Tunnelier (Free for individual use)
3. Putty-Tray

What do you use for this purpose?

compiling / building colinux

2011-10-20T13:35:00.000-07:00

I have been using colinux for the last 6 years under Windows XP and then Windows Vista. It has always worked great. However, I have never tried to build coLinux from scratch. Today, due to the reason that I need a kernel module that is not in the module list provided by the stock colinux build image, I had to go through this exercise.

Overall it is a fairly straightforward process. But I need to make the notes so that later on other people can find it useful.

It is better to have a fast machine, a fast internet connection, and over 1GB of free disk space.

My host machine is Ubuntu 10.04 running in X86-64bit mode. The final image of colinux kernel is 32-bit. The build process automatically does that for you.

1. Download the latest snapshot release source tar ball from www.colinux.org/snapshots/
2. untar it, cd to devel-coLinux...
3. ./configure
4. make
This will download gcc,bin-utils,mingw32, kernel source, etc
5. You will find you kernel (vmlinux) and modules tar.gz file in the build directory,which should be displayed on the console when the build started.

--To rebuild the kernel with your modified config file--
1. go to the kernel build directory, where your vmlinux file is located.
2. backup your current config file: cp .config good_config
3. change the config file: make ARCH=i386 menuconfig
4. go to your devel-coLinux-xxx directory, do "make kernel"

linux dynamic library executable and initialization

2011-10-12T11:03:00.000-07:00

-----------
C file
-------------

/* initialization when .so is loaded */
static int (*libc_flock)(int fd, int operation)=NULL;
int flock(int fd, int operation){
printf("my flock called. fd=%d operation=%d\n",fd,operation);

if (!handle){
handle=dlopen("/lib/libc.so.6",RTLD_LAZY);
}
if (!libc_flock){
libc_flock=dlsym(handle,"flock");
if (libc_flock==NULL)
return NULL;
}
return libc_flock(fd,operation);
}

__attribute__((constructor)) static void mylib_init(void){
printf("============>\nmylib is initialized\n================>\n");
}

/* this part of code make your .so executable */
const char my_interp[] __attribute__((section(".interp"))) = "/lib/ld-linux.so.2";

void my_main(int argc, char **argv) {
int i;

printf("Called as:");
for (i = 0; i < argc; i++)
printf(" %s", argv[i]);
printf("\n");

exit(0);
}

--------------

Makefile:

----------------

libmylib.so: mylib.c

$(CC) -shared -Wl,-soname,$@ -fPIC -O2 -s -o $@ $^ -ldl -Wl,-e,my_main

To read chinese in gvim in Windows

2011-10-07T09:41:00.001-07:00

set guifont=NSimSun:h12:cGB2312

C/C++ dbug library

2011-10-04T15:35:00.000-07:00

There are many debug libraries available for C/C++. Recently I started looking into the "dbug" library created by Fred Fish in 1988 and released to public. You can download the library at source forge .

The library has just one C file: dbug.c and one header file: dbug.h (another header file dbug_long.h has more comments can be used in place of dbug.h)

The concept is that you can turn debug on/off on the fly; enable per-function tracing; and other stuff such as profiling which usually you don't need.

To use it:

First, you initialize it by doing this:

DBUG_PUSH ("d:t:O:L:");

Then in every function, you begin your function code with:

DBUG_ENTER ("my_function_name");

End your function with either

DBUG_VOID_RETURN;

DBUG_RETURN (0);

if you return integer 0.

To use printf to debug something, use this macro:

DBUG_PRINT ("info", ("Returned value: %d", ret));

where "info" is your keyword/category of debugs. In DBUG_PUSH, you can use "d,keyword1,keyword2..." to turn on/off a list of categories that you want to debug.

Note that there is a file "example.c" in the root directory of the downloaded package, which shows you how to use the library.

Reference for the controlling string in dbug_push:
this is copied from MySQL document Appendix C

The debug control string is a sequence of colon separated fields as follows:
<field_1>:<field_2>:<field_N>
Each field consists of a mandatory flag character followed by an optional "," and comma separated list of modifiers:
flag[,modifier,modifier,...,modifier]

The currently recognized flag characters are:


d       Enable output from DBUG_ macros for
        for the current state.  May be followed
        by a list of keywords which selects output
        only for the DBUG macros with that keyword.
        A null list of keywords implies output for
        all macros.

D       Delay after each debugger output line.
        The argument is the number of tenths of seconds
        to delay, subject to machine capabilities.
        I.E.  -#D,20 is delay two seconds.

f       Limit debugging and/or tracing, and profiling to the
        list of named functions.  Note that a null list will
        disable all functions.  The appropriate "d" or "t"
        flags must still be given, this flag only limits their
        actions if they are enabled.

F       Identify the source file name for each
        line of debug or trace output.

i       Identify the process with the pid for each line of
        debug or trace output.

g       Enable profiling.  Create a file called 'dbugmon.out'
        containing information that can be used to profile
        the program.  May be followed by a list of keywords
        that select profiling only for the functions in that
        list.  A null list implies that all functions are
        considered.

L       Identify the source file line number for
        each line of debug or trace output.

n       Print the current function nesting depth for
        each line of debug or trace output.

N       Number each line of dbug output.

o       Redirect the debugger output stream to the
        specified file.  The default output is stderr.

O       As O but the file is really flushed between each
        write. When needed the file is closed and reopened
        between each write.

p       Limit debugger actions to specified processes.
        A process must be identified with the
        DBUG_PROCESS macro and match one in the list
        for debugger actions to occur.

P       Print the current process name for each
        line of debug or trace output.

r       When pushing a new state, do not inherit
        the previous state's function nesting level.
        Useful when the output is to start at the
        left margin.

S       Do function _sanity(_file_,_line_) at each
        debugged function until _sanity() returns
        something that differs from 0.
        (Mostly used with safemalloc)

t       Enable function call/exit trace lines.
        May be followed by a list (containing only
        one modifier) giving a numeric maximum
        trace level, beyond which no output will
        occur for either debugging or tracing
        macros.  The default is a compile time
        option.

Some examples of debug control strings which might appear on a shell command line (the "-#" is typically used to introduce a control string to an application program) are:

                -#d:t
                -#d:f,main,subr1:F:L:t,20
                -#d,input,output,files:n

For convenience, any leading "-#" is stripped off.

vim paste and indent

2011-09-30T17:47:00.001-07:00

If you use "dd" to cut a line in vim and want to use "p" to paste it to another place. The original indent will remain. To make the intent fit with the destination indent, use ]p.

List unused C/C++ functions

2011-09-30T15:06:00.000-07:00

You have a C/C++ project. It is more than a few files, including some libraries. Now you want to know whether you can get rid of / comment out some of the functions that are never used.

There is no well-known FREE and OPEN SOURCE tools that can do this for you as of 2011, but a not-very-well-known tool called "callcatcher" created by Caolan does the job beautifully.

First, to download the callcatcher, go to the official website: http://www.skynet.ie/~caolan/Packages/callcatcher.html

It is written in Python. So you may want to install it by becoming a root or use sudo.

To use the tool, you need to change ALL your make files to prepend "callcatcher" to your "gcc/g++" command. For example, in your original Makefile, you have

CC=gcc
AR=ar

Now change it to

CC=callcatcher gcc
AR=callarchive ar

When you are done compiling, to see which functions is not used, do:

callanalyse MY-EXE-FILE

Another caveat is that you cannot compile multiple times at the same time, like this

$(CC) -o main main.c lib1.c lib2.c (This is bad for callcatcher)

Instead you need to change it to something like this, so that you only compile one file at a time, then link it:

%.o:%.c

$(CC) $(CFLAGS) -c $^

main: $(OBJS)

$(CC) -o $@ $(OBJS)

Setup your own (small / tiny) Certificate Authority (CA)

2011-09-01T15:42:00.000-07:00

For more than 100 certificates, you could use: NewPKI, OpenCA, IDX PKI which
are opensource projects.

A smaller solution is TinyCA (http://tinyca.sm-zone.net/). Note that this is just a GUI front-end to openssl.

From Wikipedia:


EJBCA (java-based)

OpenCA (the most popular one)

OpenSSL, which is really an SSL/TLS library, but comes with tools allowing its use as a simple certificate authority.

gnoMint

DogTag (This looks really good)

XCA

simple telnet server by using socat

2011-08-30T17:16:00.001-07:00

On server side:

./socat exec:'bash -li',pty,stderr,setsid tcp-listen:8999,reuseaddr

On client side:

./socat tcp-connect:127.0.0.1:8999 file:`tty`,raw,echo=0

C, C++ source code tools to help you be productive

2011-08-30T14:44:00.000-07:00

1. Source code beautifier: helps you with coding style compliance
indent
uncrustify

2. Static anylysis
pc-lint/flexelint (needs $$)
splint
cpplint.py (google code)

3. bug finder
cppcheck
flawfinder
uno

gitignore diff_exclude

2011-08-26T11:50:00.001-07:00

A list of files usually should be ignored by git and diff when you generate a patch:

*.o
*.out
*.a
*.so
*.dll
*.swp
*.la
*.lo
*.swp
.gitignore
.depend
tags
*~
*.d
*.1
*.Po
*.Plo
*.log
*.status
*.dd

CVS

git cheat sheet

2011-08-25T16:52:00.001-07:00

http://cheat.errtheblog.com/s/git

Makefile Automatic Variables

2011-08-25T15:59:00.000-07:00

Here is a table of automatic variables:

$@: The file name of the target of the rule. If the target is an archive member, then ‘$@’ is the name of the archive file. In a pattern rule that has multiple targets (see Introduction to Pattern Rules), ‘$@’ is the name of whichever target caused the rule's recipe to be run.
$%: The target member name, when the target is an archive member. See Archives. For example, if the target is foo.a(bar.o) then ‘$%’ is bar.o and ‘$@’ is foo.a. ‘$%’ is empty when the target is not an archive member.
$<: The name of the first prerequisite. If the target got its recipe from an implicit rule, this will be the first prerequisite added by the implicit rule (see Implicit Rules).
$?: The names of all the prerequisites that are newer than the target, with spaces between them. For prerequisites which are archive members, only the named member is used (see Archives).
$^: The names of all the prerequisites, with spaces between them. For prerequisites which are archive members, only the named member is used (see Archives). A target has only one prerequisite on each other file it depends on, no matter how many times each file is listed as a prerequisite. So if you list a prerequisite more than once for a target, the value of $^ contains just one copy of the name. This list does not contain any of the order-only prerequisites; for those see the ‘$|’ variable, below.
$+: This is like ‘$^’, but prerequisites listed more than once are duplicated in the order they were listed in the makefile. This is primarily useful for use in linking commands where it is meaningful to repeat library file names in a particular order.
$|: The names of all the order-only prerequisites, with spaces between them.
$*: The stem with which an implicit rule matches (see How Patterns Match). If the target is dir/a.foo.b and the target pattern is a.%.b then the stem is dir/foo. The stem is useful for constructing names of related files. In a static pattern rule, the stem is part of the file name that matched the ‘%’ in the target pattern. In an explicit rule, there is no stem; so ‘$*’ cannot be determined in that way. Instead, if the target name ends with a recognized suffix (see Old-Fashioned Suffix Rules), ‘$*’ is set to the target name minus the suffix. For example, if the target name is ‘foo.c’, then ‘$*’ is set to ‘foo’, since ‘.c’ is a suffix. GNU make does this bizarre thing only for compatibility with other implementations of make. You should generally avoid using ‘$*’ except in implicit rules or static pattern rules. If the target name in an explicit rule does not end with a recognized suffix, ‘$*’ is set to the empty string for that rule.

‘$?’ is useful even in explicit rules when you wish to operate on only the prerequisites that have changed. For example, suppose that an archive named lib is supposed to contain copies of several object files. This rule copies just the changed object files into the archive:

lib: foo.o bar.o lose.o win.o

             ar r lib $?

Of the variables listed above, four have values that are single file names, and three have values that are lists of file names. These seven have variants that get just the file's directory name or just the file name within the directory. The variant variables' names are formed by appending ‘D’ or ‘F’, respectively. These variants are semi-obsolete in GNU make since the functions dir and notdir can be used to get a similar effect (see Functions for File Names). Note, however, that the ‘D’ variants all omit the trailing slash which always appears in the output of the dir function. Here is a table of the variants:

‘$(@D)’: The directory part of the file name of the target, with the trailing slash removed. If the value of ‘$@’ is dir/foo.o then ‘$(@D)’ is dir. This value is . if ‘$@’ does not contain a slash.
‘$(@F)’: The file-within-directory part of the file name of the target. If the value of ‘$@’ is dir/foo.o then ‘$(@F)’ is foo.o. ‘$(@F)’ is equivalent to ‘$(notdir $@)’.
‘$(*D)’
‘$(*F)’: The directory part and the file-within-directory part of the stem; dir and foo in this example.
‘$(%D)’
‘$(%F)’: The directory part and the file-within-directory part of the target archive member name. This makes sense only for archive member targets of the form archive(member) and is useful only when member may contain a directory name. (See Archive Members as Targets.)
‘$(<D)’
‘$(<F)’: The directory part and the file-within-directory part of the first prerequisite.
‘$(^D)’
‘$(^F)’: Lists of the directory parts and the file-within-directory parts of all prerequisites.
‘$(+D)’
‘$(+F)’: Lists of the directory parts and the file-within-directory parts of all prerequisites, including multiple instances of duplicated prerequisites.
‘$(?D)’
‘$(?F)’: Lists of the directory parts and the file-within-directory parts of all prerequisites that are newer than the target.

Note that we use a special stylistic convention when we talk about these automatic variables; we write “the value of ‘$<’”, rather than “the variable <” as we would write for ordinary variables such as objects and CFLAGS. We think this convention looks more natural in this special case. Please do not assume it has a deep significance; ‘$<’ refers to the variable named < just as ‘$(CFLAGS)’ refers to the variable named CFLAGS. You could just as well use ‘$(<)’ in place of ‘$<’.

vim global command

2011-08-23T16:32:00.003-07:00

The :global command is your friend - learn it well. It lets you run arbitrary :ex commands on every line that matches a regex. It abbreviates to :g.
To delete all lines that match "George Bush":

:g/George Bush/ d

The command that follows can have its own address/range prefix, which will be relative to the matched line. So to delete the 5th line after George Bush:

:g/George Bush/ .+5 d

To delete the DEBUG log entries:

:g/DEBUG/ .,+10 d

If you knew the stack trace was variable length but always ended at a blank line (or other regex):

:g/DEBUG/ .,/^$/ d

You can also execute a command on every line that does NOT match with :g!. e.g. to replace "Bush" with "Obama" on every line that does not contain the word "sucks":

 :g!/sucks/ s/Bush/Obama/

The default command is to print the line to the message window. e.g. to list every line marked TODO:

 :g/TODO

This is also useful for checking the regex matches the lines you expect before you do something destructive.
You can chain multiple commands using "|". e.g. to change Bush to Obama AND George to Barack on every line that does not contain "sucks":

 :g!/sucks/ s/Bush/Obama/g | s/George/Barack/g

vim global command

2011-08-23T16:32:00.001-07:00

The :global command is your friend - learn it well. It lets you run arbitrary :ex commands on every line that matches a regex. It abbreviates to :g.
To delete all lines that match "George Bush":

:g/George Bush/ d

The command that follows can have its own address/range prefix, which will be relative to the matched line. So to delete the 5th line after George Bush:

:g/George Bush/ .+5 d

To delete the DEBUG log entries:

:g/DEBUG/ .,+10 d

If you knew the stack trace was variable length but always ended at a blank line (or other regex):

:g/DEBUG/ .,/^$/ d

You can also execute a command on every line that does NOT match with :g!. e.g. to replace "Bush" with "Obama" on every line that does not contain the word "sucks":

 :g!/sucks/ s/Bush/Obama/

The default command is to print the line to the message window. e.g. to list every line marked TODO:

 :g/TODO

 :g!/sucks/ s/Bush/Obama/g | s/George/Barack/g

gcc default include path

2011-08-23T14:07:00.000-07:00

this is how to find out the default include path:

method 1:

`gcc -print-prog-name=cc1` -v

method 2:

You can create a file that attempts to include a bogus system header.

If you run gcc in verbose mode on such a source, it will list all the system include locations as it looks for the bogus header.

$ echo "#include <bogus.h> int main(){}" > t.c; gcc -v t.c; rm t.c

[..]

#include "..." search starts here:

#include <...> search starts here:

 /usr/local/include

 /usr/lib/gcc/i686-apple-darwin9/4.0.1/include

 /usr/include

 /System/Library/Frameworks (framework directory)

 /Library/Frameworks (framework directory)

End of search list.

[..]
t.c:1:32: error: bogus.h: No such file or directory

FREE PHP IDE editors

2011-08-19T14:16:00.001-07:00

1. aptana Studio
2. codeLobster
3. Komodo edit

Ubnt airos upgrade firmware

2011-08-08T19:53:00.001-07:00

http://www.ubnt.com/wiki/Firmware_Recovery

hold reset pin while power up the device, then

tftp -i 192.168.1.20 put XS2.ar2316.v3.4-rc.4351.090504.2146.bin

virtualbox vboxheadless high cpu usage

2011-08-04T18:11:00.000-07:00

If you are running RHEL / CentOS in virtualbox, and find that virtualbox uses high cpu even if the virtual machine is idling, it may be caused by the HZ=1000 timer interrupt issue in your linux kernel. The solution:

add divider=10 to the CentOS kernel parms in /boot/grub/grub.conf

Widows XP Remote Desktop IPv6

2011-08-02T17:39:00.001-07:00

Windows XP Remote Desktop service does not listen on IPv6 addresses. Use the follow trick to fix it:

This command will do the trick.
netsh interface portproxy add v6tov4 listenport=3389 connectaddress=127.0.0.1 connectport=3389

After using it you'll be able to connect using a recent version of Remote Desktop Client over IPv6 to a WinXP/Win2k3 box.

Simulate unplug usb device on Linux (USB Reset)

2011-07-28T07:31:00.000-07:00

http://cpansearch.perl.org/src/DPAVLIN/Biblio-RFID-0.03/examples/usbreset.c

/* usbreset -- send a USB port reset to a USB device */

/*

http://marc.info/?l=linux-usb-users&m=116827193506484&w=2

and needs mounted usbfs filesystem

 sudo mount -t usbfs none /proc/bus/usb

There is a way to suspend a USB device.  In order to use it, 
you must have a kernel with CONFIG_PM_SYSFS_DEPRECATED turned on.  To 
suspend a device, do (as root):

 echo -n 2 >/sys/bus/usb/devices/.../power/state

where the "..." is the ID for your device.  To unsuspend, do the same 
thing but with a "0" instead of the "2" above.

Note that this mechanism is slated to be removed from the kernel within 
the next year.  Hopefully some other mechanism will take its place.

> To reset a
> device?

Here's a program to do it.  You invoke it as either

 usbreset /proc/bus/usb/BBB/DDD
or
 usbreset /dev/usbB.D

depending on how your system is set up, where BBB and DDD are the bus and
device address numbers.

Alan Stern

*/

#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
#include <sys/ioctl.h>

#include <linux/usbdevice_fs.h>


int main(int argc, char **argv)
{
 const char *filename;
 int fd;
 int rc;

 if (argc != 2) {
  fprintf(stderr, "Usage: usbreset device-filename\n");
  return 1;
 }
 filename = argv[1];

 fd = open(filename, O_WRONLY);
 if (fd < 0) {
  perror("Error opening output file");
  return 1;
 }

 printf("Resetting USB device %s\n", filename);
 rc = ioctl(fd, USBDEVFS_RESET, 0);
 if (rc < 0) {
  perror("Error in ioctl");
  return 1;
 }
 printf("Reset successful\n");

 close(fd);
 return 0;
}

Tether Blackberry 9700 on Linux (Debian/Ubuntu)

2011-07-28T06:32:00.000-07:00

With the fantastic open source software "barry", tethering to Blackberry ( I use 9700 and my carrier is AT&T) is very easy and straightforward. You don't need a Tethering plan, just the regular blackberry data plan.

Warning: be careful with your data usage once you are tethered.

My system is Debian 5.0.3. But this should work on all newer debians and reasonably new Ubuntus. Run all commands as "root":

1. Download and install barry for debian from the sourceforge website. You only need two files: libbarry0 and barry-utils. The files I use are:
barry-util_0.17.1-0_i386.deb libbarry0_0.17.1-0_i386.deb
2. For AT&T users:
cd /etc/ppp/peers
change the file 'barry-att_cingular' to comment out the following lines:
#novj
#noipdefault
3. run "pppd call barry-att_cingular" to get you connected.

Notes about routes:
If your computer already have a default gateway, pppd will not change/update it. In that case, make sure you delete your default gateway before running pppd, or you can manually add the default gw.

To stop the connection, just hit "Ctrl-C". Or, you can do "killall pppd" and wait for a few seconds for it to disconnect.

I use this in Vmware Player virtual machine with Debian 5.0.3 and it works great.

Windows 2003 Server Remote Desktop Local Drive Map

2011-07-20T15:24:00.001-07:00

I could not get the local drive map to work when I remote desktop to a windows 2003 server.

Turned out the the Windows 2003 server disabled the local drive map. So

Click Start, point to All Programs, point to Administrative Tools, and then click Terminal Services Configuration.

In the left pane, click Connections.

In the right pane, right-click RDP-tcp, and then click Properties.

Click around all the tabs, and make sure disk drive map is not disabled.

airos persistent directory

2011-07-17T08:21:00.001-07:00

AirOS allow users to add scripts in the /etc/persistent directory of Ubiquiti device.
These scripts can modify configuration by starting additional services and more.
The standard scripts are:

/etc/persistent/rc.prestart
/etc/persistent/rc.poststart
/etc/persistent/rc.prestop
/etc/persistent/rc.poststop

They are called before and after the standard boot and shutdown scripts start or stop services...

Add static route via script

Reboot devices to clean all static routes and access via SSH/Telnet to AirOS device
Create a script /etc/persistent/rc.poststart and write here commands to add static route
Make sure that /etc/persistent/rc.poststart are executable
Check if your script work fine running the command line: ./etc/persistent/rc.poststart
note: the command now start with a "dot" = run the script!
Run the command: cfgmtd -w -p /etc/ to make persistent
Reboot and verify that all it's working...

openwrt make targets

2011-07-16T08:00:00.000-07:00

Useful_Trunk_Rebuild_Make_Targets

These were culled from IRC logs at irc.freenode.net on #wgt634u . The comments may not be correct. If so, please fix them. This page was put together to help with build problems related to dirty source trees.

make package/base-files-clean target/linux-clean target/lzma-clean target/utils-clean

good general cleaning and doesn't rebuild toolchain
clobbers certain changes you may have made to files in 'root' - (i.e. is it incompatible with WGT634U_USB_root building?)

make target/linux-clean

cleans kernel

make clean

duh, calls 'make dirclean'

make toolchain/uClibc-clean

clean uClibc only

make package/base-files-clean

always run before rebuild?
DANGER - remove package and unpacks the source archive again
aka: make package/package_name-clean (ie: make package/jpeg-clean)

make package/package name-rebuild

recompile the package after you make changes
if this still does a make clean (ie: deletes everything, including your changes) add this to your package/whatever/Makefile:

mostlyclean:

rm $(PKG_BUILD_DIR)/.built

Car Shipping

I can't seem to get the above to look right, so just remember to indent the rm with a tab.

make package/clean

cleans all packages, not always required

make target/clean

for specific packages, e.g. courtesy of frop

...well, if there's a new version of madwifi (from svn)... 'make package/madwifi-clean'

make target/install

rebuilds install image (and kernel too?)

make (any command) world

runs that command, and recompile (same as "make (command); make" (i think))
confirmed - yes, make takes multiple targets on the command line. Be careful when you type commands - if you miss a "-" and type make package/jpeg clean instead of make package/jpeg-clean it'll delete everything as it does the clean after a make package/jpeg. MUCH DIFFERENT!

sqlite 3 string to integer conversion, similar to C function atoi

2011-07-14T11:10:00.000-07:00

In sqlite3, if you have a column defined as string, but you want to use it as integer when doing comparison, you can do the following to convert it.

CAST(expr AS type)

For example: you have a column named "version", and you stored value "11" in it. Now you can convert that to integer by doing:

Select * from mytable where CAST(version as integer)>=11;

Hope this helps you.

Windows 2003, XP, recursive taking file and folders ownership and permissions

2011-07-12T11:22:00.000-07:00

I found that the security tab on the file properties did not work for me when taking ownership and setting permission for a full directories with multiple sub directories. The follow method did the trick:

1. Download fileacl
2. Run: cd C:\Program Files\fileacl
3. fileacl "My directory" /s myusername:RWD /FILES /SUB

Reliable UDP Library

2011-07-01T12:31:00.000-07:00

enet
UDT
http://stackoverflow.com/questions/107668/what-do-you-use-when-you-need-reliable-udp

P2P NAT HOLE PUNCHING UDP

2011-07-01T10:11:00.000-07:00

One of the best articles describing udp hole punching

http://pdos.csail.mit.edu/~baford/nat/draft-ford-natp2p-00.txt

There is another article that goes into details on how to predict port on symmetrical NAT. Here is the article/IETF DRAFT

Internet Draft                                                   B. Ford
Document: draft-ford-natp2p-00.txt                                M.I.T.
Expires: October 2003                                         April 2003


   Network Address Translation and Peer-to-Peer Applications (NATP2P)


Status of this Memo

   This document is an Internet-Draft and is subject to all provisions
   of Section 10 of RFC2026.  Internet-Drafts are working documents of
   the Internet Engineering Task Force (IETF), its areas, and its
   working groups.  Note that other groups may also distribute working
   documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet- Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/1id-abstracts.html

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

   Distribution of this document is unlimited.

Copyright Notice

   Copyright (C) The Internet Society (2003).  All Rights Reserved.

Abstract

   This document describes and recommends methods by which peer-to-peer
   (P2P) applications can operate efficiently in the presence of Network
   Address Translation (NAT).  This document also provides
   recommendations for the design of network address translators, in
   order for them to support P2P applications effectively without
   compromising security or performance.  This memo focuses on the
   interaction of P2P with NAT in the absence of any special proxy,
   gateway, or relaying protocols.  While not intending to preclude the
   use of such protocols, the goal of this memo is to enable P2P
   applications to function automatically without specific knowledge of
   the type, location, or configuration of the NAT.





Ford                                                            [Page 1]
 
draft-ford-natp2p-00.txt                                      April 2003


1. Introduction

   The ubiquitous deployment of Network Address Translation (NAT) on the
   Internet has provided an effective if hopefully temporary means of
   working around the ongoing depletion of the IP address space.  At the
   same time, however, the asymmetric addressing and connectivity
   regimes established by NAT and firewall systems have created unique
   problems for peer-to-peer (P2P) applications and protocols, such as
   teleconferencing and multiplayer on-line gaming.  This document
   discusses these issues and how they can be addressed.  Familiarity is
   assumed with NAT terminology and conventions, as specified in [NAT-
   TRAD] and [NAT-TERM].  As used throughout this document, the term
   "NAT" refers to "Traditional NAT" in both of its standard variants:
   namely Basic NAT, in which only IP addresses are translated, as well
   as Network Address/Port Translation (NAPT), where both IP addresses
   and transport-level port numbers are translated.  In general, this
   document always assumes NAPT as the standard "worst-case" scenario.

   In the traditional client/server paradigm, for which current NAT and
   firewall mechanisms are primarily designed, network communication
   typically operates in terms of sessions that originate in a privately
   addressed network behind a network address translator, and are
   directed to a well-connected public server with a stable IP address
   and DNS mapping.  The client, or originator, of these connections
   often does not have its own routable IP address on the public
   Internet, but instead must share a single public IP address with a
   number of other hosts on the same private network using the NAT as a
   multiplexor.  The lack of a stable, dedicated public IP address is
   not a problem for most client software such as web browsers, because
   the client only needs to be addressable for the duration of a
   particular session.

   In the peer-to-peer paradigm, however, Internet hosts that would
   normally be considered to be "clients" need to establish
   communication sessions directly with each other.  In this case, both
   the initiator and responder in a communication session might lie
   behind a NAT/Firewall, without any permanent IP address or other form
   of public network presence.  A common on-line gaming architecture,
   for example, is for the participating application hosts to contact a
   well-known server for initialization and administration purposes, but
   then to establish direct connections with each other for fast and
   efficient propagation of updates during gameplay.  Similarly, a file
   sharing application might contact a well-known server for resource
   discovery or searching purposes, but establish direct connections
   with peer hosts for data transfer.  NAT creates a problem for peer-
   to-peer connections because unless the NAT is specially configured,
   hosts behind the NAT have no consistent, permanently usable ports to
   which incoming TCP or UDP connections from "the Internet at large"



Ford                                                            [Page 2]
 
draft-ford-natp2p-00.txt                                      April 2003


   can be directed.  Firewall functionality, which is typically but not
   always bundled with NAT, creates a similar problem because firewalls
   are generally designed as one-way filters: sessions initiated inside
   the protected network are allowed, but attempts by external hosts on
   the Internet at large to initiate communication sessions with hosts
   inside the firewall are blocked.  RFC 3235 [NAT-APPL] briefly
   addresses this issue, but does not offer any general solutions that
   do not compromise security; filling that gap is the purpose of this
   document.

2. Techniques for P2P Communication with NAT

   This section describes in detail the currently known techniques for
   implementing peer-to-peer communication in the presence of NAT, from
   the perspective of the application or protocol designer.

2.1. Relaying

   The most reliable, but least efficient, method of implementing peer-
   to-peer communication in the presence of NAT is to make the peer-to-
   peer communication look to the network like client/server
   communication.  For example, suppose two client hosts, A and B, have
   each initiated TCP or UDP connections with a well-known server S
   having a permanent IP address.  Clients A and B both reside on
   privately addressed networks behind network address translators,
   however, and neither of them have control over a public IP address or
   permanently stable TCP or UDP port to which incoming connections can
   be directed.

                                Server S
                                   |
                                   |
            +----------------------+----------------------+
            |                                             |
          NAT A                                         NAT B
            |                                             |
            |                                             |
         Client A                                      Client B

   Instead of attempting to establish a direct TCP or UDP connection
   between clients A and B, the two clients can simply use the server S
   to relay messages between them.  For example, to send a message to
   client B, client A simply sends the message to server S along its
   already-established client/server connection, and server S then sends
   the message on to client B using its existing client/server
   connection with B.  This method has the advantage that it will always
   work as long as both clients have connectivity to the server.  Its
   obvious disadvantages are that it consumes the server's processing



Ford                                                            [Page 3]
 
draft-ford-natp2p-00.txt                                      April 2003


   power and network bandwidth unnecessarily, and communication latency
   between the two clients is likely to be increased even if the server
   is well-connected.

2.2. Connection Reversal

   The second technique works if only one of the clients is behind a
   NAT.  For example, suppose client A is behind a NAT but client B is
   not, as in the following diagram:

                                Server S
                            18.181.0.31:1235
                                   |
                                   |
            +----------------------+----------------------+
            |                                             |
          NAT A                                           |
    155.99.25.11:62000                                    |
            |                                             |
            |                                             |
         Client A                                      Client B
      10.0.0.1:1234                               138.76.29.7:1234

   Client A has private IP address 10.0.0.1, and the application is
   using TCP port 1234.  This client has established a connection with
   server S at public IP address 18.181.0.31 and port 1235.  NAT A has
   assigned TCP port 62000, at its own public IP address 155.99.25.11,
   to serve as the temporary public endpoint address for A's session
   with S: therefore, server S believes that client A is at IP address
   155.99.25.11 using port 62000.  Client B, however, has its own
   permanent IP address, 138.76.29.7, and the peer-to-peer application
   on B is accepting TCP connections at port 1234.

   Now suppose client B would like to initiate a peer-to-peer
   communication session with client A.  B might first attempt to
   contact client A either at the address client A believes itself to
   have, namely 10.0.0.1:1234, or at the address of A as observed by
   server S, namely 155.99.25.11:62000.  In either case, however, the
   connection will fail.  In the first case, traffic directed to IP
   address 10.0.0.1 will simply be dropped by the network because
   10.0.0.1 is not a publicly routable IP address.  In the second case,
   the TCP SYN request from B will arrive at NAT A directed to port
   62000, but NAT A will typically reject the connection request with a
   RST packet because only outgoing connections are allowed.

   After attempting and failing to establish a direct connection to A,
   client B can use server S to relay a request to client A to initiate
   a "reversed" connection to client B.  Client A, upon receiving this



Ford                                                            [Page 4]
 
draft-ford-natp2p-00.txt                                      April 2003


   relayed request through S, opens a TCP connection to client B at B's
   public IP address and port number.  NAT A allows the connection to
   proceed because it is originating inside the firewall, and client B
   can receive the connection because it is not behind a NAT.

   A variety of current peer-to-peer systems implement this technique.
   Its main limitation, of course, is that it only works as long as only
   one of the communicating peers is behind a NAT: if both peers are
   behind NATs, then the method fails.  Because connection reversal is
   not a general solution to the problem, it is NOT recommended as a
   primary strategy.  Applications may choose to attempt connection
   reversal, but should be table to fall back automatically on another
   mechanism such as relaying if neither a "forward" nor a "reversed"
   connection can be established.

2.3. UDP Hole Punching

   The third technique, and the one of primary interest in this memo, is
   sometimes known as "UDP Hole Punching."  UDP hole punching relies on
   well-established NAT conventions to allow appropriately designed
   peer-to-peer applications to "punch holes" through NATs and firewalls
   and establish direct connectivity with each other, even when both
   communicating hosts may lie behind a NAT.  This technique was
   mentioned briefly in section 5.1 of RFC 3027 [NAT-PROT] and has been
   informally described elsewhere on the Internet [KEGEL].  As the name
   implies, unfortunately, this technique works reliably only with UDP.

   We will consider two specific scenarios, and how applications can be
   designed to handle both of them gracefully.  In the first situation,
   representing the common case, two clients desiring direct peer-to-
   peer communication reside behind different NATs.  In the second, the
   two clients actually reside behind the same NAT, but do not
   necessarily know that they do.

2.3.1. Clients Behind Different NATs

   Suppose clients A and B both have private IP addresses and lie behind
   different network address translators.  The peer-to-peer application
   running on clients A and B and on server S each use UDP port 1234.  A
   and B have each initiated UDP communication sessions with server S,
   causing NAT A to assign its own public UDP port 62000 for A's session
   with S, and causing NAT B to assign its port 31000 to B's session
   with S, respectively.








Ford                                                            [Page 5]
 
draft-ford-natp2p-00.txt                                      April 2003


                                Server S
                            18.181.0.31:1234
                                   |
                                   |
            +----------------------+----------------------+
            |                                             |
          NAT A                                         NAT B
    155.99.25.11:62000                            138.76.29.7:31000
            |                                             |
            |                                             |
         Client A                                      Client B
      10.0.0.1:1234                                 10.1.1.3:1234

   Now suppose that client A wants to establish a UDP communication
   session directly with client B.  If A simply starts sending UDP
   requests to B's public address, 138.76.29.7:31000, then NAT B will
   typically discard these incoming messages because the source address
   and port number does not match those of S, with which the original
   outgoing session was established.  Similarly, if B simply starts
   sending UDP requests to A's public address, then NAT A will discard
   these messages.

   Suppose A starts sending UDP requests to B's public address, however,
   and simultaneously relays a request through server S to B, asking B
   to start sending UDP requests to A's public address.  A's outgoing
   messages directed to B's public address (138.76.29.7:31000) will
   cause NAT A to open up a new communication session between A's
   private address and B's public address.  At the same time, B's
   messages to A's public address (155.99.25.11:62000) will cause NAT B
   to open up a new communication session between B's private address
   and A's public address.  Once the new UDP sessions have been opened
   up in each direction, client A and B can communicate with each other
   directly without further reference to or burden on the "introduction"
   server S.

   The UDP hole punching technique has several useful properties.  Once
   a direct peer-to-peer UDP connection has been established between two
   NATted clients, either party on that connection can in turn take over
   the role of "introducer" and help the other party establish peer-to-
   peer connections with additional peers, minimizing the load on the
   initial introduction server S.  The application does not need to
   attempt to detect explicitly what kind of NAT it is behind, if any
   [STUN], since the procedure above will establish peer-to-peer
   communication channels equally well if either or both clients do not
   happen to be behind a NAT.  The hole punching technique even works
   automatically under "Twice NAT", where one or both clients are
   removed from the public Internet via two or more levels of address
   translation.



Ford                                                            [Page 6]
 
draft-ford-natp2p-00.txt                                      April 2003


2.3.2. Clients Behind the Same NAT

   Now consider the scenario in which the two clients (probably
   unknowingly) happen to reside behind the same NAT, and are therefore
   located in the same private IP address space.  Client A has
   established a UDP session with server S, to which the common NAT has
   assigned public port number 62000.  Client B has similarly
   established a session with S, to which the NAT has assigned public
   port number 62001.

                                Server S
                            18.181.0.31:1234
                                   |
                                   |
                                  NAT
                         A-S 155.99.25.11:62000
                         B-S 155.99.25.11:62001
                                   |
            +----------------------+----------------------+
            |                                             |
         Client A                                      Client B
      10.0.0.1:1234                                 10.1.1.3:1234

   Suppose that A and B use the UDP hole punching technique as outlined
   above to establish a communication channel using server S as an
   introducer.  Then A and B will learn each other's public IP addresses
   and port numbers as observed by server S, and start sending each
   other messages at those public addresses.  The two clients will be
   able to communicate with each other this way as long as the NAT
   allows hosts on the internal network to open translated UDP sessions
   with other internal hosts and not just with external hosts.  For
   example, when A sends a UDP packet to B's public address, the packet
   initially has a source IP address and port number of 10.0.0.1:124 and
   a destination of 155.99.25.11:62001.  The NAT receives this packet,
   translates it to have a source of 155.99.25.11:62000 (A's public
   address) and a destination of 10.1.1.3:1234, and then forwards it on
   to B.  Even if supported by the NAT, this translation and forwarding
   step is obviously unnecessary in this situation, and is likely to add
   latency to the dialog between A and B as well as burdening the NAT.

   The solution to this problem is straightforward, however.  When A and
   B initially exchange address information through server S, they
   should include their own IP addresses and port numbers as "observed"
   by themselves, as well as their addresses as observed by S.  The
   clients then simultaneously start sending packets to each other at
   each of the alternative addresses they know about, and use the first
   address that leads to successful communication.  If the two clients
   are behind the same NAT, then the packets directed to their private



Ford                                                            [Page 7]
 
draft-ford-natp2p-00.txt                                      April 2003


   addresses are likely to arrive first, resulting in a direct
   communication channel not involving the NAT.  If the two clients are
   behind different NATs, then the packets directed to their private
   addresses will fail to reach each other at all, but the clients will
   hopefully establish connectivity using their respective public
   addresses.  It is important that these packets be authenticated in
   some way, however, since in the case of different NATs it is entirely
   possible for A's messages to B's private address to reach some other,
   unrelated node on A's private network, or vice versa.

2.3.3. Consistent Port Mappings

   The hole punching technique has one main caveat: it works only if
   each of the NATs maintains a single, consistent mapping from a given
   (private IP address, private UDP port) pair to a translated (public
   IP address, public UDP port) pair, for as long as that UDP port is in
   use.  For a given UDP port on the private network, the NAT must
   ensure that the internal port is always mapped to the same public UDP
   port on the globally addressable Internet, even if communication
   occurs between that internal UDP port and multiple distinct external
   destinations on the Internet.  In particular, the NAT must not just
   naively allocate and assign a new public UDP port for each new
   session initiated from within the firewall boundary, where a
   "session" is defined by the addresses and port numbers of both
   communicating endpoints.  Assigning a new public port for each new
   session makes it impossible for a UDP application to reuse an
   already-established (public IP address, public UDP port) mapping for
   communication with different external destinations.

   While RFC 3022 [NAT-TRAD] suggests and explicitly allows NATs to
   maintain a single mapping from a (private IP address, private port)
   pair to a (public IP address, public port) pair, it unfortunately
   does not mandate this behavior.  Therefore, while many NATs implement
   the desirable behavior and therefore allow direct UDP-based P2P
   connections using the hole punching technique, other NATs currently
   do not support the technique.  Since this is the most efficient known
   method of establishing direct peer-to-peer communication between two
   nodes that are both behind NATs, and it works with a wide variety of
   existing NATs, it is recommended that applications use this technique
   if efficient peer-to-peer communication is required, but be prepared
   to fall back on simple relaying when direct communication cannot be
   established.

2.4. UDP Port Number Prediction

   A variant of the UDP hole punching technique discussed above exists
   that allows peer-to-peer UDP sessions to be created in the presence
   of some NATs that do not maintain a consistent mapping between



Ford                                                            [Page 8]
 
draft-ford-natp2p-00.txt                                      April 2003


   private and public UDP ports.  This method, sometimes called the
   "N+1" technique [BIDIR], works if public port numbers selected by the
   NAT are not held constant across all sessions originating at a given
   private IP address and port, but are nevertheless predictable.
   Consider again the situation in which two clients, A and B, each
   behind a separate NAT, have each established UDP connections with a
   permanently addressable server S:

                                  Server S
                              18.181.0.31:1234
                                     |
                                     |
              +----------------------+----------------------+
              |                                             |
            NAT A                                         NAT B
   A-S 155.99.25.11:62000                        B-S 138.76.29.7:31000
              |                                             |
              |                                             |
           Client A                                      Client B
        10.0.0.1:1234                                 10.1.1.3:1234

   NAT A has assigned its own UDP port 62000 to the communication
   session between A and S, and NAT B has assigned its port 31000 to the
   session between B and S.  By communicating through server S, A and B
   learn each other's public IP addresses and port numbers as observed
   by S.  Client A now starts sending UDP messages to port 31001 at
   address 138.76.29.7 (note the port number increment), and client B
   simultaneously starts sending messages to port 62001 at address
   155.99.25.11.  If NATs A and B assign port numbers to new sessions
   sequentially, and if not much time has passed since the A-S and B-S
   sessions were initiated, then a working bidirectional communication
   channel between A and B should result.  A's messages to B cause NAT A
   to open up a new session, to which NAT A will (hopefully) assign
   public port number 62001, because 62001 is next in sequence after the
   port number 62000 it previously assigned to the session between A and
   S.  Similarly, B's messages to A will cause NAT B to open a new
   session, to which it will (hopefully) assign port number 31001.  If
   both clients have correctly guessed the port numbers each NAT assigns
   to the new sessions, then a bidirectional UDP communication channel
   will have been established as shown below.











Ford                                                            [Page 9]
 
draft-ford-natp2p-00.txt                                      April 2003


                                  Server S
                              18.181.0.31:1234
                                     |
                                     |
              +----------------------+----------------------+
              |                                             |
            NAT A                                         NAT B
   A-S 155.99.25.11:62000                        B-S 138.76.29.7:31000
   A-B 155.99.25.11:62001                        B-A 138.76.29.7:31001
              |                                             |
              |                                             |
           Client A                                      Client B
        10.0.0.1:1234                                 10.1.1.3:1234

   Obviously there are many things that can cause this trick to fail.
   If the predicted port number at either NAT already happens to be in
   use by an unrelated session, then the NAT will skip over that port
   number and the connection attempt will fail.  If either NAT sometimes
   or always chooses port numbers non-sequentially, then the trick will
   fail.  If a different client behind NAT A (or B respectively) opens
   up a new outgoing UDP connection to any external destination after A
   (B) establishes its connection with S but before sending its first
   message to B (A), then the unrelated client will inadvertently
   "steal" the desired port number.  This trick is therefore much less
   likely to work when either NAT involved is under load.  For all of
   these reasons, it is NOT recommended that new applications implement
   this trick; it is described here for purely for historical and
   informational purposes.

2.5. Simultaneous TCP Connection Initiation

   There is a method that can be used in some cases to establish direct
   peer-to-peer TCP connections between a pair of nodes that are both
   behind NATs.  Most TCP sessions start with one endpoint sending a SYN
   packet, to which the other party responds with a SYN-ACK packet.  It
   is possible and legal, however, for two endpoints to start a TCP
   session by simultaneously sending each other SYN packets, to which
   each party subsequently responds with a separate ACK.  This procedure
   is known as a "simultaneous open."

   If a NAT receives a TCP SYN packet from outside the private network
   attempting to initiate an incoming TCP connection, the NAT will
   normally reject the connection attempt by sending back a TCP RST
   (connection reset) packet.  If, however, the SYN packet arrives with
   source and destination addresses and port numbers that correspond to
   a TCP session that the NAT believes is already active, then the NAT
   will allow the packet to pass through.  In particular, if the NAT has
   just recently seen and transmitted an outgoing SYN packet with the



Ford                                                           [Page 10]
 
draft-ford-natp2p-00.txt                                      April 2003


   same addresses and port numbers, then it will consider the session
   active and allow the incoming SYN through.  If clients A and B can
   each correctly predict the public port number that its respective NAT
   will assign the next outgoing TCP connection, and if each client
   initiates an outgoing TCP connection with the other client timed so
   that each client's outgoing SYN passes through its local NAT before
   either SYN reaches the opposite NAT, then a working peer-to-peer TCP
   connection will result.

   Unfortunately, this trick is even more fragile and timing-sensitive
   than the UDP port number prediction trick described above.  First,
   all the same things can go wrong with each side's attempt to predict
   the public port numbers the NATs will assign to the new sessions.  In
   addition, if either client's SYN arrives at the opposite NAT too
   quickly, then the NAT will reject the SYN with a RST packet, causing
   the local NAT in turn to close the new session.  Finally, even though
   support for simultaneous open is technically a mandatory part of the
   TCP specification [TCP], it is not implemented correctly or at all in
   many common operating systems.  For this reason, this trick is
   likewise mentioned here only for historical interest; it is NOT
   recommended for use by applications.  Applications that require
   efficient, direct peer-to-peer communication should use UDP.

3. NAT Design Guidelines

   This section discusses considerations for the design of network
   address translators, as they affect peer-to-peer applications.

3.1. Maintaining Consistent Public/Private Mappings for UDP Ports

   The primary and most important recommendation of this document for
   NAT designers is that the NAT maintain a consistent and stable
   mapping between a given (internal IP address, internal UDP port) pair
   and a corresponding (public IP address, public UDP port) pair for as
   long as any active sessions exist using that mapping.  The NAT may
   filter incoming traffic on a per-session basis, by examining both the
   source and destination IP addresses and port numbers in each packet.
   When a node on the private network starts sending outgoing packets to
   a new external destination, however, while using the same source IP
   address and UDP port as an existing translated UDP session, the NAT
   should ensure that the new UDP session is given the same public IP
   address and UDP port numbers as the existing session.

   One method of implementing this behavior, meant to be only
   illustrative and not prescriptive, is outlined here.  The "critical-
   path" processing performed by a NAT on a packet flowing in either
   direction typically involves extracting the source and destination IP
   addresses from the IP header, and the source and destination TCP/UDP



Ford                                                           [Page 11]
 
draft-ford-natp2p-00.txt                                      April 2003


   port numbers from the TCP/UDP header, and using these four values to
   index into a table of active sessions.  When a packet is received for
   which no entry is found in the session table, the NAT must decide
   whether or not and how to establish a new session, and this is where
   the typical "outgoing sessions only" firewall policy comes into
   effect.

   If the unknown packet was received from the public network, then it
   is usually dropped (or perhaps rejected with a RST in the case of
   TCP).  If a UDP packet for an unknown session arrives from the
   private network, however, and the NAT is configured in the typical
   way to allow outgoing UDP sessions, the then NAT looks up the
   packet's (source IP address, source UDP port) in a second internal
   table: a "mapping table."  For a given (private IP address, private
   UDP port) on the internal network, the NAT stores in the mapping
   table the corresponding (public IP address, public UDP port) it has
   assigned to represent that private endpoint.  If a mapping already
   exists, the NAT simply uses the existing mapping when constructing
   the new session.  If no such entry is present in the mapping table,
   then the NAT allocates a new public UDP port from its pool and
   creates a new mapping table entry along with the new session.  The
   NAT also maintains with each entry in the mapping table a list or
   count of the active sessions using that mapping, so that it can
   reassign the public UDP port to other purposes once all of the
   outstanding sessions for the mapping are deemed inactive.

3.1.1. Preserving Port Numbers

   Some NATs, when establishing a new UDP session, attempt to assign the
   same public port number as the corresponding private port number, if
   that port number happens to be available.  For example, if client A
   at address 10.0.0.1 initiates an outgoing UDP session with a datagram
   from port number 1234, and the NAT's public port number 1234 happens
   to be available, then the NAT uses port number 1234 at the NAT's
   public IP address as the translated endpoint address for the session.
   This behavior might be beneficial to some legacy UDP applications
   that expect to communicate only using specific UDP port numbers, but
   it is not recommended that applications depend on this behavior since
   it is only possible for a NAT to preserve the port number if at most
   one node on the internal network is using that port number.

   In addition, a NAT should NOT try to preserve the port number in a
   new session if doing so would conflict with the goal of maintaining a
   consistent mapping between public and private endpoint addresses.
   For example, suppose client A at internal port 1234 has established a
   session with external server S, and NAT A has assigned public port
   62000 to this session because port number 1234 on the NAT was not
   available at the time.  Now suppose port number 1234 on the NAT



Ford                                                           [Page 12]
 
draft-ford-natp2p-00.txt                                      April 2003


   subsequently becomes available, and while the session between A and S
   is still active, client A initiates a new session from its same
   internal port (1234) to a different external node B.  In this case,
   because an active mapping has already been established between client
   A's port 1234 and the NAT's public port 62000, this mapping should be
   maintained and the new session should also use port 62000 as the
   public port corresponding to client A's port 1234.  The NAT should
   NOT assign public port 1234 to this new session just because port
   1234 has become available: that behavior would not be likely to
   benefit the application in any way since the application has already
   been operating with a translated port number, and it would break any
   attempts the application might make to establish peer-to-peer
   connections using the UDP hole punching technique.

3.2. Maintaining Consistent Public/Private Mappings for TCP Ports

   For consistency with the behavior of UDP translation, it is suggested
   that NATs also maintain a consistent mapping between private and
   public (IP address, TCP port number) pairs for TCP connections, in
   the same way as described above for UDP.  Maintaining consistent
   mappings for TCP ports facilitates the operation of the simultaneous
   TCP open technique, which although not recommended in general for
   peer-to-peer applications, may be useful in controlled situations
   when the two endpoints are sufficiently well synchronized.
   Maintaining TCP endpoint mappings consistently may also increase the
   NAT's compatibility with other applications that initiate multiple
   TCP connections from the same source port.

3.3. Proxy Protocols

   Besides adopting the above recommendations to make a NAT's basic
   "transparent-mode" operation as peer-to-peer friendly as possible, it
   is helpful for NATs also to support proxy protocols that allow
   applications to request an explicitly managed presence on the public
   side of the NAT.  Unfortunately, several alternative protocols have
   been proposed with varying characteristics [SOCKS, RSIP, MIDCOM,
   UPNP], and as of this writing none of them have achieved clear
   acceptance or dominance in the Internet community.  Furthermore, it
   is not clear yet how well these protocols will work in the
   increasingly common "Twice NAT" situation where clients are located
   behind multiple levels of NAT, especially if the NATs are from
   different vendors, support different features and policies, and are
   under different administrative domains.  (In the common case, one is
   owned and managed by the ISP and the other by the end user.)  For
   these reasons, this document makes no attempt to explore this issue
   in detail or to recommend specific proxy protocols for NATs to
   implement.




Ford                                                           [Page 13]
 
draft-ford-natp2p-00.txt                                      April 2003


4. Security Considerations

   Using the UDP hole punching technique in peer-to-peer applications
   and supporting it in NATs should not create any new security issues.
   In particular, the technique does not require a NAT firewall to be
   "promiscuous" in any way about acceping incoming UDP traffic.  As
   long as outgoing UDP sessions are enabled and the firewall maintains
   consistent mappings between internal and external UDP ports, the
   firewall can still filter out all incoming UDP packets except those
   with (source IP, source port, destination IP, destination port)
   tuples that exactly match those of active sessions initiated from
   within the enclave.  Filtering incoming traffic aggressively while
   maintaining consistent mappings thus allows a firewall to be "peer-
   to-peer friendly" without compromising the standard firewall security
   principle of rejecting all unsolicited incoming traffic.

   It might be argued that maintaining a consistent internal/external
   port mapping can "leak" some information to the outside about the
   applications on the internal network, particularly about the
   relationships between different UDP sessions that cross the firewall
   boundary.  If the security requirements are so critical that such a
   subtle information channel is of concern, however, then the firewall
   almost certainly should not be configured to allow unrestricted
   outgoing UDP traffic in the first place.  Controlling information
   flow to this degree generally requires that the firewall only allow
   communication only via tightly-controllable application-level
   gateways, in which case the firewall can either implement the proper
   peer-to-peer communication behavior itself or disallow it entirely.

References

[BIDIR]    Peer-to-Peer Working Group, NAT/Firewall Working Committee,
           "Bidirectional Peer-to-Peer Communication with Interposing
           Firewalls and NATs", August 2001.
           http://www.peer-to-peerwg.org/tech/nat/

[KEGEL]    Dan Kegel, "NAT and Peer-to-Peer Networking", July 1999.
           http://www.alumni.caltech.edu/~dank/peer-nat.html

[MIDCOM]   P. Srisuresh, J. Kuthan, J. Rosenberg, A. Molitor, and
           A. Rayhan, "Middlebox communication architecture and
           framework", RFC 3303, August 2002.

[NAT-APPL] D. Senie, "Network Address Translator (NAT)-Friendly
           Application Design Guidelines", RFC 3235, January 2002.

[NAT-PROT] M. Holdrege and P. Srisuresh, "Protocol Complications
           with the IP Network Address Translator", RFC 3027,



Ford                                                           [Page 14]
 
draft-ford-natp2p-00.txt                                      April 2003


           January 2001.

[NAT-TERM] P. Srisuresh and M. Holdrege, "IP Network Address
           Translator (NAT) Terminology and Considerations", RFC
           2663, August 1999.

[NAT-TRAD] P. Srisuresh and K. Egevang, "Traditional IP Network
           Address Translator (Traditional NAT)", RFC 3022,
           January 2001.

[RSIP]     M. Borella, J. Lo, D. Grabelsky, and G. Montenegro,
           "Realm Specific IP: Framework", RFC 3102, October 2001.

[SOCKS]    M. Leech, M. Ganis, Y. Lee, R. Kuris, D. Koblas, and
           L. Jones, "OCKS Protocol Version 5", RFC 1928, March 1996.

[STUN]     J. Rosenberg, J. Weinberger, C. Huitema, and R. Mahy,
           "STUN - Simple Traversal of User Datagram Protocol (UDP)
           Through Network Address Translators (NATs)", RFC 3489,
           March 2003.

[TCP]      "Transmission Control Protocol", RFC 793, September 1981.

[UPNP]     UPnP Forum, "Internet Gateway Device (IGD) Standardized
           Device Control Protocol V 1.0", November 2001.
           http://www.upnp.org/standardizeddcps/igd.asp

Author's Address

   Bryan Ford
   Laboratory for Computer Science
   Massachusetts Institute of Technology
   77 Massachusetts Ave.
   Cambridge, MA 02139

   Phone: (617) 253-5261
   E-mail: baford@mit.edu
   Web: http://www.brynosaurus.com/

Full Copyright Statement

   Copyright (C) The Internet Society (2003).  All Rights Reserved.

   This document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implementation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph are



Ford                                                           [Page 15]
 
draft-ford-natp2p-00.txt                                      April 2003


   included on all such copies and derivative works.  However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.

   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assigns.

   This document and the information contained herein is provided on an
   "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
   TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
   BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
   HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

































Ford                                                           [Page 16]

Windows CE c# .net compact framework webException "the response did not contain an end of entity mark"

2011-06-29T14:46:00.000-07:00

This problem usually occurs on the Compact Framework of .Net.

Just setting the HTTP ProtocolVersion of the WebRequest to version 1.0 fixed my problem:


request.ProtocolVersion = System.Net.HttpVersion.Version10;

I found this solution at this blog

Windows CE c# .Net An error message cannot be displayed because an optional resource assembly containing it cannot be found.

2011-06-29T14:27:00.000-07:00

Copy this cab file to your CE device, double-click to install it on your CE device,and now you should have the detailed debug messages now.

If anyone is still having this issue the cab file you are looking for, for CF2 is actually in
"C:\Program Files\Microsoft.NET\SDK\CompactFramework\v2.0\WindowsCE\Diagnostics"
I am running WM5 with messaing and security pack and the cab required was System_SR_ENU.CAB and not the System_SR_ENU_wm.CAB version

There is also a stack overflow question with the same answer

popular free http proxy software

2011-06-27T09:28:00.000-07:00

1. Privoxy, block ads
2. polipo, fast light-weight proxy
3. squid, heavy-duty enterprise proxy
4. tinyproxy

All are Free and Open Source software

I use tinyproxy most of the time, and it works great.

How to use curl to upload a file

2011-06-20T18:16:00.001-07:00

curl -F "myfile=@testfile.exe" http://123.45.5.6/test/upload.php

A fairly new comparison of openswan and strongswan

2011-06-17T10:49:00.000-07:00

Openswan vs strongSwan

From the beginning of my VPN project, I knew about strongSwan... but I stuck to Openswan because that’s what is covered in the Openswan book I bought and read.

After perusing the strongSwan website for a few minutes, one thing became apparent: the strongSwan project has superior documentation. The comparison isn’t even close; most of the Openswan documentation hasn’t been updated in years; it often refers to Openswan 3.0 - a branch on which development has stopped for at least 3 years, if its git repository is accurate.

Additionally, when I looked at features, a few trends emerged:

Openswan moved in the direction of the networking industry

- And as a result, supports aggressive mode (which the Openswan devs ask you not to use).
- Openswan supports the legacy KLIPS IPsec kernel stack.

strongSwan is interested in authentication and security:

- No surprise, given its originator provided the x.509 patch.

- - strongSwan has better support for authentication mechanisms in general
- - Supports EAP methods, including EAP-RADIUS
- - PKCS#11 smart cards

strongSwan only supports KLIPS on 2.4 linux kernels; if you’re running 2.6, they use the in-kernel NETKEY IPsec stack.
strongSwan also supports the new IKEv2 standard (and interoperates well with other IKEv2 implementations.

- IKEv2 allows for automatic IP address assignment, DNS assignment, and routing.
- IKEv2 is in its infancy in Openswan.

strongSwan aupports Mobility and Multihomed IKEv2 (also known as MOBIKE)
strongSwan supports additional ciphers, such as TwoFish, and elliptic curve crypto.
strongSwan is modular (vs. Openswan’s monolithic nature)
strongSwan also has IP address pools/assignment with IKEv1, which is not offered by Openswan.

With the data available to me, strongSwan looks like the clear winner. About the only thing I’ve heard about that Openswan does that strongSwan doesn’t are:

KLIPS/MAST on 2.6 kernels

- This allows (with a patched & recompiled kernel) some NAT mapping that doesn’t work well with the NETKEY stack. Cases where NAT clients have the same “internal” IP address as the server, or each other have problems with NETKEY currently.

IKEv1 Aggressive mode: Which is something that even the Openswan developers suggest you avoid if at all possible.

Cross compile wget on Linux for Windows

2011-06-10T11:46:00.000-07:00

GNU wget does not release binary on Windows, and it is not very straight-forward to compile for Windows, but it works great on Windows. I just compiled wget 1.12 for Windows. Please DOWNLOAD IT HERE

I did this on Debian 5.0, and I have the following mingw package installed (not https supported, shouldn't be hard to add if you want to compile the openssl):

dpkg -l | grep -i mingw

ii mingw32 4.2.1.dfsg-1 Minimalist GNU win32 (cross) compiler

ii mingw32-binutils 2.18.50-20080109-1 Minimalist GNU win32 (cross) binutils

ii mingw32-runtime 3.13-1 Minimalist GNU win32 (cross) runtime

This is how to compile wget for Windows on a Linux host using the MingW32 for Linux tool chain

1. download the gnu wget source at http://www.gnu.org/software/wget/

2. tar zxvf wget-1.12.tgz; cd wget-1.12;

3. ./configure CC=i586-mingw32msvc-gcc AR=i586-mingw32msvc-ar RANLIB=i586-mingw32msvc-ranlib --disable-ipv6 --host=mingw32

4. change "src/config.h" to add

#define WINDOWS 1

#define INHIBIT_WRAP 1

and comment out these two lines:

//#define HAVE_SYMLINK 1
//#define HAVE_USLEEP 1
4.1 comment out "src/host.c" line 67
//extern int h_errno;

5. vi src/Makefile, add "mswindows.o" to the end of the "am_wget_OBJECTS" assignment

6. search for "LIBS =", and append "-l ws2_32 -s"

7. Make

The wget.exe file is located in src/wget.exe

Enjoy!

vsftpd config file

2011-06-08T16:46:00.001-07:00

anonymous_enable=YES
listen_port=34567
anon_root=/tmp/ftproot
run_as_launching_user=YES
pasv_promiscuous=YES
write_enable=YES
local_umask=022
anon_upload_enable=YES
dirmessage_enable=YES
xferlog_enable=NO
connect_from_port_20=YES
listen=YES

persistent ssh tunnel

2011-06-07T18:00:00.000-07:00

MyEnTunnel

Work really well. It retries to connect after the ssh session is disconnected.

A good blog about using Windows CE platform builder

2011-06-03T09:58:00.000-07:00

http://geekswithblogs.net/BruceEitman/archive/2008/06/06/platform-builder-summary-of-building-windows-ce.aspx

Platform Builder: Summary of Building Windows CE

Windows CE static IP registry setting

2011-06-02T13:27:00.000-07:00

; Enable static IP address for VMINI1
[HKEY_LOCAL_MACHINE\Comm\VMINI1\Parms\Tcpip]
"EnableDHCP"=dword:0 ; Disable the DHCP/ enable static IP
"IPAddress"="192.168.1.3"
"SubnetMask"="255.255.255.0"

Note : VMINI1 is the correct key path to point to the registry settings used by this driver.
VMINI is the adapter name

source: http://nicolasbesson.blogspot.com/2007/11/static-ip-address-for-kitl-vmini.html

hostapd madwifi in ap mode

2011-05-31T16:23:00.000-07:00

If you are using hostapd, and hostpad is showing the following error:

ioctl[IEEE80211_IOCTL_DELKEY]: Invalid argument
ioctl[IEEE80211_IOCTL_SETMLME]: Invalid argument

Try to add :

wme_enabled=0

in hostapd.conf

This is documented in http://madwifi-project.org/ticket/1561 and it worked for me.

udp port forward

2011-05-25T15:11:00.000-07:00

------
Option 1: user-application based udp port forwarding

Ivan Tikhonov wrote a simple udp_redirect.c program to do just this. I like it.

The link: http://brokestream.com/udp_redirect.html

-------------

Option 2 is to use iptables.

You will need to set the middle computer (Linux) to
1. NAT
2. port forward to the local computer

iptables -t nat -A POSTROUTING -j MASQUERADE -o eth0
iptables -t nat -A PREROUTING -p udp -i eth0 --dport 161 -j DNAT --to 192.168.1.10

Then set the local computer's default gateway to the middle computer:

route add default gw 192.168.1.100

------------
If you are looking for TCP port forwarding, I use and recommend the open source program rinetd. It works on both Windows and Linux, and supports multiple TCP connections, and is very stable. The link: http://www.boutell.com/rinetd/

php internal server error

2011-05-25T12:43:00.001-07:00

If you just see a blank page instead of an  error reporting and you have no server access so you can't edit php  configuration files like php.ini try this:

- create a new file in which you include the faulty script:

<?php
 error_reporting(E_ALL);
 ini_set("display_errors", 1);
 include("file_with_errors.php");
?>

- execute this file instead of the faulty script file 

now errors of your faulty script should be reported.
this works fine with me. hope it solves your problem as well!

Ubuntu Host, Windows 7 Guest, Raw disk, Virtual Box

2011-05-20T14:07:00.000-07:00

Command in Ubuntu to create the win7 virtual machine using the raw disk Windows 7 partition that comes with the Dell computer.

This is tested on Virtual Box 4.0.4
Make sure you run everything as root
Make sure you use "IDE controller" type ICH6, otherwise the win7 will blue screen (BSOD)

This is based on the following blogs:
http://www.rajatarya.com/website/taming-windows-virtualbox-vm
http://www.researchut.com/site/virtualbox-native-partition

First install Virtualbox. I found it easier to install it using the "all platform" exectuable. Make sure you also download and install the Oracle Extension Package, which is installed using this command:
VBoxManage extpack install Extention_file_name

Then do the following commands: make sure you are root, and you are at "/root".

----------------
VBoxManage createvm --name win7 --ostype Windows7_64 --register

VBoxManage modifyvm "win7" --memory 1024 --acpi on --boot1 disk --nic1 nat

VBoxManage storagectl "win7" --name "IDE Controller" --add ide --controller ICH6

VBoxManage internalcommands createrawvmdk -filename /root/win7.vmdk -rawdisk /dev/sda -partitions 1,2,3 -mbr /root/vm.mbr -relative

VBoxManage storageattach win7 --storagectl "IDE Controller" --port 0 --device 0 --type hdd --medium /root/win7.vmdk

------ Other useful commands------------

vboxmanage unregistervm win7 --delete

VBoxManage storageattach win7 --storagectl "IDE Controller" --port 0 --device 1 --type dvddrive --medium "/root/Windows 7 64-bit Repair Disc.iso"

VBoxManage storageattach win7 --storagectl "IDE Controller" --port 0 --device 1 --type dvddrive --medium /opt/VirtualBox/additions/VBoxGuestAdditions.iso

vboxmanage controlvm win7 reset

----If you still cannot boot your Windows 7, try to use the Windows 7 Rescue image to fix it. Use the above "vboxmanage" command to attach the repair disk ISO file and hit F12 when vbox boots to boot from CD, let it fixes it and then boot to regular Win7

boot Windows in Virtualbox

2011-05-19T14:51:00.001-07:00

Use Xmount + Opengates, for MAC users, use openjobs

https://www.pinguin.lu/index.php

Headless Virtualbox on Ubuntu 10.04 LTS

2011-05-18T18:27:00.000-07:00

http://tuxnetworks.blogspot.com/2010/05/howto-virtualbox-31-headless-on-lucid.html

That's it, Virtualbox should be installed and ready to go.
Now we can move on to creating a virtual machine
Create a machine named "io"

VBoxManage createvm -name io --ostype Ubuntu -register

Configure it with a nic bridged to eth0, 256Mb RAM, enable acpi and set to boot from DVD

VBoxManage modifyvm io --memory 256 --pae on --acpi on --boot1 dvd --nic1 bridged --bridgeadapter1 eth0

Create a virtual IDE controller

VBoxManage storagectl io --name IDE0 --add ide

Create a virtual HDD

VBoxManage createvdi -filename ~/.VirtualBox/Machines/io/sda.vdi -size 48000 -register

Attach the virtual HDD

VBoxManage storageattach io --storagectl IDE0 --port 0 --device 0 --type hdd --medium ~/.VirtualBox/Machines/io/sda.vdi

Create and attach a virtual DVD drive to the controller and insert the DVD image

VBoxManage storageattach io --storagectl IDE0 --port 1 --device 0 --type dvddrive --medium /store/archive/ISO/ubuntu-10.04-server-i386.iso

The default vrdp port for machines is 3389, however, if you intend to run more than one guest then each one will need to listen on a different port. I use the 3xxx range with the xxx being the last octet of the machines IP address. For example, 192.168.0.1 would be 3001.

VBoxManage modifyvm io --vrdpport 3001

And thats it, your machine has been created. Time to start it up and give it a test drive!

Using the virtual machine

Start the machine

nohup VBoxHeadless -startvm io &

On a GUI workstation, establish a remote desktop connection to the machine. In my case, the host server is called "jupiter" so I type;

rdesktop -a 8 jupiter:3001

After you have installed the OS, you need to tell the machine to boot from the hdd.

VBoxManage modifyvm io --boot1 disk

You can also deregister the dvd image if you don't intend to use it again.

VBoxManage unregisterimage dvd /store/archive/ISO/ubuntu-10.04-server-i386.iso

Here are some other useful commands;

VBoxManage showvminfo io
VBoxManage list hdds
VBoxManage list runningvms
VBoxManage controlvm io poweroff
VBoxManage unregistervm io --delete
VBoxManage controlvm io savestate
VBoxManage closemedium disk UUID
VBoxManage modifyhd UUID --type immutable

Linux Command Line FTP/HTTP segemented download accelartor software

2011-05-17T13:26:00.000-07:00

aria2c
axel
lftp (use the "pget" command of lftp)

ari2c and axel supports FTP, HTTP and HTTPS, while lftp supports more protocols such as FTPS, SFTP, etc.

/etc/hosts not working?

2011-05-16T16:03:00.000-07:00

If you have a Linux machine and your /etc/hosts is not working, make sure you have a file called

/etc/nsswitch.conf

with the following content:

passwd:         files
group:          files
hosts:          files dns
networks:       files dns
services:       files
protocols:      files
netmasks:       files

Windows CE 5.0 QFE download

2011-05-13T15:25:00.001-07:00

Microsoft has moved on to Windows CE 7 (Compact 7) and revamped their download site so that all the windows CE 5.0 downloads cannot be located anymore.

To patch your platform builder 5.0 to the latest, download the "rollup" files from here:

http://www.microsoft.com/windowsembedded/en-us/downloads/download-windows-embedded-ce-5.aspx

Once you installed any QFE, you will have a application called CEQFECHECK.exe under C:\WINDOWS\system32\ceqfecheck. This application checks which QFE you have installed and did not install. Check it out.

Linux Network Interface statistics

2011-05-05T10:03:00.000-07:00

http://stackoverflow.com/questions/596590/how-can-i-get-the-current-network-interface-throughput-statistics-on-linux-unix

Tools include:

1. bwm-ng
2. iftop
3. iptraf
4. sar
5. dstat

jQuery TableSorter "o is undefined" problem

2011-04-18T18:13:00.001-07:00

If you use jQuery TableSorter and see the "o is undefined" problem, it is the probably the case that you are using "<td>" inside the <thead> section instead of <th>. Use <th> and you will be good.

sqlite pdo write lock problem solved

2011-03-17T18:11:00.000-07:00

When using PHP and PDO to access an sqlite dababase, and when you do a "select *" using the $handle->query($sqlcmd), you must use "foreach" to read all the select results before you can issue the next "query" or "exec" call. Otherwise, the database will be locked.

Win32 applications GUI vs Console

2011-03-04T11:23:00.003-08:00

This web page explains well the difference between a WIN32 GUI app vs. Console app.

php header redirect IE issue

2011-03-03T13:54:00.000-08:00

In web development, a HTTP POST is often processed on the server ending with a header("Location:page.php") call, so that user can refresh their page without re-posting. This usually works well.

The only issue is IE. After posting or file upload, using the PHP header() call won't work with IE, although it works with Firefox. You will have to use the HTML direct instead.

Windows CE ssh server and client

2011-02-09T18:05:00.000-08:00

Windows CE development is not as fun as Linux because of the lack of open source tools. However, once I am able to make ssh server and client running on it, it becomes better. Here is how to get ssh server and client running on windows CE 5 (ARM based):

1. Download CESSH at http://www.codeplex.com/wikipage?projectname=CESSH
Download sshd_config at http://www.codeplex.com/Project/Download/FileDownload.aspx?ProjectName=CESSH&DownloadId=3561
Download file http://www.codeplex.com/Project/Download/FileDownload.aspx?ProjectName=CESSH&DownloadId=3562

2. copy the files to your windows CE device, suppose "\ssh"
3. mkdir "\Hard Disk"
4. copy sshd_config "\Hard Disk"
5. mkdir "\NandFlash"
6. mkdir "\NandFlash\ssh"
7. copy ssh_host_dsa_key to "\NandFlash\ssh"
8. run "adduser.exe root rootpass" to add the user "root" with password "rootpass"
9. copy SocketToFile.DLL to \windows
10. run "sshd.exe"

To test it, get pocketputty
(http://www.pocketputty.net/download.html)

If make this work over ActiveSync, use the remote port tunnel in putty.

Another great tool to have is netcat for CE: http://www.dr-bischoff.de/wince/Andreas_WINCE_stuff.html#netcat

Hope this helps.

Slick board and XML/SWF charts

2011-02-04T07:02:00.000-08:00

SlickBoard and XML/SWF charts seem like very great products. Let me know if you have used it.

And also the google visualization tools which give you access to google finance like charts (http://code.google.com/apis/visualization/documentation/gallery/annotatedtimeline.html)

Free php web stat analyzer

2011-01-28T06:48:00.001-08:00

1. TraceWatch
2. BBClone

Both GPL and Free.

git, cgit and mini_httpd

2011-01-27T17:36:00.000-08:00

git allows very easy local installation. We need a easy way to view the local git repository. mini_httpd is a great local web server to use, and git web app can either be "gitweb" or "cgit".

I tried "gitweb" but couldn't get it to see my projects. But I got "cgit" to work well. Here is how:

1. download and compile cgit. very simple, just follow the README file.
2. change cgit Makefile, "CGIT_CONFIG = /yourpath/cgitrc", then "touch cgit.c" and "make" again. We do this because by default cgitrc is located at /etc/ and we want it local.
3. Create your cgitrc file from the example file. I use something like this:

cache-size=0
css=/cgit.css
enable-index-links=1
enable-log-filecount=1
enable-log-linecount=1
max-stats=quarter
root-title=My git repository
root-desc=Tracking my project development
root-readme=about.html
snapshots=tar.gz
mimetype.git=image/git
mimetype.html=text/html
mimetype.jpg=image/jpeg
mimetype.jpeg=image/jpeg
mimetype.pdf=application/pdf
mimetype.png=image/png
mimetype.svg=image/svg+xml
repo.url=linux
repo.path=/opt/gittest/.git
repo.desc=My project
repo.owner=me@me.com
repo.readme=info/web/about.html
repo.snapshots=tar.gz
repo.enable-log-linecount=0
repo.max-stats=month

4. start minihttpd and your are in business. :-)

Vim non greedy regular expression search

2011-01-26T15:36:00.001-08:00

* (0 or more) greedy matching
\+ (1 or more) greedy matching
\{-} (0 or more) non-greedy matching
\{-n,} (at least n) non-greedy matching

http://blog.vinceliu.com/2008/02/non-greedy-regular-expression-matching.html

CSS Positioning explained

2011-01-26T12:09:00.000-08:00

http://www.barelyfitz.com/screencast/html-training/css/positioning/

Simple truth:

1. Use positon:relative in the outer box and then positoin:absolute in the inner box. This is very effective.

2. When using position:absolute with background image, make sure you explicitly specify "width" and "height", otherwise it won't work.

CSS and box shadow (blurred shadow)

2011-01-26T09:29:00.000-08:00

All major browsers except IE (IE9 may support it) supports the CSS3 box shadow. So you need the javascript to do some tricks for IE. Here is the link. http://www.hintzmann.dk/testcenter/js/jquery/boxshadow/

Use Linux to control power outlet via USB

2011-01-24T16:28:00.000-08:00

Much of information on this page is based the web page at here

1. buy EcoStrip , and Linksys USB2HUB4 USB 2.0 hub. This hub supports power control. Internally it uses NEC chipset.

2. Download hub-ctrl.c, OR download it at this site

3. Use lsusb utilities with -v option, you can inspect 'Hub Descriptor', such like:

Hub Descriptor:
   [...]
     wHubCharacteristic 0x0089
       Per-port power switching
       Per-port overcurrent protection
       Port indicators
   [...]

4.compile hub-ctrl.c with -lusb (you need to install libusb-dev if you don't have it)

5. run "sudo ./hub-ctrl" to list the possible hub that supports power control

Bus 002 Device 003: ID 0409:0058 NEC Corp. HighSpeed Hub
Bus 002 Device 002: ID 8087:0020
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 8087:0020
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

6.From above, you can see that my Linksys hub is located at Bus 2 Device 3. So I use the following command to turn the ecoStrip (which is plugged into the Linksys Hub port 1) power off:

sudo ./hub-ctrl -b 2 -d 3 -P 1

And the following command to turn it on

sudo ./hub-ctrl -b 2 -d 3 -P 1 -p 1

transfer or populate sqlite table to another table

2011-01-21T06:53:00.001-08:00

INSERT INTO TABLE2 (COL1, COL2, COL3) SELECT COL1, COL4, COL7 FROM TABLE1

Use PHP to receive email

2011-01-19T17:25:00.000-08:00

I use "sendmail". Other MTA should be similar.

1. Supposed you want php to receive emails destined to "support@example.com". Go to /etc/aliases and add a line:
support: "|/usr/local/bin/getemail.php"

2. restart sendmail
3. go to /etc/smrsh, and create two symbolic link files:

ln -s /usr/bin/php php
ln -s /usr/local/bin/getemail.php getemail.php

These symbolic links basically tells sendmail you know these programs are safe for it to run

4. create your php script. A simple php script looks like this. Of course you should add your logic to it.

$fd = fopen("php://stdin", "r");
$email = "";
while (!feof($fd)) {
         $email .= fread($fd, 1024);
}
fclose($fd);

file_put_contents("/tmp/email.txt",$email);
die();

See which file is preventing you from mount read-only

2011-01-18T15:12:00.000-08:00

In embedded systems, you can make your entire root file system read only (therefore preventing flash corruption) by doing:

mount -o remount,ro /

But sometimes this fails, because some files are being open as "writable". We need to find a way to identify these processes. lsof comes to rescue.

lsof +D / | awk $4~/w$/{print}

This commands list all open files recursively then uses an one-line awk script to find the lines that have files open for write.

lsof is a really good friend. :-)

PHP PDF import and modification

2011-01-11T07:35:00.000-08:00

There is a php library that allows importing a page from an existing PDF file, and then write data/graph on top of it. This essentially allows one to modify a pdf file.

The library is called FPDI (http://www.setasign.de/products/pdf-php-solutions/fpdi/about/). It uses the basic PDF library FPDF, or TCPDF (which is a fork of FPDF).

Seems useful. haven't tried it yet.

More: tried it yesterday and it works beautifully!! Use GSview32 to find the exact x/y of the place where you want to insert text/graph.

Another gem found.

https, TLS, SSL and multiple hosts

2011-01-10T07:20:00.001-08:00

http://en.wikipedia.org/wiki/Server_Name_Indication

Unfortunately Windows XP with IE does not support this.

best free screencast software

2011-01-05T15:00:00.000-08:00

This is on Windows.

It looks like BB Flashback Express (free) may be the best. Here is why one organization chooses to use it.

ActivePresenter Free Edition looks good too. Any one has experiences on using it?

Any other free screencast software you recommend? I've tried Camstudio, but I am still looking. :-)

Use sqlite3 with codeigniter

2010-12-28T09:49:00.000-08:00

Codeigniter is a small, fast, and well-documented web framework. However, currently (as of version 1.7.3), only sqlite2 is supported. I made codeigniter 1.7.3 work with sqlite3, including the scaffolding feature, which I really like.

Here is how to make codeigniter 1.7.3 work with sqlite3:

1. Download the php-based sqlite3 PDO driver for codeigniter. This driver is based on the driver in the codeigniter wiki, but fixed up so it works with v1.7.3 and with scaffolding.

Download the sqlite3 PDO driver (based on the wiki driver 0.2)

2. Edit system/database/DB_driver.php, around line 831, add the 4 lines below starting with '+'.


foreach($query->result_array() as $row)
{
if (isset($row['COLUMN_NAME']))
{
$retval[] = $row['COLUMN_NAME'];
}
+            else if ($this->platform()=="pdo")
+            {
+                $retval[] = next($row);
+            }
else
{
$retval[] = current($row);
}
}

3. Create directory /pdo in /database/drivers and copy to this directory
driver *.php files
4. Create SQLite3 database file, and put it to any directory.
My database file is [APPPATH]/db/base.db
5. In application database config [APPPATH]/config/database.php set next settings:

  
$db['default']['hostname'] = ''; 
$db['default']['username'] = '';
$db['default']['password'] = '';
$db['default']['database'] = 'sqlite:'.APPPATH.'db/base.db';
$db['default']['dbdriver'] = 'pdo';   

Enjoy.

vimdiff ignoring white space

2010-12-27T12:19:00.001-08:00

I had been searching for a way to ignore white spaces while using vimdiff. Unfortunately, vimdiff -h yields the generic Vim help. I finally found that including the following line in vimrc solves the problem.

set diffopt+=iwhite

From the command line:

vimdiff -c 'set diffopt+=iwhite' ...

codeignitor removes index.php

2010-12-22T11:31:00.001-08:00

Ok, try

 <IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteBase /

    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d

    RewriteRule ^(.*)$ /index.php?/$1 [L] <-- do not forget the ? after index.php!
</IfModule>

<IfModule !mod_rewrite.c>
    # If we don't have mod_rewrite installed, all 404's
    # can be sent to index.php, and everything works as normal.

    ErrorDocument 404 /index.php</IfModule>   

in config.php set

$config['index_page'] = "";$config['uri_protocol'] = "AUTO";

With this htaccess index.php-hiding works for me with PHP4 and PHP5 running PHP as a CGI

Corss Compile tcpdump for linux/mips

2010-12-22T10:25:00.000-08:00

1. download libpcap (I use version 1.1.1 at the time of post) and tcpdump (v4.1.1 as of time of post)

2. unzip both directory under the same directory, such as download/libpcap-1.1.1 and download/tcpdump-4.1.1

Build libpcap:

3. cd libpcap-1.1.1 ;

4. vi configure; search for "linux version", and remove the entire section of "case" under "linux)" until "do we have the wireless extensions". This is so that ./configure does not try to detect linux (and fail). Our linux is fine.

5. CC=/YOUR-CROSS-COMPILER-PATH/mips-openwrt-linux-gcc ./configure --host=mips-linux --with-pcap=linux
6. make. after make is successful, you'll have a libpcap.a

Build tcpdump:

7. cd ../tcpdump-4.1.1

8 .vi configure; earch for "linux version", and remove the entire section of "case" under "linux*)" until ";; \n *)".

9. CC=/YOUR-CROSS-COMPILER-PATH/mips-openwrt-linux-gcc ./configure --host=mips-linux --with-pcap=linux

10. vi Makefile; search for /usr/include and remove them; the Makefile by mistake include including files in the host system. Remove them.

11. make. you can strip the final tcpdump if you like. all done.

How to cross compile tinyproxy for mips

2010-12-17T10:07:00.000-08:00

tinyproxy version: 1.8.2

1. Change the configure file to comment the section containing "Check for asciidoc" until all the way to "ac_config_files=" (not including). Also Remove the lines that test "HAVE_A2X_TRUE".

2. CC=/home/tzhang/filter/trunk/toolchain_bin/mips-openwrt-linux-gcc ./configure --host=mips-linux --enable-filter --disable-upstream --disable-reverse --enable-transparent --disable-regexcheck

3. modify config.h: search for "rpl_", and comment the two lines that contains rpl_malloc and rpl_remalloc. Use /* */ syntax to comment out.

4. vi src/Makefile, search for "LDFLAGS", and append " -s" to it (to strip the final binary)

5. Chanage Makefile to only build src. Find SUBDIRS , comment out all other directories other than "src".

6. Make. find the binary at src/tinyproxy.

The two important articles to read to understand Windows7/Vista Arp behavior

2010-12-17T06:48:00.001-08:00

http://blogs.technet.com/b/networking/archive/2009/03/30/tcp-ip-networking-from-the-wire-up.aspx

http://support.microsoft.com/kb/949589

bfilter

2010-12-16T07:12:00.001-08:00

A web proxy filter that blocks ads, achieves effects like Firfox AdBlock Plus.

http://Bfilter.sourceforge.net

vim global command

2010-12-15T12:15:00.000-08:00

http://www.networkcomputing.com/unixworld/tutorial/009/009.part3.html

syntax: g/stuff-to-search/[range]EX command

example:

global /^/ + delete

:g/blah/d  , this deletes all lines containging 'blah'

:g/blah/ ,+ delete , this deletes all lines and next lines that containing 'blah'

See more at http://vimdoc.sourceforge.net/htmldoc/cmdline.html#Command-line

Git Data Transport Commands

2010-12-14T12:47:00.001-08:00

iptables SO_ORIGINAL_DST

2010-12-11T05:01:00.001-08:00

this option in socket call can let you retrieve the original IP and port number of a redirected TCP session. Brilliant.

Git Get Started

2010-12-09T04:51:00.001-08:00

Getting Started
Let's say you have a development project in the directory devproject. Let's start using Git to manage this project.
First off install Git. In Debian and Ubuntu we just need to do "aptitude install git"

cd devproject
git init
git add .
git commit -m "My first commit"

At this point you have all the benefits of a local version control system but no one can see your work. To make it available to other people we'll need to install a remote repository on your server. At home, I only allow people to access my code through SSH so that's the method I am going to talk about here.

ssh alex
mkdir -p /var/git/devproject.git
cd /var/git/devproject.git
git --bare init
exit

Your remote Git server is now configured so let's set up our local repository to talk to the remote repository

cd devproject
git remote add origin ssh://alex/var/git/devproject.git

We can now push our changes to that repository:

git push origin master

git clone: clone from a remote a new local repository

git fetch: update the local "remote" directory

git pull: = git fetch + git merge

Compile Openssl for Linux MIPS

2010-12-08T06:05:00.000-08:00

1. Grab openssl source 1.0.c
2. Grab the Linux-MIPS patch at http://svn.cross-lfs.org/svn/repos/patches/openssl/openssl-0.9.8k-mips_support-1.patch
3. apply the patch: the patch basically just add a few lines to the Configure file. you can do that manually if you want.
4. CC=YOUR-MIPS-CC ./configure linux-mips (add "shared" if you want to build shared library)
5. make

Increase VNC Speed (tightvnc)

2010-12-07T18:06:00.001-08:00

If you select the "Low-bandwidth connection" option under "Connection profile" a lot of the default options will be changed. You will notice a drastic increase in performance from that allowance.
If you want to tweak the connection further, click the "Options" button to see what is under the hood. Choose the "Tight" encoding option to use TightVNC's compression. Select "Use 8-bit color" to reduce the number of colors and therefore increase speed. Below that, slide the compression bar down to its fastest point to make major changes on performance. Keep in mind that this will also affect the image quality.

rSync for Windows

2010-12-06T12:37:00.000-08:00

1. Download the zipfile for DelteCopy (Without Installer)
2. Unzip it. It includes rsync.exe for Windows
3. copy deltecd.conf to rsyncd.conf
4. run it as a daemon: rsync --daemon --no-detach --config=\PATH\rsyncd.conf
5. if that works, it will open a port on 873 (make sure you firewall is off).
6. to make it permanent, you can use "srvany" to install it as a service.

More detailed instructions can be found at HERE.

Edit Motion Jpeg on Windows

2010-12-06T10:41:00.001-08:00

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Background:
Digital Still Cameras for the last couple of years have been able to record video. The cameras put the video (and usually PCM aka RAW audio) in either Quicktime (MOV) or AVI containers.

The video codec is usually Motion JPEG (FourCC:MJPG) because the chip in the camera usually makes JPG and making Motion JPEG is a trivial extension since MJPEG is close to sequential JPEGs.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Technical hurdle on the PC:
To be able to play video or audio you need to have a de-compressor to understand the container (eg MOV, AVI) AND a de-compressor for each stream in the container.
Container examples: "MOV" –QuickTime Movie, "AVI" – Windows’ implementation of the “RIFF” format.
video examples: "M2V" -mpeg2 video, "MJPG" - Motion JPEG.
audio example: "PCM" - raw audio, "AC3" - Dolby Digital audio.

The algorithm is either built into the tool (eg MOV decode in QuickTime, Adobe & free tools like MPlayer etc) OR it has to be present as installed component on the PC.

On Windows there are TWO sorts of CODECs. The original "Video For Windows" (VfW) and DirectShow.
Editing tools (in general) ONLY work with VfW codecs.

Windows XP, Vista and Win7 all ship with DirectShow MJPEG codecs, but not VfW ….so easy to watch but you can’t EDIT.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
The problem:
So you have these MOV[ MJPEG + PCM ] or AVI[ MJPEG + PCM ] files and want to edit them to do something as simple as put them on a DVD for mom to see the new baby.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
The solution:
You can buy a VfW Motion JPEG decoder (eg http://www.morgan-multimedia.com/M-JPEG2000/index.htm )
OR you can get it in freeware.

FFDshow ( http://sourceforge.net/projects/ffdshow/ ) uses the libavcodec library developed in the FFmpeg Movie Player ( http://ffmpeg.org/ ) open-source effort. The libavcodec library has TONS of codecs. It is better known for its MPEG4, Xvid, DivX but it also has a lot of others. One of those “others” is MJPG.

The FFDshow project takes that decode (and in some cases encode too) capability and presents them as VfW (as well as DirectShow) codecs!

Viola, you now have the ability to understand MJPEG in video editing tools. (and tons of other codecs if you enable them).

The only thing you need to do after installation is go to the Start Menu’s “FFDShow VFW Configuration” and under the “Decoder” tab’s “Codecs” (at the top). Then on the right side scroll down to MJPEG and change it from “disabled” to “libavcodec”.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
An Aside:
This will become more and more important in the years to come because motion films are shot and delivered to digital cinemas as “2k”, “4k” or “8k” ( http://en.wikipedia.org/wiki/Digital_cinema ) in MOTION JPEG2000 ( http://en.wikipedia.org/wiki/JPEG_2000#Motion_JPEG_2000 ) . It’s only so long before INDIE film makers and then pro-sumers want to edit in these resolutions.

‘thanks for reading.

[EDIT - LeChineur had problems with this solution. Instead of you having to read this entire thread I'm putting an edit here to summarize the problem]
In the end it came down to this. LeChineur's FFDshow install didn't work. In summary:

Originally Posted by LeChineur

This got me thinking. So I checked and I indeed did not have the ff_vfw.dll on my system. So I uninstalled the version 3.054 of FFDShow (dated 08/04/2009) which was installed on my system, downloaded and installed version 3.052 (dated 08/03/2009) and voila, I then had the ff_vfw.dll.

That solved the problem. PP now plays the video clip fine, as well as all the other "AVI" (MJPEG) clips from my Canon camera. Incredible!

So rallymax, your original post was in fact correct. Why the version of FFDShow that I first downloaded didn't install the correct dll the first time is still a mystery, but whatever...

use .htaccess to protect your web folder

2010-12-04T15:29:00.000-08:00

Generate the `password file`

htpasswd -c .htpasswd fred

(where fred is the username you want to use). You'll be prompted to enter and retype your password, then the .htpasswd file will be created for you.

Next, upload this file to your website. Make sure you place it outside the Web root of your site if possible, as you don't want just anyone to be able to view the file! For example, place it above your public_html or htdocs folder. (Having said this, Apache is often set up by default to block web-based access to files beginning with .ht. Better safe than sorry though!)
If you can't place your .htpasswd file outside your Web root, name it something that's not easily guessable - for example, .htxuymwp - so that people won't be able to find it easily. (In addition, it helps to start the filename with .ht; as mentioned earlier, Apache usually blocks access to files starting with .ht.)

Creating the `.htaccess` file

Protecting a folder

To password protect a folder on your site, you need to put the following code in your .htaccess file:


AuthUserFile /full/path/to/.htpasswd
AuthType Basic
AuthName "My Secret Folder"
Require valid-user

/full/path/to/.htpasswd should be the full path to the .htpasswd file that you uploaded earlier. The full path is the path to the file from the Web server's volume root - for example, /home/username/.htpasswd or C:\wwwroot\username\.htpasswd. (If you're not sure of the full path to your site or home directory, ask your Web hosting company for this info.)
The above .htaccess file will password protect all files in the folder that it is placed in, and all sub-folders under that folder too. So if you wanted to password protect your entire site, you would place the .htaccess file in your Web root folder.

Protecting a file

To password protect just a single file in a folder, use the following .htaccess file:


AuthUserFile /full/path/to/.htpasswd
AuthType Basic
AuthName "My Secret Page"

<Files "mypage.html">
  Require valid-user
</Files>

This will password protect just the mypage.html file in the folder where you put the .htaccess file.

Export Audio from Windows Movie Maker

2010-12-03T16:52:00.001-08:00

In Windows Movie Maker....
To save only the audio track of a video clip...
drag the video to the "Audio/Music" track.
Then go to...File > Publish Movie and you'll get options
for audio quality and save it to a wma audio file.

This will result in a .wma audio file. Be sure
to drag the clip all the way to the left on the
timeline or you will have dead air at the
beginning.

an IM proxy

2010-12-03T05:10:00.001-08:00

http://www.imspector.org/

How to turn a Windows application into Windows Service

2010-12-01T14:30:00.000-08:00

http://www.tacktech.com/display.cfm?ttid=197

The Windows NT/2000 Resource Kit provides two utilities that allow you to create a Windows user-defined service for Windows applications and some 16-bit applications (but not for batch files).

Whats needed for Windows NT/2000:
Instrsrv.exe installs and removes system services from Windows NT/2000
Srvany.exe allows any Windows application to run as a service.
You can download both files here srvany.zip

This zip includes three files. The two you need srvany.exe and instsrv.exe to install the services and also srvany.wri which documents everything you can do with the program.
Note: Make sure the Services Manager is closed while running the DOS commands.

You will need to put these files in a directory called reskit At a MS-DOS command prompt(Start | Run | "cmd.exe"), type the following command:
<path>\reskit\INSTSRV.EXE "Service Name" <path>\reskit\SRVANY.EXE
This creates the service in the Services manager and the registry keys to setup what program to run.

Next open regedit.exe Start | run | regedit.exe

Next navigate to this registry key.
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\service name

From the Edit menu, click Add Key and name it Parameters
Next from the Edit menu, click Add Value and type this information.
Value Name: Application
Data Type : REG_SZ
String : <path>\<application.ext>

Now you can start your service from the Service Manager

With this same program you can remove the service also. Just run this command from command prompt.
<path>\reskit\INSTSRV.EXE "Service Name" REMOVE

Free online screen sharing website

2010-11-30T13:24:00.001-08:00

http://www.mikogo.com/

If you couple this with freeconference.com, you've got a free webcast solution. :-)

To map a list of addresses

2010-11-30T10:57:00.001-08:00

If you have a list of addresses you want to map, you can use Google Fusion Tables. Just import your EXCEL file and tell it which column is address, it will do the rest. Great product.

Journey of Life

IE and IPv6 address

Change WireShark Dispaly Time to GMT/UTC or other timezone

rsync using a different ssh port

Disable Windows Temporary IPv6 Address

Shrew Soft VPN client group auth key auth-mutual-psk

Running User Mode Linux as normal non-root user

StrongSwan Configuration Guide

ubuntu kvm KSM high cpu usage

Net-snmp debug token list

crontab for non-root user

Samba shared drive on Ubuntu/Debian

cvs server setup in debian / ubuntu

Setup CVS server on Debian / Ubuntu

Windows SSH client with automatic reconnect capability

compiling / building colinux

linux dynamic library executable and initialization

To read chinese in gvim in Windows

C/C++ dbug library

vim paste and indent

List unused C/C++ functions

Setup your own (small / tiny) Certificate Authority (CA)

simple telnet server by using socat

C, C++ source code tools to help you be productive

gitignore diff_exclude

git cheat sheet

Makefile Automatic Variables

vim global command

vim global command

gcc default include path

FREE PHP IDE editors

Ubnt airos upgrade firmware

virtualbox vboxheadless high cpu usage

Widows XP Remote Desktop IPv6

Simulate unplug usb device on Linux (USB Reset)

Tether Blackberry 9700 on Linux (Debian/Ubuntu)

Windows 2003 Server Remote Desktop Local Drive Map

airos persistent directory

Add static route via script

openwrt make targets

Useful_Trunk_Rebuild_Make_Targets

sqlite 3 string to integer conversion, similar to C function atoi

Windows 2003, XP, recursive taking file and folders ownership and permissions

Reliable UDP Library

P2P NAT HOLE PUNCHING UDP

Windows CE c# .net compact framework webException "the response did not contain an end of entity mark"

Windows CE c# .Net An error message cannot be displayed because an optional resource assembly containing it cannot be found.

popular free http proxy software

How to use curl to upload a file

A fairly new comparison of openswan and strongswan

Openswan vs strongSwan

Cross compile wget on Linux for Windows

vsftpd config file

persistent ssh tunnel

MyEnTunnel

Work really well. It retries to connect after the ssh session is disconnected.

A good blog about using Windows CE platform builder

Platform Builder: Summary of Building Windows CE

Windows CE static IP registry setting

hostapd madwifi in ap mode

udp port forward

php internal server error

Ubuntu Host, Windows 7 Guest, Raw disk, Virtual Box

boot Windows in Virtualbox

Headless Virtualbox on Ubuntu 10.04 LTS

Linux Command Line FTP/HTTP segemented download accelartor software

aria2c

axel

lftp (use the "pget" command of lftp)

ari2c and axel supports FTP, HTTP and HTTPS, while lftp supports more protocols such as FTPS, SFTP, etc.

/etc/hosts not working?

Windows CE 5.0 QFE download

Linux Network Interface statistics

jQuery TableSorter "o is undefined" problem

sqlite pdo write lock problem solved

Win32 applications GUI vs Console

php header redirect IE issue

Windows CE ssh server and client

Slick board and XML/SWF charts

Free php web stat analyzer

Generate the `password file`

Creating the `.htaccess` file