Thoughts on cloud server testing

Cloud service can be treated like a control system, which takes input and produces output.

Control system has two characteristics: Controllability and Observability

Control system is a tested by applying an impulse to it and observe the response of the system.

From this we get the inspiration of cloud server testing:

1. Increase observability: create observation points and record their values. Performance counters, Instrumentation, etc are the tools for this. This can be used for monitoring, but also to observe the internal state of the service for debugging and trouble-shooting.

2.  Apply different level of impulse input to observe the output of the system, and the observable states of the system.

3. Find the break point of the system and the corresponding impulse input.

How to install pnp4nagios on ubuntu 16.04

This is for Redhat, but it's close enough:

https://linuxhowtoguide.blogspot.com/2017/03/how-to-install-and-configure-pnp4nagios.html

Good Documentation on SO_REUSEADDR

https://stackoverflow.com/questions/14388706/socket-options-so-reuseaddr-and-so-reuseport-how-do-they-differ-do-they-mean-t

run X program headless in Linux

On remote server:
1. run "xvfb-run xterm" (replace xterm with your program). The runs a virtual frame-buffer/X server. The default display is :99. You can change that.
2. find out where the X auth file is written. Default iat /tmp/xvfb-run.XXXX/Xauthority.
3. x11vnc -display :99 -nopw -auth /tmp/xvfb-run.o3K0jQ/Xauthority

 on your local desktop, run vnc viewer to connect to the remote server. You can set up password in the server if you want to.

Linux L2tp client setup for Mac OS X vpn server

Instructions below are based on the work at https://github.com/hwdsl2/setup-ipsec-vpn/blob/master/docs/clients.md#linux
Commands must be run as root on your VPN client.
To set up the VPN client, first install the following packages:

# Ubuntu & Debian
apt-get update
apt-get -y install strongswan xl2tpd

# CentOS & RHEL
yum -y install epel-release
yum -y install strongswan xl2tpd

# Fedora
yum -y install strongswan xl2tpd

Create VPN variables (replace with actual values):

VPN_SERVER_IP='your_vpn_server_ip'
VPN_IPSEC_PSK='your_ipsec_pre_shared_key'
VPN_USER='your_vpn_username'
VPN_PASSWORD='your_vpn_password'

Configure strongSwan:

cat > /etc/ipsec.conf <<EOF
# ipsec.conf - strongSwan IPsec configuration file

# basic configuration

config setup
  # strictcrlpolicy=yes
  # uniqueids = no

# Add connections here.

# Sample VPN connections

conn %default
  ikelifetime=60m
  keylife=20m
  rekeymargin=3m
  keyingtries=1
  keyexchange=ikev1
  authby=secret
  ike=aes128-sha1-modp1024,3des-sha1-modp1024!
  esp=aes128-sha1-modp1024,3des-sha1-modp1024!

conn myvpn
  keyexchange=ikev1
  left=%defaultroute
  auto=add
  authby=secret
  type=transport
  leftprotoport=17/1701
  rightprotoport=17/1701
  right=$VPN_SERVER_IP

  rightid=%any

EOF

cat > /etc/ipsec.secrets <<EOF
: PSK "$VPN_IPSEC_PSK"
EOF

chmod 600 /etc/ipsec.secrets

# For CentOS/RHEL & Fedora ONLY
mv /etc/strongswan/ipsec.conf /etc/strongswan/ipsec.conf.old 2>/dev/null
mv /etc/strongswan/ipsec.secrets /etc/strongswan/ipsec.secrets.old 2>/dev/null
ln -s /etc/ipsec.conf /etc/strongswan/ipsec.conf
ln -s /etc/ipsec.secrets /etc/strongswan/ipsec.secrets

Configure xl2tpd:

cat > /etc/xl2tpd/xl2tpd.conf <<EOF
[lac myvpn]
lns = $VPN_SERVER_IP
ppp debug = yes
pppoptfile = /etc/ppp/options.l2tpd.client
length bit = yes
EOF

cat > /etc/ppp/options.l2tpd.client <<EOF
ipcp-accept-local
ipcp-accept-remote
refuse-eap
require-chap
noccp
noauth
mtu 1280
mru 1280
noipdefault
defaultroute
usepeerdns
connect-delay 5000
name $VPN_USER
password $VPN_PASSWORD
EOF

chmod 600 /etc/ppp/options.l2tpd.client

The VPN client setup is now complete. Follow the steps below to connect.

Note: You must repeat all steps below every time you try to connect to the VPN.
Create xl2tpd control file:

mkdir -p /var/run/xl2tpd
touch /var/run/xl2tpd/l2tp-control

Restart services:

service strongswan restart
service xl2tpd restart

Start the IPsec connection:

# Ubuntu & Debian
ipsec up myvpn

# CentOS/RHEL & Fedora
strongswan up myvpn

Start the L2TP connection:

echo "c myvpn" > /var/run/xl2tpd/l2tp-control

Run ifconfig and check the output. You should now see a new interface ppp0.
Check your existing default route:

ip route

Compile GoLang for AR9341

Golang support for MIPS 32 has been added since version 1.8. However, Soft FPU is not added, making chipsets like AR9341 not able to run Go program. "vstafanovic" has submitted the patch but
it has not been accepted yet in 1.9.0. Hopefully it will make to 1.10

At the same time, you can apply the patch yourself to version 1.8.3:

1. Download the patch
2. Download Golang 1.8.3 source code and apply the patch
3. cd src; ./bash.all

If everything goes well, you will have a compiled go toolchain.

To compile your application to MIPS, do:
GOOS=linux GOARCH=misp GOMIPS=softfloat go build

automatically set gnu screen window title

http://scie.nti.st/2008/8/19/1-minute-post-hostname-as-screen-window-title/

In short, add this line to the remote host's .bashrc:

[ "$TERM" = "screen" ] && PROMPT_COMMAND='echo -ne "\033k$HOSTNAME\033\\"'

Setting ssh server to an user to only SFTP to the user's home directory

Here is a guide for setting up SFTP users who’s access is restricted to their home directory.

Add the following to the end of the /etc/ssh/sshd_config file:

Subsystem sftp internal-sftp

# This section must be placed at the very end of sshd_config
Match Group sftponly
    ChrootDirectory %h
    ForceCommand internal-sftp
    AllowTcpForwarding no

This means that all users in the ‘sftponly’ group will be chroot’d to their home directory, where they only will be able to run internal SFTP processes.

Now you can create the group sftponly by running the following command:

$ groupadd sftponly

Set a user’s group:

$ usermod steve -g sftponly

To deny SSH shell access, run the following command:

$ usermod steve -s /bin/false

And set the user’s home directory:

$ sudo chown root /home/steve

$ sudo chmod go-w /home/steve

$ sudo mkdir /home/steve/writable
$ sudo chown steve:sftponly /home/steve/writable
$ sudo chmod ug+rwX /home/steve/writable

Finally, you probably need to restart SSH

$ service ssh restart

The SSH part should now be in order, but you should make sure that file permissions also are correct. If the chroot environment is in a user’s home directory both /home and /home/username must be owned by root and should have permissions along the lines of 755 or 750.
In other words, every folder leading up to and including the home folder must be owned by root, otherwise you will get the following error after logging in:

Write failed: Broken pipe
Couldn't read packet: Connection reset by peer

Great post on jffs2 loop mount and endian issue

https://integriography.wordpress.com/2015/03/16/mounting-a-jffs2-dd-image-in-linux/

xxd reverse with an offset

When using xxd to reverse a hex dump file, if you hexdump file has a non-0 offset like this:

bc000000: 01 02 03 04 05 06 07 08  ........

You would need to use the "-s offset" option of the xxd. However, there is a bug in the code that makes this options only works as the FIRST option. Otherwise, it wouldn't work.

You want to do this:

xxd -s -0xbc000000 -r -g 1 test.dump test.bin

Basically the xxd is hardcoded to look for the offset at argv[2].

Another alternative:
https://github.com/pheehs/hexdump2bin/blob/master/hexdump2bin.py

ios command line console log viewing

use "idevice_id --list" to list the UUIDs.
use "deviceconsole" to actually view the logs
deviceconsole -u <UUID>

unbrick TPLINK Archer C7 V2 (2017-05 from Amazon)

I bricked my Archer C7 v2 with bad configuration.

TFTP boot didn't work for me. It turned out that the product id doesn't match.

Had to connect to console.

1. The pinout is as follows. The warning on this page (https://wiki.openwrt.org/toh/tp-link/tl-wdr7500#tftp_recovery_de-bricking) itself is wrong.

2. I used the Openwrt Snapshot image. I tried the official image from TPLink website but that didn't work. I didn't try the "cut" process described in the above link.

3. The command is as follows

type "tpl" really fast at boot time to stop the autoboot

tftpboot 0x81000000 [name of your firmware file].bin
erase 0x9f020000 +f80000
cp.b 0x81000000 0x9f020000 0xf80000
reset

That's it.

socket buffer size

To find the current socket buffer size:

getsockopt(fdsocket,SOL_SOCKET,SO_RCVBUF,(void *)&n, &m);

getsockopt(fdsocket,SOL_SOCKET,SO_SNDBUF,(void *)&n, &m);

To find out more, do "man getsockopt"

To find the current bytes in the socket's buffer:

ioctl(fd,FIONREAD,&bytes_available)

ioctl(fd,FIONWRITE,&bytes_available)

To find out more, do "man ioctl"

FreeBSD recompile kernel

Download:
ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/amd64/10.3-RELEASE/src.txz

untar this to /usr/src

Go to the kernel source directory which contains the configurations.

cd /usr/src/sys/amd64/conf

Create a folder named kernel in the home directory of root user i.e. /root.

mkdir /root/kernels

config -x /root/kernels/MYKERNEL

The above command generates the current configuration of the kernel

Now you can add the options you want to change.

Create a soft link in the /usr/src/sys/amd64/conf named “MYKERNEL” which links to /root/kernels/MYKERNEL file.

ln -s /root/kernels/MYKERNEL

Goto the folder created above.

cd /root/kernels/

To build a file which contains all available options, run the following commands.

cd /usr/src/sys/amd64/conf
make LINT

Goto the main source folder.

cd /usr/src

Build and Install the new kernel with reference from the file “MYKERNEL”.

make buildkernel KERNCONF=MYKERNEL
make installkernel KERNCONF=MYKERNEL

Now reboot the machine to boot into the new kernel you just created now.

use "uname -a" and "sysctl -a" to check


Enable multi routing table in the kernel:
https://www.mmacleod.ca/2011/06/source-based-routing-with-freebsd-using-multiple-routing-table/

FreeBSD dump current kernel config

config -x /boot/kernel/kernel

Adding swap to your VM

sudo fallocate -l 1G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
sudo swapon --show
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

seafiles upload error

The solution:
I needed to go into the webui system admin -> settings and change FILE_SERVER_ROOT to be seafile.example.com/seafhttp as per the local help text.

Proxy settings in Windows 10, LAN settings

right click start -> control panel -> internet options -> connections tab -> LAN settings

R7000 serial consle

With the top facing up, and Ethernet ports facing away from you, i.e. the 4 console pins on the right bottom of the board, the pinout from left to right is (yellow,red,green for my personal setup):

RX, which should connects to TX of your CP2102
TX
GND
Unused

openwrt kernel config change

In OpenWRT build, to change kernel config, you need to do the following:
1. make kernel_menuconfig CONFIG_TARGET=subtarget
This updates the .config file in build_dir/Linux-xxxxx
2. copy the file .config from the above directory to "target/Linux/your-board-name/config-3.14"
3. make V=99
The make command will copy the kernel config file from target/Linux to build_dir

Reference:
https://wiki.openwrt.org/doc/howto/build

golang vim-go setup

1. start with a fresh ~/.vim directory
2. install vim-plug if don't have it:

curl -fLo ~/.vim/autoload/plug.vim --create-dirs \
    https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim

3. add the following to your .vimrc file
call plug#begin()
Plug 'tpope/vim-sensible'
Plug 'fatih/vim-go'
Plug 'tmhedberg/matchit'
call plug#end()
au FileType go nmap <C-K> <Plug>(go-doc)
let g:go_fmt_command = "goimports"4. inside vim, run "PlugInstall". This installs the plugins specified above. Restart vim. All plugins should be working now.
5. For vim-go, run ":GoInstallBinaries" to install missing tools needed by vim-go
6. Now Shift-K/Ctrl-K should run go-doc for the keyword under cursor.

 Note for go-doc to work correctly, the go source code file has to be in $GOPATH/src/package-name. It cannot not be a symlink outside of GOPATH. It cannot be a raw *.go file under $GOPATH/src either. It needs to be under a package directory.

windows 10 folder sort slow in file explorer

I solved this by unchecking the "Allow Files in the folder to have contents indexed in addition to file properties" option. You do this by first getting the Properties of the folder, then on the General tab press the Advanced button in the Attributes section. Apply the changes to the folder and , subfolders and files.