On Virtualbox 4.2, if you assign multiple cores to your VM, and you are running Linux Guest, you may experience high network latency (ssh typing is sporadic even on local GigE network).
This is a bug with Virtualbox.
The solution: Change your VM Ethernet type to PCnet. Then it works!
Here is the link to the bug report: https://www.virtualbox.org/ticket/10157