Bell Canada throttles bittorrent traffic during "peak hours." This means that I'm limited to 30Kb/s evenings and weekends. This is not a situation with which I have been satisfied; besides being inconvenient, I don't much like being pushed around by Bell Canada in violation of our contract. I could have perfectly valid reasons for downloading via bittorrent, by golly, and they shouldn't be able just to shut me down without cause, etc. Plus I couldn't download Farscape fast enough when I was jonesing.

In the end, I figured the simplest solution was to download through a VPN, so Bell couldn't see that the packets were torrenty. I didn't really have any idea how to do this, but I have Google, and that generally gives me an inflated sense of ability, so I set to with a will. Here's what I did (let me also say off the bat that I was successful without too much trouble):

1. Get a VPN endpoint on the Internet somewhere. I'd used VPSLink before with perfectly adequate results, and their cheapest plan is like six bucks a month if you buy a year up front. I did so and installed Hardy. That was that.


2. Set up a torrent server locally. I have been using torrentflux-b4rt as my client for at least a year now, and it works great. In this case, I wanted it running on its own server, so I could just change its gateway to the VPN endpoint and be done with it. I set up a new VM, installed Apache and torrentflux-b4rt, made the downloads directory on my previous server available via NFS, mounted that on the VM, and disabled the original torrentflux instance. I ended up with the same setup as before, only on its own server.


3. Create a point-to-point VPN connection. To connect my torrent server to my VPN endpoint, I set up the simplest of OpenVPN configurations. No trouble there. That example worked without a hitch.


4. Route all traffic from the torrent server through the VPN endpoint. I was working on a bunch of fancy ways to do this when I learned that OpenVPN can already do it for you. I added:
   

   redirect-gateway

   
   to the config file on my client. Once I reconnected, it set up all the routes necessary to direct all traffic through the VPN connection. I tried a traceroute to Google, and the first hop was indeed the VPN endpoint, but of course it stalled there because that box wouldn't do IP forwarding yet.


5. Enable IP forwarding on the VPN endpoint. This was a little trickier than I expected, because the VPS didn't have MASQUERADE support. Luckily, it has a static IP, so I could use SNAT (it was just luck I figured out how to do this. I still don't know what any of this really does):
   

   # echo "1" > /proc/sys/net/ipv4/ip_forward
   # iptables -t nat -A POSTROUTING -o venet0 -j SNAT --to-source 209.59.210.206

   
   A traceroute from the torrent server to Google was now able to complete. And that was that!

Now I don't have any throttling problems. Also, if I connect my laptop to the VPN endpoint using redirect-gateway, I'll be able to watch US content that's blocked in Canada, like the Olympics stuff on NBC. Too bad that ended this weekend.

Read More...

I was writing some Python tools for svn access, and found myself needing to parse the XML that svn log and svn info return. Python's XML libraries are pretty annoying to use, to say the least. I wrote up a quick way to walk an XML document using dotted name syntax, and it seemed like it could be generally useful, so I cleaned it up, documented it, and packaged it up for release.

Besides using dotted-name syntax to walk an XML document, simplexmlapi can map attributes to dotted names, which will let you make an easy API for known XML structures. For instance, you can map the svninfo.repository.url dotted name to the repourl attribute. Thus, one can simply:

>>> d = simplexmlapi.loads(
...         '<xml><doc><item>somevalue</item></doc></xml>',
...          map=dict(val="doc.item"))
>>> d.val
u'somevalue'

The project is hosted at http://simplexmlapi.googlecode.com/ It's also available from the Cheese Shop and via easy_install simplexmlapi. API docs are hosted at http://packages.python.org/simplexmlapi.

Read More...

Either the IPsec client I've been using to connect to the Zenoss colo or the VPN server on the other end kills my connection every twenty-five minutes. We've got an OpenVPN server running at the colo as well, and that connection's been much more stable. Since I have multiple boxes here that I'd like to have colo access, I decided to try to connect the two networks semi-permanently, as a fun weekend project. It actually turned out to take around two hours.

1. Set up the VPN connection. This ended up being very, very easy. My nameserver VM (IP: 10.42.1.2) was hardly overtaxed, so I decided to use that as the VPN gateway. First, I installed OpenVPN:
   

   ian@tryphon$ sudo apt-get install openvpn

   
   I already had an OpenVPN config, which I'd been using (via TunnelBlick) to connect to the OpenVPN server at the colo. I dumped that, along with the necessary keys and whatnot, into /etc/openvpn, and fired up the connection:
   

   ian@tryphon$ sudo /etc/init.d/openvpn start zenoss
* Starting virtual private network daemon.
* zenoss (OK)
...done.


   
   Then I verified connectivity by pinging a server at the colo:
   

   ian@tryphon:/etc/openvpn$ ping 10.175.211.10
PING 10.175.211.10 (10.175.211.10) 56(84) bytes of data.
64 bytes from 10.175.211.10: icmp_seq=1 ttl=62 time=52.5 ms
64 bytes from 10.175.211.10: icmp_seq=2 ttl=62 time=48.1 ms

--- 10.175.211.10 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 48.150/50.338/52.527/2.199 ms

   
   


2. Route colo traffic through the gateway. This was the only tricky part, mostly (I think) because my understanding of IP routing is nonexistent to limited, which is why I thought this would be a fun project in the first place.
   
   I should mention here that the <code>zenoss.conf</code> file I had sets routes appropriately (obviously, or step 1 wouldn't have worked):
   

   ian@tryphon$ netstat -rn
   Kernel IP routing table
   Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
   10.254.0.1      10.254.0.9      255.255.255.255 UGH       0 0          0 tun0
   10.254.0.9      0.0.0.0         255.255.255.255 UH        0 0          0 tun0
   10.175.211.0    10.254.0.9      255.255.255.0   UG        0 0          0 tun0
   10.42.1.0       0.0.0.0         255.255.255.0   U         0 0          0 eth0
   0.0.0.0         10.42.1.1       0.0.0.0         UG        0 0          0 eth0
   

   
   So I just needed to get the rest of the network to use that connection.
   
   First, I added a static route on my router, sending 10.175.211.0/24 traffic to my VPN box (user interfaces will vary, of course; mine's a Linksys with the default firmware. dd-wrt won't work on my router, which causes me no end of annoyance, but that's another story):
   

   

   
   That gets me to the box but not out through the VPN. Next I needed to enable IP forwarding on the VPN box:
   

   ian@tryphon$ sudo su -
root@tryphon# echo 1 > /proc/sys/net/ipv4/ip_forward
root@tryphon# echo "net.ipv4.ip_forward=1" > /etc/sysctl.conf
root@tryphon# sysctl net.ipv4.ip_forward=1


   
   I don't know if the first line is necessary given the sysctl stuff, but The Web seemed to think it was the way to go, and it didn't break anything. People who understand this stuff better may educate me if they so wish.
   
   I got to this point, and it wasn't working. I don't fully understand why, but comments from a co-worker led me to investigate how to tell the box to which interface it should forward things. Googling around led me to try installing iptables and adding a rule:
   

   ian@tryphon$ sudo apt-get install iptables
...
ian@tryphon$ sudo /sbin/iptables -A FORWARD -i tun0 -j ACCEPT

   
   This worked! From anywhere on my network, I was able to ping the colo. Hooray! In order to get it to work on boot, I added the above command to /etc/rc.local
   


3. Use the remote nameserver. This was simple. Since I'm already running BIND locally, I just added a forwarder to /etc/bind/named.conf.local:
   

   zone "zenoss.loc" IN {
type forward;
forwarders {10.175.211.10;};
};

   
   Restarted BIND, and all was well.
   
   Oh, I had to add zenoss.loc to the search line of /etc/resolv.conf. I have yet to find a good way around that, though I'm certain one exists. I may move DHCP from my router to that box, so I can specify more search domains.
   

Read More...

So today I learned about an incredibly simple way to split up essential services on my home network with VMs. It isn't that hard to do normally, of course, but there's a lot of rote overhead in VM creation. Enter ubuntu-vm-builder!

1. First things first. You'll need VMWare Server or KVM installed, obviously, to run VMs. VirtualBox would work too, but it hasn't been implemented yet in ubuntu-vm-builder -- the option's listed in the usage description, but don't be fooled. I personally used VMWare Server, to which this post will continue to refer. This tutorial ought to get you up and running, although I also had to install the libxtst6 package to get it to accept my serial number.


2. Next, set up apt-proxy, which is a must-have if you've got a lot of Ubuntu boxes. It acts as...well, as a proxy to repositories, caching package files. Thus, if you upgrade a bunch of Ubuntu boxes through the proxy, you'll only have to download the packages once. This will let you create all your VMs after the first in about five minutes. I used this tutorial, and it worked without a hitch.
   


3. Finally, install ubuntu-vm-builder, which is in the Hardy repos:
   
   

   $ sudo apt-get install ubuntu-vm-builder

   
   
   It'll install some dependencies as well. Once that's all done, you're ready to create a VM, which couldn't be simpler:
   
   

   
$ sudo ubuntu-vm-builder vmserver hardy
--hostname dupondt --addpkg ssh -d dupondt
--mirror http://10.42.1.6:9999/ubuntu --ip 10.42.1.8


   
   
   A little explanation. The first argument is the type of VM -- kvm, vbox (which, as I mentioned, doesn't work), vmserver, and a few others. Second is the Ubuntu suite you want installed -- hardy, feisty, gutsy, etc. The -d option is for specifying the path where you want the VM created, which is important because otherwise it'll create it in $PWD named something like ubuntu-hardy-i386-vm, which is fairly useless. --addpkg lets you specify packages you want installed on the VM, clearly; it can be repeated (e.g. --addpkg mysql-server --addpkg apache2). Finally, the --mirror option is the repository you want it to use. That's where you specify your apt-proxy box. The rest are basic configuration options; see the usage for the full list. There are some useful ones; you can specify the name and password of the default user to be created, and even feed it an SSH pub key to be added to authorized_keys.
   
   That'll chug away for a while (if it's the first time you're populating the proxy; otherwise it'll be very fast). At the end, you'll have a folder wherever you specified with a VM in it. If you're using VMWare Server, as I did, register it:
   
   

   $ sudo vmware-cmd -s register /path/to/vms/dupondt/dupondt.vmx


   
   
   And start it up:
   
   

   $ sudo vmware-cmd /path/to/vms/dupondt/dupondt.vmx start


   
   
   And you're good to go! If you didn't specify a user/password during creation, it's ubuntu/ubuntu.
   

So that's how I got me a nameserver and a Zenoss server in no time. Now I need to think of reasons for more VMs.

Read More...