Django gets support for IPv6 fields

My favorite web development framework recently committed some code to add support for IPv6 addresses in data models. The commit closes a ticket I opened 6 years ago. Congratulations to Erik Romijn for finally closing this off. Better late than never :)

The new code differs from my original code in a number of ways. First of all it has better testing and documentation. Second, it does IPv6 address validation programmatically rather than using a regular expression.

The code also borrows from ipaddr-py, which I also contribute to.

A variation on my original code is still running and has been used to manage IPv6 addresses so I can vouch for it. Most of my code doesn't actually use IPAddressFields in Django because of this even older issue which was fixed 3 years ago.

Now both of them are fixed I might look at using the new fields.

Python ipaddr performance

Last weekend while I was cleaning up my IP address summarization script (I added a setup.py, created a Cheese Shop entry and a downloadable archive) I had a look at the state of IP address manipulation in Python and found a new module called ipaddr. What sparked my interest in this module was that it had already been integrated into upcoming Python 2.7 and 3.1 as a standard library.

It seemed to contain all the functionality of IPy but with much cleaner code. It also has a function called collapse_address_list() that is similar to my summarize function but it can handle a list of non-contiguous networks and IP addresses and collapse them down.

When I wrote my summarize() function I spent quite a while optimizing the algorithm so that it can handle large address ranges in a reasonable amount of time. For example, on my development box it can summarize the two worst case ranges: IPv4 0.0.0.0 to 255.255.255.254 in 0.1s and IPv6 :: to ffff:ffff:ffff:ffff:ffff:ffff:ffff:fffe in 0.2s.

I had to see how well ipaddr performed so I wrote a little benchmarking script that summarized a /24. The results were very poor with it taking 64.27 seconds to perform 1000 runs. I then ported my summarize() code to use ipaddr and modified collapse_address_list() to use it. The result was a 50 times improvement in performance (1000 runs took 1.21 seconds).

ipaddr is developed by Google and the coders use Reitveld for reviewing patches. So I've submitted my patches there. Feel free to review and comment on the code.

Here's hoping that they are accepted and benefit everyone by making it into the next release(s) of Python!

IPv4 Exhaustion Counter

I've added the IPv4 Exhaustion Counter to my side bar. It uses data provided by Geoff Huston in his IPv4 Address Report. His models currently predict that IANA will run out of addresses on 24th January 2011 with the RIR's running out by 11th December 2011.

IPMP with IPv6 test addresses

Quite a while back Dave Miner discussed how IP Multipathing can be done on the cheap in Solaris 10 by not using IPv4 test addresses at all and relying on the physical link state or alternatively using IPv6 link local addresses as test addresses. My first attempts at configuring the latter failed and not getting any response to my comment in his blog, I left it for more pressing work.

Recently I got back to this and managed to get it working.

Firstly you need to have /etc/inet/hosts and/or /etc/inet/ipnodes set up correctly with your IP address and hostname. In this example our host is called myhost with two interfaces bge0 and bge1.

/etc/inet/hosts:
192.168.1.1 myhost

Then configure your interfaces.

/etc/hostname.bge0:
myhost group production failover up

/etc/hostname.bge1:
up

/etc/hostname6.bge0:
group production -failover up

/etc/hostname6.bge1:
group production -failover up

You need to reboot for these configuration changes to take effect or you can pass the contents of the /etc/hostname* files as arguments to ifconfig.

Update: You need to ensure that both the host and the switch are set to autonegotiate for this to work.

Update 2007.09.20: According to this Sun document you do not need to mark an IPv6 test address as deprecated to prevent applications from using the test address. I've updated the configurations above to reflect this.

Summarizing an IP address range

As I've mentioned previously, I've found IPy python module to be extremely useful for manipulating IP addresses. One such use is a script I've written to summarize an IP address range into the networks that make it up. The script supports both IPv4 and IPv6 addresses.

Example usage:

$ ./summarize.py 192.168.1.0 192.168.1.8
192.168.1.0/29
192.168.1.8

or alternatively as a python module:

>>> from summarize import summarize
>>> summarize('192.168.1.0', '192.168.1.254')
['192.168.1.0/25', '192.168.1.128/26', '192.168.1.192/27', '192.168.1.224/28', '192.168.1.240/29', '192.168.1.248/30', '192.168.1.252/31', '192.168.1.254']

The source for this script is available for download from the Wad of Stuff repository.

Sorting IP addresses in python

Here's a quick example of a function to sort a list of IP addresses in python using the decorate-sort-undecorate idiom (aka Schwartzian Transform) and the IPy module.

def sort_ip_list(ip_list):
    """Sort an IP address list."""
    from IPy import IP
    ipl = [(IP(ip).int(), ip) for ip in ip_list]
    ipl.sort()
    return [ip[1] for ip in ipl]

And here is an example of it in use:

>>> l = ['127.0.0.1', '192.168.0.10', '192.168.0.1', '10.1.1.1', '172.16.255.254']
>>> sort_ip_list(l)
['10.1.1.1', '127.0.0.1', '172.16.255.254', '192.168.0.1', '192.168.0.10']

Manipulating IPv4 and IPv6 addresses in python

Eighteen months ago, after many years of programming in perl for most of my systems programming needs, I decided to give python a go after much coaxing by a colleague, Alec Thomas. At the time I had started developing an IP address management system, designed for ISPs/Telcos who need to manage hundreds of address blocks and allocations to customers and internal infrastructure. I did a quick evaluation of both Ruby on Rails and Django and decided on Django for a few reasons:

• I didn't have to manage my database schema and models separately. Django allowed me to define my data models in a single place and it handled the job of creating the database tables. (This was before Rails had migrations).
  
• Django's built in administrative interface was a huge time saver and allowed me to focus on developing my application rather than designing forms.
• After programming in perl for so long the cleanliness of the python language really appealed to me. Ruby to me just seemed like OO-Perl done properly but with all the $@!#{} perlisms left in.
  

While developing the application I found an extremely useful python module called IPy (originally developed here), that handles IPv4 and IPv6 addresses and networks.

Here's a sample of how you can use it:

>>> from IPy import IP
>>> ip = IP('127.0.0.0/30')
>>> for x in ip:
...  print x
...
127.0.0.0
127.0.0.1
127.0.0.2
127.0.0.3
>>> ip2 = IP('0x7f000000/30')
>>> ip == ip2
1
>>> ip.reverseNames()
['0.0.0.127.in-addr.arpa.', '1.0.0.127.in-addr.arpa.',
'2.0.0.127.in-addr.arpa.', '3.0.0.127.in-addr.arpa.']

I'll be discussing some tools I've developed with this module at a later date.