New Solaris PatchFinder tool

Anyone who has ever used the existing Solaris patch search tool will know how limited it is. Well Sun have released a much improved search page called Software Update Finder. Gerry Haskins has detailed blog about its new capabilities which include:

• Filtering results by OS release and architecture.
• Limiting search results to Security and/or Recommended patches.
• Searching for patches that fix a particular CR or BugId.

See Gerry's article for more features. This is just the initial release and there are plans to enhance it further in the future. Let Sun know what you want the tool to do.

Python ipaddr performance

Last weekend while I was cleaning up my IP address summarization script (I added a setup.py, created a Cheese Shop entry and a downloadable archive) I had a look at the state of IP address manipulation in Python and found a new module called ipaddr. What sparked my interest in this module was that it had already been integrated into upcoming Python 2.7 and 3.1 as a standard library.

It seemed to contain all the functionality of IPy but with much cleaner code. It also has a function called collapse_address_list() that is similar to my summarize function but it can handle a list of non-contiguous networks and IP addresses and collapse them down.

When I wrote my summarize() function I spent quite a while optimizing the algorithm so that it can handle large address ranges in a reasonable amount of time. For example, on my development box it can summarize the two worst case ranges: IPv4 0.0.0.0 to 255.255.255.254 in 0.1s and IPv6 :: to ffff:ffff:ffff:ffff:ffff:ffff:ffff:fffe in 0.2s.

I had to see how well ipaddr performed so I wrote a little benchmarking script that summarized a /24. The results were very poor with it taking 64.27 seconds to perform 1000 runs. I then ported my summarize() code to use ipaddr and modified collapse_address_list() to use it. The result was a 50 times improvement in performance (1000 runs took 1.21 seconds).

ipaddr is developed by Google and the coders use Reitveld for reviewing patches. So I've submitted my patches there. Feel free to review and comment on the code.

Here's hoping that they are accepted and benefit everyone by making it into the next release(s) of Python!

Django Serializer Updates

I've had a couple of emails and forum posts where users were having difficulty with my serialization module. The problems mostly centered around installing it correctly.

I did a little work tonight to clean up the installation side of things. As a result you can now find the module in the Cheese Shop.

The latest stable release for the serialization module can also be obtained by:

• Running easy_install wadofstuff-django-serializers
• Downloading the 1.0.0 release and running python setup.py install.

The source tree has been moved around a little to accommodate these changes and can be found here.

There were no new features or bug fixes made but I have a few interesting tickets from users that I'll be reviewing over the coming weeks.

Solaris 10 5/09 (Update 7) is out...

...well almost. The DVD ISO isn't available yet for download but you can check out What's New and the Release Notes on docs.sun.com while you wait.

From my reading so far there doesn't seem to be many exciting new features but the Solaris 10 5/09 Patch List at least lists some patches that I'll no longer have the pain of applying to the new miniroot boot archives.

Full Django Serializers - Part II

In the first part of this article I covered the excludes and extras options to the Wad of Stuff serializer. In this article I introduce the relations option.

Relations

The Wad of Stuff serializers allow you to follow and serialize related fields of a model to any depth you wish. This is why it is considered a "full serializer" as opposed to Django's built-in serializers that only return the related fields primary key value.

When using the relations argument to the serializer you may specify either a list of fields to be serialized or a dictionary of key/value pairs. The dictionary keys are the field names of the related fields to be serialized and the values are the arguments to pass to the serializer when processing that related field.

This first example shows the simplest way of serializing a related field. The Group model has a ManyToManyField to the Permission model. In this case there is only one permission but if there were more then each would be serialized.

>>> print serializers.serialize('json', Group.objects.all(), indent=4, relations=('permissions',))
[
    {
        "pk": 2,
        "model": "auth.group",
        "fields": {
            "name": "session",
            "permissions": [
                {
                    "pk": 19,
                    "model": "auth.permission",
                    "fields": {
                        "codename": "add_session",
                        "name": "Can add session",
                        "content_type": 7
                    }
                }
            ]
        }
    }
]

The simple case may be all you need but if you want more control over exactly which fields or extras are included, excluded, and the depth of relations to follow then you need to pass a dictionary in the relations option. This dictionary is a series of nested dictionaries that are unrolled and passed as arguments when serializing each related field.

>>> print serializers.serialize('json', Group.objects.all(), indent=4, relations={'permissions':{'fields':('codename',)}})
[
    {
        "pk": 2,
        "model": "auth.group",
        "fields": {
            "name": "session",
            "permissions": [
                {
                    "pk": 19,
                    "model": "auth.permission",
                    "fields": {
                        "codename": "add_session"
                    }
                }
            ]
        }
    }
]

The relations option in this example roughly translates to a call to serialize('json', permissions_queryset, fields=('codename',)) when the permissions field is serialized.

Serializing deeper relations

The power of the relations option becomes obvious when you see it in action serializing related fields that are 2 or more levels deep. Below the content_type ForeignKey field on the Permission model is also serialized.

>>> print serializers.serialize('json', Group.objects.all(), indent=4, relations={'permissions':{'relations':('content_type',)}})
[
    {
        "pk": 2,
        "model": "auth.group",
        "fields": {
            "name": "session",
            "permissions": [
                {
                    "pk": 19,
                    "model": "auth.permission",
                    "fields": {
                        "codename": "add_session",
                        "name": "Can add session",
                        "content_type": {
                            "pk": 7,
                            "model": "contenttypes.contenttype",
                            "fields": {
                                "model": "session",
                                "name": "session",
                                "app_label": "sessions"
                            }
                        }
                    }
                }
            ]
        }
    }
]

Combining options

You may also combine the other options when serializing related fields. In the example below I am excluding the content_type.app_label field from being serialized.

>>> print serializers.serialize('json', Group.objects.all(), indent=4, relations={'permissions':{'relations':{'content_type':{'excludes':('app_label',)}}}})
[
    {
        "pk": 2,
        "model": "auth.group",
        "fields": {
            "name": "session",
            "permissions": [
                {
                    "pk": 19,
                    "model": "auth.permission",
                    "fields": {
                        "codename": "add_session",
                        "name": "Can add session",
                        "content_type": {
                            "pk": 7,
                            "model": "contenttypes.contenttype",
                            "fields": {
                                "model": "session",
                                "name": "session"
                            }
                        }
                    }
                }
            ]
        }
    }
]

That wraps up this series of articles. Head over to Wad of Stuff at Google Code to grab the source and don't hesitate to open a ticket if you find any bugs or have any enhancement requests.