Tuesday, 17 November 2009

Behaviour Driven Infrastructure

I've been following the development of puppet for many years and this gem of a thread caught my attention recently. Martin Englund asks the Puppet Users mailing list:
how do you validate that puppet has done what it is supposed to, and even troublesome, how you validate that it has done what you intended it to do?
This is something I've struggled with over the years with my JASS/SST-based jumpstart build system. I've gone so far as to automate the build testing process using buildbot and pass or fail a build by using regexps to search for errors in the installation process. But my testing ends when the console login prompt appears. Validating whether a system build functions as intended is beyond what I currently test now and even beyond the capabilities of puppet.

This is where Martin's Behaviour Driven Infrastructure (BDI) approach comes into play. Martin is using Cucumber, a Behaviour Driven Development testing tool, to describe a system's behaviour using natural language that is readable and easy to understand by non-technical users (i.e. your IT helpdesk or even your business stakeholders).

Where this really gets interesting is combining puppet, cucumber and a monitoring system such as Nagios to do Test Driven Infrastructure. For example, you can use cucumber-nagios to integrate your cucumber tests with Nagios, then write a test for a new feature you want the system to have, e.g. "Should be able to send email". Initially this would result in Nagios marking the system as having a fault because you haven't yet implemented the feature that passes the test. You then proceed to implement the feature using puppet such that the test passes.

Over time I would imagine the amount of test coverage to grow such that system behaviours like DNS resolution, LDAP authentication, host based firewall policies etc would be tested. Any change to the system that broke one of these tests could be quickly pinpointed and fixed.

A perfect example in the Solaris world is the application of the latest recommended patches to a box. At a minimum it tends to break the sendmail and snmpd configs on my systems and I know to manually backup and restore these files before and after applying the patches. With BDI combined with a monitoring system you could be alerted to these breakages or any others that you weren't previously aware of and rapidly respond to them.

Brilliant!

Thursday, 8 October 2009

Speaking of Solaris 10 Update 8...

You can now read about What's New.

Quite a few ZFS changes including Flash Archive support integrated into installer, cache devices, and a bunch of new properties that breakdown space usage by child dataset, snapshot, etc.

Solaris 10 Kernel PatchID Sequence - Patch Corner

Watch this page Solaris 10 Kernel PatchID Sequence - Patch Corner. It is regularly updated with the Solaris 10 kernel patch IDs as well as the sustaining patch IDs. Solaris 10 Updates 8 and 9 have just been added.

Sunday, 16 August 2009

Improvements to Solaris 10 Recommended and Sun Alert Patch Clusters released

Sun have made quite a significant revamp to the Solaris 10 Recommended and Sun Alert patch clusters.

  • Filtering out "false negatives" from the patch utility return codes.
  • The new 'installcluster' script will exit as soon as it encounters an unexpected failure.
  • The new 'installcluster' script includes context intelligence for patching operations.
  • The new 'installcluster' script provides better integration with Solaris Live Upgrade.
  • The new 'installcluster' script performs space checking prior to installing each patch, and will halt if it believes there is insufficient space to complete the installation successfully.
  • The messages and log files produced by the 'installcluster' script are clear and well structured. For example, a "failed" log is created if a patch fails to apply.
  • The 'patch_order' places patches in an optimal order for installation to avoid known issues.
  • The patches have been moved to a 'patches' sub-directory, to de-clutter the top level directory of the unzipped cluster.
  • Please see the cluster README file for further information. Customers should read the cluster README file and look at the Special Install Instructions in the patches within the cluster prior to installation.

They have also improved the QA process so that patch clusters get tested prior to release. Amazing!

Wednesday, 12 August 2009

Requiring at least one inline FormSet

Last month I posted an article about my Improved Django FormWizard, well this month I've release a simple subclass of Django's BaseInlineFormSet that demonstrates how you can require a user to enter at least one entry in an inline formset.

After updating to wadofstuff.django.forms 1.1.0 you can use the RequireOneFormSet class as the formset argument to inlineformset_factory().

When the formset is validated and it does not contain one or more entries, then a ValidationError is raised which gets put into formset.non_form_errors. You will need to check this in your templates if you wish to display the error message to your users.