YAML question, and a nose-testconfig thought

by jesse in , ,

So, I find myself using more and more YAML lately via the pyyaml package. When I was writing nose-testconfig my "preferred" format was/is YAML. Now, an interesting thing I've noticed about all of the test configurations I am developing/working with is that they have a lot of "shared" attributes (that change infrequently) and a good number of things which change all the time.

This is the perfect spot for something like a dictionary merge. If you have a test config like this:

application: capability: 1 url: http://foo subsystem: max_users: 20

For each of your configuration files, you might only override something like, max_users. For cases like this, it makes sense to load the template document (the file above) and then perform a dict.merge() after loading the second document (overriding the values in the first load) or something akin to that.

This is where my mental dilemma comes in. I could in theory, add a custom !!tag to the yaml which would take a /path/to/file.yaml and load it first, then load the second document or I could do it within nose-testconfig where you might run:

nosetests . --tc-file=myconfig.yaml --tc-rootconfig=parent.yaml

And then I would jump through the hoops (with a merge probably) within the plugin. The problem with that is that I'm worried about coupling the plugin too closely to yaml.

Now, the plugin already supports overriding multiple values: However, this doesn't scale if you have to override a lot of them.

The most common reason I've found for this so far is adding new parameters and values to the YAML files - not all child configurations need to override/define the new values, instead they could just inherit from the parent.

So, the question is - how do (would) you do this so you:

  • Don't sacrifice clarity/readability
  • Scales
  • Doesn't require the root document to be in the same location or have a hard coded path in the child document
  • Doesn't couple the loader (nose-testconfig) tightly with the file format

Right now, it's copy, paste, edit all configuration files I know about, etc.