I've been looking at the puppet testing setups (rspec etc) and while I may be missing the point, they don't appear to meet my immediate requirements.
I have a *lot* of puppet manifests that have been coded to allow for complex server sets, for example a server A may have 10+ configuration sets applied, server B 20+ etc. A complicating factor is that these configuration sets (manifests) will be sourced from different parts of the business, so I've been looking for a testing solution I can run on the CI server (bamboo) to test these assorted manifesto's before they hit and potentially break in production.
Enter Docker.
I've included this simple example at: https://github.com/agronaught/puppettest
$puppetroot/modules -- puppet modules.$puppetroot/scripts -- scripts for automating the use of puppet.
$puppetroot/docker -- docker test scripts.
The modules directory contains, oddly enough, puppet modules. For this example I've included one of the sudo modules from the puppet forge and my test manifest:
$puppetroot/modules/sudo -- sudo manifests from puppet forge
$puppetroot/modules/testbase -- the test manifests
In this example the test manifest itself is very simple:
class testbase {class { '::sudo':purge => true,config_file_replace => false,}sudo::conf { "root-notty":content => "Defaults:root !requiretty"}}
it essentially installs sudo and then adds a config to allow the root account to run sudo in a batch script.
The docker script is also very simple:
This container is based on the publicly available centos image and:# DOCKER-VERSION 1.0.1FROM centos:6.4MAINTAINER Jason Ball <jason@ball.net># Install yum repo for puppet and install itRUN rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpmRUN yum install -y puppet# Mount the puppet modules...RUN mkdir -p /tmp/puppetADD . /tmp/puppetRUN /usr/bin/puppet apply -e 'class { "::testbase": }' --verbose --modulepath=/tmp/puppet/modules
- installs puppet from the puppet repository
- mounts the manifests to be tested under /tmp/puppet
- runs the 'testbase' manifests on the container
There is a high level script to run the docker build and any test scripts if finds against the resultant image. This should allow for test scripts to be run against the resultant image to ensure configurations load, check for errors/failures, test new features, etc.
The rest should be fairly self explanatory, the only prerequisite is that docker is setup on the system thats going to run the scripts.#!/bin/bash### housekeeping##function cleanup {rm Dockerfileecho "Stopping autotest docker"docker ps -a | grep autotest | awk '{print $1}' | xargs docker rmdocker images | grep autotest | awk '{print $3}' | xargs docker rmi}#trap cleanup EXIT## Run the scripts#echo "Running Autotest"echo "Building test image"cp ./docker/autotest/Dockerfile .docker build --no-cache=true --tag="autotest" .echo "Running Test Scripts"for script in `find docker -name "test*.sh" | xargs`; doecho "Running $script..."$scriptstatus=$?if [ $status -ne 0 ]; thenecho "ERROR: Script returned non zero status: $status -- error in $script"breakfidoneexit $status
The run for the docker build:
And the test scripts:
$ ./scripts/run_autotest.sh Running AutotestBuilding test imageSending build context to Docker daemon 878.1 kBSending build context to Docker daemon Step 0 : FROM centos:6.4 ---> 539c0211cd76Step 1 : MAINTAINER Jason Ball <jason@ball.net> ---> Running in a48a2627ec29 ---> cbda1f07a5beRemoving intermediate container a48a2627ec29Step 2 : ---> Running in 1b3869f27a98 ---> 9c047488909aRemoving intermediate container 1b3869f27a98Step 3 : ---> Running in 9e04aab7d891 ---> 9b83b773effeRemoving intermediate container 9e04aab7d891Step 4 : RUN rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm ---> Running in c3c1f0fe0f6ewarning: /var/tmp/rpm-tmp.tSV15D: Header V4 RSA/SHA1 Signature, key ID 4bd6ec30: NOKEYRetrieving http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpmPreparing... ##################################################puppetlabs-release ################################################## ---> d325d0e86a97Removing intermediate container c3c1f0fe0f6eStep 5 : RUN yum install -y puppet ---> Running in 67e098975729Loaded plugins: fastestmirrorSetting up Install ProcessResolving Dependencies--> Running transaction check---> Package puppet.noarch 0:3.6.2-1.el6 will be installed--> Processing Dependency: facter >= 1:1.7.0 for package: puppet-3.6.2-1.el6.noarch--> Processing Dependency: ruby >= 1.8.7 for package: puppet-3.6.2-1.el6.noarch--> Processing Dependency: hiera >= 1.0.0 for package: puppet-3.6.2-1.el6.noarch--> Processing Dependency: ruby >= 1.8 for package: puppet-3.6.2-1.el6.noarch--> Processing Dependency: ruby-rgen >= 0.6.5 for package: puppet-3.6.2-1.el6.noarch--> Processing Dependency: ruby(selinux) for package: puppet-3.6.2-1.el6.noarch--> Processing Dependency: /usr/bin/ruby for package: puppet-3.6.2-1.el6.noarch--> Processing Dependency: rubygem-json for package: puppet-3.6.2-1.el6.noarch--> Processing Dependency: ruby-shadow for package: puppet-3.6.2-1.el6.noarch--> Processing Dependency: ruby-augeas for package: puppet-3.6.2-1.el6.noarch--> Running transaction check---> Package facter.x86_64 1:2.1.0-1.el6 will be installed--> Processing Dependency: dmidecode for package: 1:facter-2.1.0-1.el6.x86_64--> Processing Dependency: pciutils for package: 1:facter-2.1.0-1.el6.x86_64--> Processing Dependency: virt-what for package: 1:facter-2.1.0-1.el6.x86_64--> Processing Dependency: which for package: 1:facter-2.1.0-1.el6.x86_64---> Package hiera.noarch 0:1.3.4-1.el6 will be installed---> Package libselinux-ruby.x86_64 0:2.0.94-5.3.el6_4.1 will be installed--> Processing Dependency: libselinux = 2.0.94-5.3.el6_4.1 for package: libselinux-ruby-2.0.94-5.3.el6_4.1.x86_64---> Package ruby.x86_64 0:1.8.7.352-13.el6 will be installed--> Processing Dependency: ruby-libs = 1.8.7.352-13.el6 for package: ruby-1.8.7.352-13.el6.x86_64--> Processing Dependency: libruby.so.1.8()(64bit) for package: ruby-1.8.7.352-13.el6.x86_64---> Package ruby-augeas.x86_64 0:0.4.1-3.el6 will be installed--> Processing Dependency: augeas-libs >= 0.8.0 for package: ruby-augeas-0.4.1-3.el6.x86_64--> Processing Dependency: libaugeas.so.0(AUGEAS_0.10.0)(64bit) for package: ruby-augeas-0.4.1-3.el6.x86_64--> Processing Dependency: libaugeas.so.0(AUGEAS_0.1.0)(64bit) for package: ruby-augeas-0.4.1-3.el6.x86_64--> Processing Dependency: libaugeas.so.0(AUGEAS_0.11.0)(64bit) for package: ruby-augeas-0.4.1-3.el6.x86_64--> Processing Dependency: libaugeas.so.0(AUGEAS_0.8.0)(64bit) for package: ruby-augeas-0.4.1-3.el6.x86_64--> Processing Dependency: libaugeas.so.0(AUGEAS_0.12.0)(64bit) for package: ruby-augeas-0.4.1-3.el6.x86_64--> Processing Dependency: libaugeas.so.0()(64bit) for package: ruby-augeas-0.4.1-3.el6.x86_64---> Package ruby-rgen.noarch 0:0.6.5-2.el6 will be installed---> Package ruby-shadow.x86_64 1:2.2.0-2.el6 will be installed---> Package rubygem-json.x86_64 0:1.5.5-1.el6 will be installed--> Processing Dependency: rubygems for package: rubygem-json-1.5.5-1.el6.x86_64--> Running transaction check---> Package augeas-libs.x86_64 0:1.0.0-5.el6_5.1 will be installed---> Package dmidecode.x86_64 1:2.12-5.el6_5 will be installed---> Package libselinux.x86_64 0:2.0.94-5.3.el6 will be updated--> Processing Dependency: libselinux = 2.0.94-5.3.el6 for package: libselinux-utils-2.0.94-5.3.el6.x86_64---> Package libselinux.x86_64 0:2.0.94-5.3.el6_4.1 will be an update---> Package pciutils.x86_64 0:3.1.10-2.el6 will be installed--> Processing Dependency: pciutils-libs = 3.1.10-2.el6 for package: pciutils-3.1.10-2.el6.x86_64--> Processing Dependency: libpci.so.3(LIBPCI_3.1)(64bit) for package: pciutils-3.1.10-2.el6.x86_64--> Processing Dependency: libpci.so.3(LIBPCI_3.0)(64bit) for package: pciutils-3.1.10-2.el6.x86_64--> Processing Dependency: libpci.so.3()(64bit) for package: pciutils-3.1.10-2.el6.x86_64---> Package ruby-libs.x86_64 0:1.8.7.352-13.el6 will be installed--> Processing Dependency: libssl.so.10(libssl.so.10)(64bit) for package: ruby-libs-1.8.7.352-13.el6.x86_64--> Processing Dependency: libcrypto.so.10(libcrypto.so.10)(64bit) for package: ruby-libs-1.8.7.352-13.el6.x86_64--> Processing Dependency: libreadline.so.5()(64bit) for package: ruby-libs-1.8.7.352-13.el6.x86_64---> Package rubygems.noarch 0:1.3.7-5.el6 will be installed--> Processing Dependency: ruby-rdoc for package: rubygems-1.3.7-5.el6.noarch---> Package virt-what.x86_64 0:1.11-1.2.el6 will be installed---> Package which.x86_64 0:2.19-6.el6 will be installed--> Running transaction check---> Package compat-readline5.x86_64 0:5.2-17.1.el6 will be installed---> Package libselinux-utils.x86_64 0:2.0.94-5.3.el6 will be updated---> Package libselinux-utils.x86_64 0:2.0.94-5.3.el6_4.1 will be an update---> Package openssl.x86_64 0:1.0.0-27.el6_4.2 will be updated---> Package openssl.x86_64 0:1.0.1e-16.el6_5.14 will be an update--> Processing Dependency: make for package: openssl-1.0.1e-16.el6_5.14.x86_64---> Package pciutils-libs.x86_64 0:3.1.10-2.el6 will be installed---> Package ruby-rdoc.x86_64 0:1.8.7.352-13.el6 will be installed--> Processing Dependency: ruby-irb = 1.8.7.352-13.el6 for package: ruby-rdoc-1.8.7.352-13.el6.x86_64--> Running transaction check---> Package make.x86_64 1:3.81-20.el6 will be installed---> Package ruby-irb.x86_64 0:1.8.7.352-13.el6 will be installed--> Finished Dependency Resolution
Dependencies Resolved
================================================================================ Package Arch Version Repository Size================================================================================Installing: puppet noarch 3.6.2-1.el6 puppetlabs-products 1.3 MInstalling for dependencies: augeas-libs x86_64 1.0.0-5.el6_5.1 updates 309 k compat-readline5 x86_64 5.2-17.1.el6 base 130 k dmidecode x86_64 1:2.12-5.el6_5 updates 73 k facter x86_64 1:2.1.0-1.el6 puppetlabs-products 89 k hiera noarch 1.3.4-1.el6 puppetlabs-products 23 k libselinux-ruby x86_64 2.0.94-5.3.el6_4.1 base 99 k make x86_64 1:3.81-20.el6 base 389 k pciutils x86_64 3.1.10-2.el6 base 85 k pciutils-libs x86_64 3.1.10-2.el6 base 34 k ruby x86_64 1.8.7.352-13.el6 updates 534 k ruby-augeas x86_64 0.4.1-3.el6 puppetlabs-deps 21 k ruby-irb x86_64 1.8.7.352-13.el6 updates 314 k ruby-libs x86_64 1.8.7.352-13.el6 updates 1.6 M ruby-rdoc x86_64 1.8.7.352-13.el6 updates 377 k ruby-rgen noarch 0.6.5-2.el6 puppetlabs-deps 237 k ruby-shadow x86_64 1:2.2.0-2.el6 puppetlabs-deps 13 k rubygem-json x86_64 1.5.5-1.el6 puppetlabs-deps 763 k rubygems noarch 1.3.7-5.el6 base 207 k virt-what x86_64 1.11-1.2.el6 base 24 k which x86_64 2.19-6.el6 base 38 kUpdating for dependencies: libselinux x86_64 2.0.94-5.3.el6_4.1 base 108 k libselinux-utils x86_64 2.0.94-5.3.el6_4.1 base 81 k openssl x86_64 1.0.1e-16.el6_5.14 updates 1.5 M
Transaction Summary================================================================================Install 21 Package(s)Upgrade 3 Package(s)
Total download size: 8.3 MDownloading Packages:--------------------------------------------------------------------------------Total 1.4 MB/s | 8.3 MB 00:06 warning: rpmts_HdrFromFdno: Header V3 RSA/SHA1 Signature, key ID c105b9de: NOKEYRetrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6Importing GPG key 0xC105B9DE: Userid : CentOS-6 Key (CentOS 6 Official Signing Key) <centos-6-key@centos.org> Package: centos-release-6-4.el6.centos.10.x86_64 (@febootstrap/$releasever) From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6warning: rpmts_HdrFromFdno: Header V4 RSA/SHA512 Signature, key ID 4bd6ec30: NOKEYRetrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-puppetlabsImporting GPG key 0x4BD6EC30: Userid : Puppet Labs Release Key (Puppet Labs Release Key) <info@puppetlabs.com> Package: puppetlabs-release-6-10.noarch (installed) From : /etc/pki/rpm-gpg/RPM-GPG-KEY-puppetlabsRunning rpm_check_debugRunning Transaction TestTransaction Test SucceededRunning TransactionWarning: RPMDB altered outside of yum. Updating : libselinux-2.0.94-5.3.el6_4.1.x86_64 1/27 Installing : 1:dmidecode-2.12-5.el6_5.x86_64 2/27 Installing : virt-what-1.11-1.2.el6.x86_64 3/27 Updating : libselinux-utils-2.0.94-5.3.el6_4.1.x86_64 4/27 Installing : augeas-libs-1.0.0-5.el6_5.1.x86_64 5/27 Installing : libselinux-ruby-2.0.94-5.3.el6_4.1.x86_64 6/27 Installing : 1:make-3.81-20.el6.x86_64 7/27 Updating : openssl-1.0.1e-16.el6_5.14.x86_64 8/27 Installing : compat-readline5-5.2-17.1.el6.x86_64 9/27 Installing : ruby-libs-1.8.7.352-13.el6.x86_64 10/27 Installing : ruby-1.8.7.352-13.el6.x86_64 11/27 Installing : 1:ruby-shadow-2.2.0-2.el6.x86_64 12/27 Installing : ruby-rgen-0.6.5-2.el6.noarch 13/27 Installing : ruby-irb-1.8.7.352-13.el6.x86_64 14/27 Installing : ruby-rdoc-1.8.7.352-13.el6.x86_64 15/27 Installing : rubygems-1.3.7-5.el6.noarch 16/27 Installing : rubygem-json-1.5.5-1.el6.x86_64 17/27 Installing : hiera-1.3.4-1.el6.noarch 18/27 Installing : ruby-augeas-0.4.1-3.el6.x86_64 19/27 Installing : pciutils-libs-3.1.10-2.el6.x86_64 20/27 Installing : pciutils-3.1.10-2.el6.x86_64 21/27 Installing : which-2.19-6.el6.x86_64 22/27 Installing : 1:facter-2.1.0-1.el6.x86_64 23/27 Installing : puppet-3.6.2-1.el6.noarch 24/27 Cleanup : libselinux-utils-2.0.94-5.3.el6.x86_64 25/27 Cleanup : libselinux-2.0.94-5.3.el6.x86_64 26/27 Cleanup : openssl-1.0.0-27.el6_4.2.x86_64 27/27 Verifying : ruby-libs-1.8.7.352-13.el6.x86_64 1/27 Verifying : which-2.19-6.el6.x86_64 2/27 Verifying : libselinux-2.0.94-5.3.el6_4.1.x86_64 3/27 Verifying : 1:ruby-shadow-2.2.0-2.el6.x86_64 4/27 Verifying : libselinux-utils-2.0.94-5.3.el6_4.1.x86_64 5/27 Verifying : puppet-3.6.2-1.el6.noarch 6/27 Verifying : pciutils-libs-3.1.10-2.el6.x86_64 7/27 Verifying : ruby-augeas-0.4.1-3.el6.x86_64 8/27 Verifying : virt-what-1.11-1.2.el6.x86_64 9/27 Verifying : compat-readline5-5.2-17.1.el6.x86_64 10/27 Verifying : ruby-rdoc-1.8.7.352-13.el6.x86_64 11/27 Verifying : 1:facter-2.1.0-1.el6.x86_64 12/27 Verifying : pciutils-3.1.10-2.el6.x86_64 13/27 Verifying : 1:make-3.81-20.el6.x86_64 14/27 Verifying : rubygems-1.3.7-5.el6.noarch 15/27 Verifying : 1:dmidecode-2.12-5.el6_5.x86_64 16/27 Verifying : ruby-rgen-0.6.5-2.el6.noarch 17/27 Verifying : ruby-irb-1.8.7.352-13.el6.x86_64 18/27 Verifying : rubygem-json-1.5.5-1.el6.x86_64 19/27 Verifying : openssl-1.0.1e-16.el6_5.14.x86_64 20/27 Verifying : augeas-libs-1.0.0-5.el6_5.1.x86_64 21/27 Verifying : libselinux-ruby-2.0.94-5.3.el6_4.1.x86_64 22/27 Verifying : ruby-1.8.7.352-13.el6.x86_64 23/27 Verifying : hiera-1.3.4-1.el6.noarch 24/27 Verifying : libselinux-2.0.94-5.3.el6.x86_64 25/27 Verifying : libselinux-utils-2.0.94-5.3.el6.x86_64 26/27 Verifying : openssl-1.0.0-27.el6_4.2.x86_64 27/27
Installed: puppet.noarch 0:3.6.2-1.el6
Dependency Installed: augeas-libs.x86_64 0:1.0.0-5.el6_5.1 compat-readline5.x86_64 0:5.2-17.1.el6 dmidecode.x86_64 1:2.12-5.el6_5 facter.x86_64 1:2.1.0-1.el6 hiera.noarch 0:1.3.4-1.el6 libselinux-ruby.x86_64 0:2.0.94-5.3.el6_4.1 make.x86_64 1:3.81-20.el6 pciutils.x86_64 0:3.1.10-2.el6 pciutils-libs.x86_64 0:3.1.10-2.el6 ruby.x86_64 0:1.8.7.352-13.el6 ruby-augeas.x86_64 0:0.4.1-3.el6 ruby-irb.x86_64 0:1.8.7.352-13.el6 ruby-libs.x86_64 0:1.8.7.352-13.el6 ruby-rdoc.x86_64 0:1.8.7.352-13.el6 ruby-rgen.noarch 0:0.6.5-2.el6 ruby-shadow.x86_64 1:2.2.0-2.el6 rubygem-json.x86_64 0:1.5.5-1.el6 rubygems.noarch 0:1.3.7-5.el6 virt-what.x86_64 0:1.11-1.2.el6 which.x86_64 0:2.19-6.el6
Dependency Updated: libselinux.x86_64 0:2.0.94-5.3.el6_4.1 libselinux-utils.x86_64 0:2.0.94-5.3.el6_4.1 openssl.x86_64 0:1.0.1e-16.el6_5.14
Complete! ---> 3929c2fd7335Removing intermediate container 67e098975729Step 6 : RUN mkdir -p /tmp/puppet ---> Running in 1862953608fa ---> 4eaf5c96f9a3Removing intermediate container 1862953608faStep 7 : ADD . /tmp/puppet ---> c47c7a0a6743Removing intermediate container 0e2be2ebd336Step 8 : RUN /usr/bin/puppet apply -e 'class { "::testbase": }' --verbose --modulepath=/tmp/puppet/modules ---> Running in e728af515dffInfo: Loading facts in /tmp/puppet/modules/stdlib/lib/facter/puppet_vardir.rbInfo: Loading facts in /tmp/puppet/modules/stdlib/lib/facter/root_home.rbInfo: Loading facts in /tmp/puppet/modules/stdlib/lib/facter/pe_version.rbInfo: Loading facts in /tmp/puppet/modules/stdlib/lib/facter/facter_dot_d.rbWarning: Config file /etc/puppet/hiera.yaml not found, using Hiera defaultsNotice:secondsWarning: The package type's allow_virtual parameter will be changing its default value from false to true in a future release. If you do not want to allow virtual packages, please explicitly set allow_virtual to false. (at /usr/lib/ruby/site_ruby/1.8/puppet/type.rb:816:in `set_default')Info: Applying configuration version '1404712257'Notice: /Stage[main]/Sudo::Package/Package[sudo]/ensure: createdNotice: /Stage[main]/Sudo/File[/etc/sudoers.d/]/mode: mode changed '0750' to '0550'Notice: /Stage[main]/Testbase/Sudo::Conf[root-notty]/File[10_root-notty]/ensure: createdInfo: /Stage[main]/Testbase/Sudo::Conf[root-notty]/File[10_root-notty]: Scheduling refresh of Exec[sudo-syntax-check for file /etc/sudoers.d/10_root-notty]Notice: /Stage[main]/Testbase/Sudo::Conf[root-notty]/Exec[sudo-syntax-check for file /etc/sudoers.d/10_root-notty]: Triggered 'refresh' from 1 eventsInfo: Creating state file /var/lib/puppet/state/state.yamlNotice: Finished catalog run in 2.17 seconds ---> 0abda7c7aed6Removing intermediate container e728af515dffSuccessfully built 0abda7c7aed6
Successfully built 0abda7c7aed6
Running Test Scripts
Running docker/autotest/test_sudo.sh...
Running docker/autotest/test_sudo_bad.sh...
sudo: unknown user: baduser
sudo: unable to initialize policy plugin
ERROR: Script returned non zero status: 1 -- error in docker/autotest/test_sudo_bad.sh
Nice and simple and easily automated in Bamboo. In my case I have an auto test dock for each logical group of manifests representing a production environment and this approach effectively allows me to perform a sociability test on scripts prior to hitting production.
The test scripts can be coded using whatever tool set your happiest with.
No comments:
Post a Comment