Puppet

Warning

Use at your own risk!!! Commands, procedures etc. described in this page worked for me. That does not mean they will automatically work for you.

Puppet is a tool much like CFEngine and others allowing you the manage a set of systems by defining their desired state. Puppet allows the user to use a Domain Specific Language to define the state in which resources on a machine should be. With this the user defines the state of the machine. This makes the tool largely idempotent meaning the tool must be runnable any number of times with the machine being left in the same state the 100th time as it was in the 1st state.

This state includes but is not limited to:

  • users
  • groups
  • software
  • services
  • targets

To manage nodes Puppet uses a client/server model. The server is called master and the client is a node. The node runs an agent which periodically (30m) starts a Puppet run:

Puppet Manifest

Puppet Domain-specific Language (DSL)

Todo

description of DSL

Example:

resource_type { 'title':
  attribute => value,
  ...
}

Simple examples

Hello World

A simple puppet manifest.

notify { 'Hello World': }

Validate code with puppet parser validate:

# puppet parser validate helloworld.pp

# echo $?
0

Validate code with puppet apply:

# puppet apply --noop helloworld.pp
Notice: Compiled catalog for loki.doubtfull.snd in environment production in 0.01 seconds
Notice: /Stage[main]/Main/Notify[Hello World]/message: current_value absent, should be Hello World (noop)
Notice: Class[Main]: Would have triggered 'refresh' from 1 events
Notice: Stage[main]: Would have triggered 'refresh' from 1 events
Notice: Finished catalog run in 0.13 seconds

Execute the code with puppet apply:

# puppet apply helloworld.pp
Notice: Compiled catalog for loki.doubtfull.snd in environment production in 0.01 seconds
Notice: Hello World
Notice: /Stage[main]/Main/Notify[Hello World]/message: defined 'message' as 'Hello World'
Notice: Finished catalog run in 0.05 seconds

Hello World 2

“Hello World” using the file resource type:

file { '/tmp/helloworld.txt':
  ensure  => 'file',
  content => 'Hello World',
  mode    => '664',
}

Validate code with puppet parser validate:

# puppet parser validate helloworld2.pp

# echo $?
0

Validate code with puppet apply:

# puppet apply --noop helloworld2.pp
Notice: Compiled catalog for loki.doubtfull.snd in environment production in 0.08 seconds
Notice: /Stage[main]/Main/File[/tmp/helloworld.txt]/ensure: current_value absent, should be file (noop)
Notice: Class[Main]: Would have triggered 'refresh' from 1 events
Notice: Stage[main]: Would have triggered 'refresh' from 1 events
Notice: Finished catalog run in 0.03 seconds

Execute the code with puppet apply:

# puppet apply helloworld2.pp
Notice: Compiled catalog for loki.doubtfull.snd in environment production in 0.09 seconds
Notice: /Stage[main]/Main/File[/tmp/helloworld.txt]/ensure: defined content as '{md5}b10a8db164e0754105b7a99be72e3fe5'
Notice: Finished catalog run in 0.09 seconds

Check the results:

# cat /tmp/helloworld.txt
Hello World#

Warning

notice the lack of newline.

Fix:

--- helloworld2.pp.orig 2017-07-19 14:17:03.696247324 +0200
+++ helloworld2.pp      2017-07-19 14:17:14.588234830 +0200
@@ -1,5 +1,5 @@
 file { '/tmp/helloworld.txt':
   ensure  => 'file',
-  content => 'Hello World',
+  content => "Hello World\n",
   mode    => '664',
 }

Re-execute the code with puppet apply:

# puppet apply helloworld2.pp
Notice: Compiled catalog for loki.doubtfull.snd in environment production in 0.09 seconds
Notice: /Stage[main]/Main/File[/tmp/helloworld.txt]/content: content changed '{md5}b10a8db164e0754105b7a99be72e3fe5' to '{md5}e59ff97941044f85df5297e1c302d260'
Notice: Finished catalog run in 0.19 seconds

Check the results again:

# cat /tmp/helloworld.txt
Hello World
#

No sense of order

Without coercion the puppet manifests are executed in seemingly random order.

Code:

notify { 'Hello World 1': }
notify { 'Hello World 2': }
notify { 'Hello World 3': }
notify { 'Hello World 4': }
notify { 'Hello World A': }

Execute the code with puppet apply:

# puppet apply nosenseoforder.pp
Notice: Compiled catalog for loki.doubtfull.snd in environment production in 0.01 seconds
Notice: Hello World 1
Notice: /Stage[main]/Main/Notify[Hello World 1]/message: defined 'message' as 'Hello World 1'
Notice: Hello World 3
Notice: /Stage[main]/Main/Notify[Hello World 3]/message: defined 'message' as 'Hello World 3'
Notice: Hello World A
Notice: /Stage[main]/Main/Notify[Hello World A]/message: defined 'message' as 'Hello World A'
Notice: Hello World 4
Notice: /Stage[main]/Main/Notify[Hello World 4]/message: defined 'message' as 'Hello World 4'
Notice: Hello World 2
Notice: /Stage[main]/Main/Notify[Hello World 2]/message: defined 'message' as 'Hello World 2'
Notice: Finished catalog run in 0.09 seconds

This has some impact if you for example have a package type and a service type in the same manifest!

Bring order to chaos

Puppet assumes that most resources are not related to each other and will manage the resources in whatever order it deems most efficient.

Puppet uses four metaparameters to establish relationships. Each of them can be set as an attribute in any resource.

attribute Causes a resource to be applied
before before the target resource.
require after the target resource.
notify before the target resource. The target resource will refresh if the notifying resource changes.
subscribe after the target resource. The subscribing resource will refresh if the target resource changes.

Form:

Resource_type[ resource_title ]

Note

capitalize the resource

The two examples below create the same ordering relationship:

package { 'openssh-server':
  ensure => present,
  before => File['/etc/ssh/sshd_config'],
}

file { '/etc/ssh/sshd_config':
  ensure  => file,
  mode    => 600,
  source  => 'puppet:///modules/sshd/sshd_config',
  require => Package['openssh-server'],
}

The two examples below create the same notification relationship:

file { '/etc/ssh/sshd_config':
  ensure => file,
  mode   => 600,
  source => 'puppet:///modules/sshd/sshd_config',
  notify => Service['sshd'],
}

service { 'sshd':
  ensure    => running,
  enable    => true,
  subscribe => File['/etc/ssh/sshd_config'],
}

Chaining Arrows

You can create relationships between two resources or groups of resources using the -> and ~> operators.

-> ordering arrow Causes the resource on the left to be applied before the resource on the right. Written with a hyphen and a greater-than sign.
~> notification arrow Causes the resource on the left to be applied first, and sends a refresh event to the resource on the right if the left resource changes. Written with a tilde and a greater-than sign.

Example 1:

# ntp.conf is applied first, and will notify the ntpd service if it changes:
File['/etc/ntp.conf'] ~> Service['ntpd']

Example 2:

# Anchor this as per #8040 - this ensures that classes won't float off and
# mess everything up.  You can read about this at:
# http://docs.puppetlabs.com/puppet/2.7/reference/lang_containment.html#known-issues
anchor { 'ntp::begin': } ->
class { '::ntp::install': } ->
class { '::ntp::config': } ~>
class { '::ntp::service': } ->
anchor { 'ntp::end': }

Note

anchor { ‘ntp::begin’: } and anchor { ‘ntp::end’: } are just markers.

Facts

Todo

Add Puppet run

In the early stages of a Puppet run a node will collect system infomation with Facter and present them as pre-set variables also known as facts.

In the manifests they are simply used as variables. Listing of facter-example.pp:

if $::osfamily == 'redhat' {
  $fpath = '/tmp/example'
}

Facts also appear in a $facts hash. Example:

if $facts['os']['family'] == 'redhat' {
  $fpath = '/tmp/example'
}

These facts can be queried using the facter command and extended by adding targets to one of these directories:

  • /etc/facter/facts.d
  • /etc/puppetlabs/facter/facts.d

Conventions for new facts

There are four types of extension accepted by Facter:

A plain text variant (extension .txt):

foo=bar

Note

NO quotes!!!!

A yaml variant (extension .yaml):

---
foo: 'bar'

A json variant (extension .json):

{
  "foo": "bar"
}

And an executeable variant, output same as txt variant.

Variables

Variables are defines much like any other, however Puppet only allows a given variable to be assigned once within a given scope.

Example:

$boolean_var = true
$number_var = 5
$string_var = "Custom message.\n"
$array_var = [ 'a', 'b', 'c' ]
$hash_var = {
 'first'  => 'primary',
 'second' => 'secondary',
 'third'  => 'tertiary',
}

$:: prefix for facter variables

Example scope-example.pp:

$myvar = "Top scope value"
node 'www1.example.com' {
  $myvar = "Node scope value"
  notice( "from www1: $myvar" )
  include myclass
}
node 'db1.example.com' {
  notice( "from db1: $myvar" )
  include myclass
}
class myclass {
  $myvar = "Local scope value"
  notice( "from myclass: $myvar" )
}

Results on www1.example.com:

Notice: Scope(Node[www1.example.com]): from www1: Node scope value
Notice: Scope(Class[Myclass]): from myclass: Local scope value
Notice: Compiled catalog for www1.example.com in environment production in 0.01 seconds
Notice: Finished catalog run in 0.04 seconds

Results in db1.example.com:

Notice: Scope(Node[db1.example.com]): from db1: Top scope value
Notice: Scope(Class[Myclass]): from myclass: Local scope value
Notice: Compiled catalog for db1.example.com in environment production in 0.01 seconds
Notice: Finished catalog run in 0.20 seconds

Conditionals

if / unless

if $::is_virtual {
  file { '/tmp/virtual.txt':
    ensure  => 'file',
    content => "This is a virtual system. Disk is ${disk}.\n",
  }
}
if $::osfamily == 'RedHat' or $::osfamily == 'CentOS' {
  notify { "This is a Red Hat family system." : }
}
elsif $::osfamily == 'Solaris' {
  notify { "This is a Solaris system." : }
}
else {
  notify { "Not sure what OS family this system is." : }
}

case

case $::osfamily {
  'RedHat', 'CentOS': {
    notify { "This is a Red Hat family system." : }
  }
  'Solaris': {
    notify { "This is a Solaris system." : }
  }
  default: {
    notify { "Not sure what OS family this system is." : }
  }
}

selector

$message = $operatingsystem ? {
  /Linux/      => 'Powered by Linux.',
  'Solaris'    => 'Powered by Solaris.',
  default      => 'Welcome.',
}

Implementing Regular Expressions

Puppet regexps are like ruby regexp:

if $hostname =~ /^www(\d+)\./ {
  notice("Welcome to web server number $1")
}

if $hostname !~ /^www/ {
  notice("Dunno where you are")
}
Metacharacter Description
. character
[ ] character within the brackets
[^ ] character not contained within the brackets
\w alphanumeric character
\W nonalphanumeric character
\d digit character
\D nondigit character
\s space character
\S nonspace character
Modifier Description
* zero or more times
? zero or one time
+ one or more times
{x} exactly x times
{x,} x or more times
{,y} y or less times
{x,y} at least x but not more than y times
Metacharacter Description
^ Matches the beginning of the line
$ Matches the end of the line

Implementing Classes

minimal:

class class_name {
  resource definitions...
}

parameters:

class class_name ($param = 'value') {
  resource definitions...
}

Note

Puppet modules should define Puppet classes, one per manifest, and the main class defined in manifests/init.pp should have the same name as the module.

Puppet Modules

Puppet Modules are comparable to ansible’s roles. You can roll you own or download the

Todo

FIX THIS

Creating Modules

In short:

generate

To create a new module the puppet module generate <author>-<module> command is executed. Take care to adhere to this since when including the module only the module’s name is mentioned. So a module that is generated like this:

puppet module generate foo-bar

Will be included like this:

include bar

editing the content

When the module is generated take care to look at the metadata.json the reference to puppetlabs-stdlib is often not needed and would create useless dependencies.

Note

Update the version in metadata.json when making changes!!

testing

The skeleton contains a test/init.pp manifest. This is a manifest that references the created module making it possible to test it.

building

To build the module puppet module build this generates a tar file with the name in pkg/<author>-<modulename>-<version>.tar.gz

installing

Install the module by executing:

puppet module install <name>|<forgename>

puppet module install

http://docs.puppetlabs.com/puppet/3.6/reference/modules_fundamentals.html

Files

Files

source => 'puppet:///modules/MODULENAME/FILENAME',

Note

/etc/puppet/modules/MODULENAME/files/FILENAME

file { "/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-${os_maj_release}":
  ensure => present,
  owner  => 'root',
  group  => 'root',
  mode   => '0644',
  source => "puppet:///modules/epel/RPM-GPG-KEY-EPEL-${os_maj_release}",
}

Note

/etc/puppet/modules/epel/files/RPM-GPG-KEY-EPEL-7

Templates

content => template('MODULENAME/FILENAME.erb'),
file { 'mysql-config-file':
  path                    => $mysql::server::config_file,
  content                 => template('mysql/my.cnf.erb'),
  mode                    => '0644',
  selinux_ignore_defaults => true,
}

Note

/etc/puppet/modules/mysql/templates/my.cnf.erb

<%# COMMENT %>
<%= EXPRESSION %>
<%= @ipaddress %>   <%= @fqdn %>

Test code:

validate_erb() {
  erb -P -x -T - $1 | ruby -c
}

Implementing Puppet

Implementing a Puppet Master

Commands:

yum install -y puppet-server
systemctl start puppetmaster.service
systemctl enable puppetmaster.service
firewall-cmd --permanent --add-port=8140/tcp
firewall-cmd --reload

File that contains nodes: /etc/puppet/manifests/site.pp.

Implementing a Puppet Client

Client commands:

yum install -y puppet

echo "server = <puppetmaster fqdn>" >> /etc/puppet/puppet.conf

systemctl start puppet.service
systemctl enable puppet.service

Commands on the master:

puppet cert list
puppet cert sign node.example.com.
[agent]
runinterval = 15m
puppet agent --configprint runinterval

Commands

puppet agent

Print the run interval of an agent:

# puppet agent --configprint runinterval
1800

puppet resource

The puppet resource command uses the Puppet RAL to directly interact with the system. You can use it list or generate example code for a type.

List resource types http://docs.puppetlabs.com/references/3.6.0/type.html:

# puppet resource --type
augeas
computer
cron
exec
file
filebucket
group
host
interface
k5login
macauthorization
mailalias
maillist
mcx
mount
...
zfs
zone
zpool
# puppet resource package foo
package { 'foo':
  ensure => 'absent',
}

# puppet resource package bash
package { 'bash':
  ensure => '4.2.46-21.el7_3',
}

# puppet resource file /etc/passwd
file { '/etc/passwd':
  ensure  => 'file',
  content => '{md5}25d97bb55f526f3db405e66124f6f926',
  ctime   => '2017-07-18 08:56:00 +0200',
  group   => '0',
  mode    => '644',
  mtime   => '2017-07-18 08:56:00 +0200',
  owner   => '0',
  type    => 'file',
}

# puppet resource service sshd
service { 'sshd':
  ensure => 'running',
  enable => 'true',
}

puppet doc

Generate a reference for all Puppet types.:

# puppet doc | less

Note

the output is long so use less

puppet describe

Prints help about Puppet resource types, providers, and metaparameters.:

# puppet describe yumrepo

puppet parser validate

Validate the syntax of one or more Puppet manifests.:

# puppet parser validate lala.pp
Error: Could not parse for environment production: Syntax error at '->'; expected '}' at /home/username/lala.pp:4

puppet apply

Apply a standalone Puppet manifest to the local system.:

# puppet apply --noop lala.pp

Note

this is a dry-run used for syntax testing.

# puppet apply lala.pp

puppet module generate

Generate boilerplate for a new module:

# puppet module generate author-modulename
We need to create a metadata.json file for this module.  Please answer the
following questions; if the question is not applicable to this module, feel free
to leave it blank.

Puppet uses Semantic Versioning (semver.org) to version modules.
What version is this module?  [0.1.0]
-->

Who wrote this module?  [author]
-->

What license does this module code fall under?  [Apache 2.0]
-->

How would you describe this module in a single sentence?
-->

Where is this module's source code repository?
-->

Where can others go to learn more about this module?
-->

Where can others go to file issues about this module?
-->

----------------------------------------
{
  "name": "author-modulename",
  "version": "0.1.0",
  "author": "author",
  "summary": null,
  "license": "Apache 2.0",
  "source": "",
  "project_page": null,
  "issues_url": null,
  "dependencies": [
    {
      "name": "puppetlabs-stdlib",
      "version_range": ">= 1.0.0"
    }
  ]
}
----------------------------------------

About to generate this metadata; continue? [n/Y]
-->

Notice: Generating module at /home/username/examples/author-modulename...
Notice: Populating ERB templates...
Finished; module generated in author-modulename.
author-modulename/Rakefile
author-modulename/manifests
author-modulename/manifests/init.pp
author-modulename/spec
author-modulename/spec/classes
author-modulename/spec/classes/init_spec.rb
author-modulename/spec/spec_helper.rb
author-modulename/tests
author-modulename/tests/init.pp
author-modulename/README.md
author-modulename/metadata.json

Or shorter:

# puppet module generate --skip-interview foo-bar

Notice: Generating module at /home/username/examples/foo-bar...
Notice: Populating ERB templates...
Finished; module generated in foo-bar.
foo-bar/Rakefile
foo-bar/manifests
foo-bar/manifests/init.pp
foo-bar/spec
foo-bar/spec/classes
foo-bar/spec/classes/init_spec.rb
foo-bar/spec/spec_helper.rb
foo-bar/tests
foo-bar/tests/init.pp
foo-bar/README.md
foo-bar/metadata.json

puppet module build

# puppet module build author-modulename
Notice: Building /home/username/examples/author-modulename for release
Module built: /home/username/examples/author-modulename/pkg/author-modulename-0.1.0.tar.gz

puppet module install

# sudo puppet module install /home/username/examples/author-modulename/pkg/author-modulename-0.1.0.tar.gz
Notice: Preparing to install into /etc/puppet/modules ...
Notice: Downloading from https://forgeapi.puppetlabs.com ...
Notice: Installing -- do not interrupt ...
/etc/puppet/modules
└─┬ author-modulename (v0.1.0)
  └── puppetlabs-stdlib (v4.5.1) [/usr/share/puppet/modules]

puppet module install from the Puppet Forge

# sudo puppet module install puppetlabs-mysql --version 3.11.0
Notice: Preparing to install into /etc/puppet/modules ...
Notice: Downloading from https://forgeapi.puppetlabs.com ...
Notice: Installing -- do not interrupt ...
/etc/puppet/modules
└─┬ puppetlabs-mysql (v3.11.0)
  ├── puppet-staging (v2.2.0)
  └── puppetlabs-stdlib (v4.17.1)

facter

Collect and display facts about the system.:

# facter |egrep "^(architecture|kernel|hardwaremodel)\ "
architecture => x86_64
hardwaremodel => x86_64
kernel => Linux

Or more specific:

# facter  architecture kernel hardwaremodel
architecture => x86_64
hardwaremodel => x86_64
kernel => Linux

Warning

A regular user does not get the same output as root!

hiera

Get the hiera configured domain name:

puppet lookup domain  --merge deep --environment production --explain --node <nodename>

Glossary

agent
Retrieves the client configuration from the puppet master and applies it to the local host.
attribute
Attributes specify the desired state of a given configuration resource. For example the package resource would have an ensure attribute, whose value could be present, latest, absent, or a version number
catalog
A compilation of all the resources that the puppet agent applies to a given node, as well as the relationships between those resources.
class
A class is a collection of related resources that, once defined, can be declared as a single unit. For example, a class can contain all of the resources (such as files, settings, modules, and scripts) needed to configure the Apache webserver on a host. Classes can also declare other classes. For more information, see the Classes page in the Puppet language reference.
fact

A piece of information about a node, such as its hostname, IP address, and operating system, is a fact.

Facter reads facts from a node and makes them available to Puppet. You can extend Facter with custom facts, which can expose site-specific details of your systems to your Puppet manifests. For more information, see the Custom Facts documentation.

facter
Facter is Puppet’s system inventory tool. Facter reads facts about a node, such as its hostname, IP address, and operating system, and makes them available to Puppet.
idempotent

Idempotence refers to the concept of doing something multiple times with the same outcome. Puppet resources are idempotent, since they describe a desired final state rather than a series of steps to follow.

The most prominent exception among Puppet resources is the exec resource type, which should be idempotent but rely on the user to design them accordingly.

manifest

A manifest file contains code written in the Puppet language and is named with the .pp file extension. The Puppet code in a manifest can:

  • Declare resources and classes
  • Set variables
  • Evaluate functions
  • Define classes, defined types, functions, and nodes

Most manifests are contained in modules. Every manifest in a module should define a single class, defined type, or function.

The Puppet master service reads an environment’s main manifest. This manifest usually defines nodes, so that each managed agent receives a unique catalog.

See: The main manifest for more information

manifest ordering
Puppet uses manifest ordering to apply unrelated resources in the order that they’re declared in their manifests. For related resources, or for more information about manifest ordering, see the Relationships and Ordering page in the Puppet language reference.
master

In a standard Puppet client-server deployment, the server is known as the Puppet master. The Puppet master compiles and serves configuration catalogs on demand to Puppet agents on client nodes.

The Puppet master provides catalogs to agents using an HTTP server. It can run as a standalone daemon with a built-in web server, or as part of Puppet Server. (We don’t recommend using the built-in daemon when managing more than 10 nodes.)

For more information, see the Overview of Puppet’s Architecture page in the Puppet documentation.

module
A collection of classes, resource types, files, functions and templates, organized around a particular purpose is a module. For example, a module can configure an Apache webserver instance or Rails application. There are many modules available for download in the Puppet Forge. For more information, see the Module Fundamentals and Installing Modules pages in the Puppet documentation.
node
A node is a device managed by Puppet.
node definition
A node definition is a collection of classes, resources, and variables in a manifest that are only applied to a certain agent node. Node definitions begin with the node keyword, and can match a node by full name or regular expression.
puppet language
You write Puppet code in the Puppet language. Puppet language files are called manifests and are named with the .pp extension. The Puppet master compiles this Puppet code into a catalog during a Puppet run. For a summary of the Puppet language, see the Basics page in its reference.
relationship
A rule that sets the order in which resources should be managed creates a relationship between those resources. For more information, see the Relationships and Ordering page in the Puppet language reference.
resource
A resource is a unit of configuration whose state can be managed by Puppet. Every resource has a type (such as file, service, or user), a title, and one or more attributes with specified values.
ral
resource abstraction layer
Refers to the components of Puppet that interact with the system. The RAL provides an abstract concept of something you can manage, and it defines concrete ways of managing things.
subclass
A subclass is a class that inherits from another class. See inheritance.
template
A template is a partial document that is filled in with data from variables. Puppet can use Embedded Puppet (EPP) written in the Puppet language, or Embedded Ruby (ERB) templates written in Ruby, to generate configuration files tailored to an individual system. For more information, see the Using Templates page in the Puppet language reference.
title

A title is the unique identifier of a resource or class in a given Puppet catalog.

  • In a class, the title is simply the class’s name.

  • In a resource declaration, the title is the part after the first curly brace and before the colon; in the example below, the title is /etc/passwd:

    file  { '/etc/passwd':
      owner => 'root',
      group => 'root',
    }
    
  • In native resource types, the name or namevar uses the title as its default value if you don’t explicitly specify a name.

  • In a defined type or a class, the title is available for use throughout the definition as the $title variable.

Unlike the name or namevar, a resource’s title doesn’t need to map to any attribute of the target system; it is only a referent. You can give a resource a single title even if its name must vary across different kinds of systems, like a configuration file whose location differs on Solaris.

TODO

Todo

description of DSL

(The original entry is located in /builds/homepage/book_puppet/sphinxdoc/index.rst, line 67.)

Todo

Add Puppet run

(The original entry is located in /builds/homepage/book_puppet/sphinxdoc/index.rst, line 330.)

Todo

FIX THIS

(The original entry is located in /builds/homepage/book_puppet/sphinxdoc/index.rst, line 614.)

Todo

add body

(The original entry is located in /builds/homepage/book_puppet/sphinxdoc/index.rst, line 680.)