Back to Articles
puppet May 12, 2020

Scaling Puppet Community to Thousands of Nodes

JS
Jerald Sheets

Not everyone runs Puppet Enterprise. The open-source Puppet Community edition is a powerful platform in its own right, and with the right architecture, it can scale to manage thousands of nodes reliably.

Architecture Overview

The key to scaling Puppet Community is separating the Certificate Authority from your compilation infrastructure and load-balancing across multiple compilers:

  • CA Master — Handles certificate signing and serves as the primary Puppet server
  • Compilers — Additional PuppetServer instances behind a load balancer
  • HAProxy — Load balancer distributing agent requests across compilers
  • PuppetDB — Centralized data store (can also be clustered)

Load Balancer Configuration

We use HAProxy to distribute agent check-ins across compilers. The key is configuring Subject Alternative Names (SANs) in your compiler certificates so agents trust any compiler they connect to:


/etc/puppetlabs/puppet/puppet.conf on compilers

[main]
dns_alt_names = puppet,puppet.example.com,compiler01.example.com

[master]
ca_server = ca.example.com
ca = false

CA Configuration

On the CA master, configure it to be the sole certificate authority:


/etc/puppetlabs/puppetserver/conf.d/ca.conf

certificate-authority: {
allow-subject-alt-names: true
}

Agent Bootstrap

When bootstrapping agents, point them at the load balancer VIP but configure them to get certificates from the CA master:


puppet agent -t --server puppet.example.com --ca_server ca.example.com

This architecture has been tested on AWS but is platform-agnostic. With proper tuning (JVM heap sizes, max-active-instances), three to four compilers behind HAProxy can comfortably manage 5,000+ nodes with 30-minute run intervals.

Related Articles

Need Help With Your Infrastructure?

We can help you implement the patterns and practices discussed in our articles.

Get in Touch