In this post of our Cloud consulting and DevOps series, we detail exactly how why to use Chef for infrastructure management! Specifically, we’ll provide a step-by-step guide to setting up Chef automation for server configuration through script updates automated across multiple servers.
Chef is a configuration management tool in the same category as Puppet and Ansible used in DevOps automation. Once considered a cutting edge DevOps tool, Chef and its peers are now mainstream and used by most enterprises and DevOps teams working on cloud applications.
As software, especially cloud-native software, becomes more sophisticated, infrastructure automation has become a standard must-have. They provide a centralised DevOps environment for the management of all systems, networks and devices, both in the cloud and on-premises, allowing DevOps engineers to reliably configure, update and deploy infrastructure and applications more efficiently.
Agile & DevOps teams and consultants
Supercharge your next cloud development project!
Using manual processes in infrastructure management is both error prone and, crucially for today’s software, doesn’t scale well. It is difficult to maintain consistency when using custom scripts and regular maintenance requirements like updating operating systems across different devices can take days or weeks.
Manual processes also don’t work for CI/CD pipelines which require consistency between development, test and production environments across all systems.
Moving to infrastructure as code allows for CI/CD, version control and automated testing across infrastructure and applications. Infrastructure as code doesn’t mean there are no physical servers involved but rather that scripts provision and manage infrastructure consistently across environments.
Another advantage of defining and documenting things within code is that the consistency keeps everyone on the same page, greatly improving collaboration. And it adds visibility and stability, which is crucial to maintaining quality and performance in large, agile software systems.
Configuration management platforms like Chef make it possible to treat infrastructure as code. Chef uses prewritten code, which it calls “recipes” to define infrastructure, system dependencies and security policies.
The platform also automates the detection and repair of inconsistencies in configuration across environments and devices. And you can also use Chef to spin up VMs, cloud instances and containers or automate security updates.
In short, Chef cuts out manual infrastructure management processes that suck up time and increase the risk of errors but add nothing to the end user’s experience, except keeping an application running. Automating those tasks allow software development teams to focus on the things that add genuine user value.
Let’s look at how we use Chef automation to configure servers.
We use self-compiled scripts for content updates (for real-time debugging). Using the server-client principle, that is the initializer script and the executable script on the servers. But since there are a lot of servers, making changes to a script on each server will take ages.
Let’s get started
1. We already have the OS with updated packages.
2. We have a FQDN name.
3. Curl and GNU Wget should already be installed.
1. Go to https://www.opscode.com/chef/install
2. Click the tab “Chef Server”
3. Select the operating system and the architecture
4. Select Chef version
5. Install the package
rpm -ivh https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-server-11.1.3-1.el6.x86_64.rpm
Configure Chef server 11.*. Run command
# chef-server-ctl reconfigure
It will install and set up the required packages itself
Then we stop the webserver, if any, and run the verification script:
# chef-server-ctl test
After the test, go to:
# https://FQDN-OR-IP-OF-CHEF-SERVER
Note: Default UserName/Password: admin/p@ssw0rd1
Run command (on Linux like)
# curl -L https://www.opscode.com/chef/install.sh | bash
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
101 6790 101 6790 0 0 3826 0 0:00:01 0:00:01 —:—:— 12190
Downloading Chef for el…
Installing Chef
warning: /tmp/tmp.KnyQTnqz/chef-.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
Preparing… ########################################### [100%] 1:chef ########################################### [100%] Thank you for installing Chef!
After the installation is complete, verify the client is installed
# chef-client -v
Chef: 11.6.0
Create a Chef directory
Copy Cert Keys from Chef Server to Workstation User Folder
$ mkdir ~/.chef $ scp root@chef-server:/etc/chef-server/admin.pem ~/.chef $ scp root@chef-server:/etc/chef-server/chef-validator.pem ~/.chef
Now, configure the client using the “knife” command
$ knife configure -i
Overwrite /root/.chef/knife.rb? (Y/N) y
Please enter the chef server URL: [https://test.example.com:443] https://chef-server.example.com:443/
Please enter a name for the new user: [root] knife-user1
Please enter the existing admin name: [admin] Enter
Please enter the location of the existing admin’s private key: [/etc/chef-server/admin.pem] ~/.chef/admin.pem
Please enter the validation clientname: [chef-validator] Please enter the location of the validation key: [/etc/chef-server/chef-validator.pem] ~/.chef/chef-validator.pem
Please enter the path to a chef repository (or leave blank):
Creating initial API user…
Please enter a password for the new user: Created user[knife-user1] Configuration file written to /root/.chef/knife.rb
$ cat ~/.chef/knife.rb
log_level :info
log_location STDOUT
node_name ‘knife-user1’
client_key ‘/root/.chef/knife-user1.pem’
validation_client_name ‘chef-validator’
validation_key ‘/root/.chef/admin.pem’
chef_server_url ‘https://chef-server.example.com:443/’
syntax_check_cache_path ‘/root/.chef/syntax_check_cache’
Check your installation by running the respective commands:
$ knife client list
chef-validator
chef-webui
$ knife user list
admin
knife-user1
Run command (on Linux like)
# curl -L https://www.opscode.com/chef/install.sh | bash
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
101 6790 101 6790 0 0 3826 0 0:00:01 0:00:01 —:—:— 12190
Downloading Chef for el…
Installing Chef
warning: /tmp/tmp.KnyQTnqz/chef-.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
Preparing… ########################################### [100%] 1:chef ########################################### [100%] Thank you for installing Chef!
Create Chef directory
# mkdir /etc/chef
Copy Chef Server Validation Cert Keys from Chef Server to our node in “/etc/chef”:
# scp root@chef-server:/etc/chef-server/chef-validator.pem /etc/chef
Run the command and register yourself in Chef Server:
# chef-client -S https://FQDN-OR-IP-OF-CHEF-SERVER -K /etc/chef/chef-validator.pem
Once the client is verified, we have to create a file in “client.rb” in directory “/etc/chef”.
# vi /etc/chef/client.rb
log_level :info
log_location STDOUT
chef_server_url ‘https://FQDN-OR-IP-OF-CHEF-SERVER’
Check successful registration of the node
On WorkStation, run command:
# knife node list
And on the server in web interface:
# https://FQDN-OR-IP-OF-CHEF-SERVER
Run the client:
# chef-client # chef-client -l debug (In case if you want to debug)
Create a simple Cookbook and write a Recipe there
Log in to WorkStation:
# vi /root/.chef/knife.rb
cookbook_path [ ‘/usr/local/src/chef/cookbooks’ ]
Create a directory for Cookbook:
# mkdir -p /usr/local/src/chef/cookbooks
Now, create a simple Cookbook:
# knife cookbook create cookbook-test
Go to the directory:
# cd /usr/local/src/chef/cookbooks # tree cookbook-test
cookbook-test/
├── attributes
├── CHANGELOG.md
├── definitions
├── files
│ └── default
├── libraries
├── metadata.rb
├── providers
├── README.md
├── recipes
│ └── default.rb
├── resources
└── templates
└── default
Now, let’s create a recipe for the new group (system-admins) and user “sanjay”.
# cat /usr/local/src/chef/cookbooks/cookbook-test/recipes/default.rb
#
# Cookbook Name:: cookbook-test
# Recipe:: default
#
# Copyright 2016, YOUR_COMPANY_NAME
#
# All rights reserved — Do Not Redistribute
#
group «system-admins» do
gid 1001
end
user «sanjay» do
comment «Sanjay User»
shell «/bin/bash»
home «/home/sanjay»
gid «system-admins»
uid 1002
supports :manage_home => true
password «$1$QwuUa80Z$KZkYq8CqICVyIsK1tHZ7s0»
end
To upload cookbooks to the server, do the following:
# knife upload cookbooks
Note: this will upload all cookbooks to the server
To upload a particular book to the server:
# knife upload cookbooks cookbook-test
Now, add our recipe to run_list:
# knife node list
node1.example.com
node2.example.com
node3.example.com
# knife node run_list add node1.example.com cookbook-test
node1.example.com:
run_list: recipe[cookbook-test
Now, log in to the machine node1.example.com, and run command
# chef-client
[2016-10-25T04:47:36-07:00] INFO: Forking chef instance to converge…
Starting Chef Client, version 11.6.2
[2016-10-25T04:47:36-07:00] INFO: *** Chef 11.6.2 ***
[2016-10-25T04:47:37-07:00] INFO: Run List is
] [2016-10-25T04:47:37-07:00] INFO: Run List expands to [cookbook-test] [2016-10-25T04:47:37-07:00] INFO: Starting Chef Run for node1.example.com
[2016-10-25T04:47:37-07:00] INFO: Running start handlers
[2016-10-25T04:47:37-07:00] INFO: Start handlers complete.
resolving cookbooks for run list: [«cookbook-test»] [2016-10-25T04:47:37-07:00] INFO: Loading cookbooks [cookbook-test] Synchronizing Cookbooks:
[2016-10-25T04:47:37-07:00] INFO: Storing updated cookbooks/cookbook-test/recipes/default.rb in the cache.
[2016-10-25T04:47:37-07:00] INFO: Storing updated cookbooks/cookbook-test/metadata.rb in the cache.
[2016-10-25T04:47:37-07:00] INFO: Storing updated cookbooks/cookbook-test/README.md in the cache.
[2016-10-25T04:47:37-07:00] INFO: Storing updated cookbooks/cookbook-test/CHANGELOG.md in the cache.
— cookbook-test
Compiling Cookbooks…
Converging 1 resources
Recipe: cookbook-test::default
* group[system-admins] action create[2016-10-25T22:23:38-07:00] INFO: Processing group[system-admins] action create (cookbook-test::default line 9)
(up to date)
* user[sanjay] action create[2016-10-25T04:47:37-07:00] INFO: Processing user[sanjay] action create (cookbook-test::default line 9)
(up to date)
[2016-10-25T04:47:37-07:00] INFO: Chef Run complete in 0.48225768 seconds
[2016-10-25T04:47:37-07:00] INFO: Running report handlers
[2016-10-25T04:47:37-07:00] INFO: Report handlers complete
Chef Client finished, 0 resources updated
To create one more sendmail installation and run recipe, type on WorkStation:
# vim /usr/local/src/chef/cookbooks/cookbook-test/recipes/sendmail.rb
package 'sendmail' do action :install end service 'sendmail' do action [ :enable,:start ] end
Upload the cookbook:
# knife upload cookbooks cookbook-test
Run on the node:
# chef-client
Recipe: cookbook-test::sendmail
* package[sendmail] action install[2016-10-25T22:05:22-07:00] INFO: Processing package[sendmail] action install (cookbook-test::sendmail line 1)
[2016-10-25T22:06:14-07:00] INFO: package[sendmail] installing sendmail-8.14.4-8.el6 from base repository
— install version 8.14.4-8.el6 of package sendmail
* service[sendmail] action enable[2016-10-28T04:12:10-07:00] INFO: Processing service[sendmail] action enable (system-users::sendmail line 5)
(up to date)
* service[sendmail] action start[2016-10-28T04:12:11-07:00] INFO: Processing service[sendmail] action start (system-users::sendmail line 5)
[2016-10-28T04:12:11-07:00] INFO: service[sendmail] started
— start service service[sendmail]
And that’s how server configuration with Chef automation works. The official site contains a large amount of information about writing recipes and setting up the program.
If we can help support your digital projects, from DevOps consulting to team augmentation or a dedicated team and delivery management for an end-to-end project, we’d love to hear from you. Ask us anything and everything you’d like to know about who we are and how we work or request a quote.
K&C - Creating Beautiful Technology Solutions For 20+ Years . Can We Be Your Competitive Edge?
Drop us a line to discuss your needs or next project