This page should have enough information for new developers to setup Chef environment from a newly provisioned Mac OSX.

Hosted Chef

Register for an Account

1. Create an account at https://manage.chef.io/signup (naming convention can be company_name_, cs_, _company_name, etc)
2. Once you have created an account, have someone with admin access to send you an invite to join the organization

Key Setup

If you don’t have your secret key you will have to reset. Login to the portal https://manage.chef.io/login.

1. Click on “Administration” tab on the top.
2. Click on “Users” located on the left side tree.
3. Highlight your name and click on the gear icon on right side of your email address.
4. Click on “Reset Key”
5. Download your secret key and save to local.

Make sure to chmod 600 your secret key.

Mac - Softwares/Dev Tools

• Homebrew
  • We utilize homebrew to install additional softwares.

      $ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
    

• Brew Cask
  • Cask is used to install GUI applications

      $ brew install cask
    

• Ruby
  • Install the latest version of ruby using brew

      $ brew install ruby
    

• ChefDK/Vagrant/Virtual Box
  • You can download and install from their websites, or just use Cask to install from command line

      $ brew cask install chefdk vagrant virtualbox
    

• Additional Development Tools
  • Install the additional tools and gems

      $ brew install terraform awscli packer go git wget
      $ gem install berkshelf bundler test-kitchen
    

Windows 10 - Bash Setup

NOTE: Below steps are for Windows 10 w/ Bash - ckim 8/15/16

• Update ubuntu

  $ sudo apt-get update
  $ sudo apt-get install git build-essential wget unzip virtualbox libssl-dev openssl -y
  

• install rvm as there will be issues with gem’s zlib not referencing properly if you compile rub from source before this step.

  $ gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
  $ curl -sSL https://get.rvm.io | bash
  $ curl -sSL https://get.rvm.io | bash -s stable --ruby
  

• download ruby > 2.2 and install

  $ wget https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.1.tar.gz
  $ ./configure
  $ make
  $ sudo make install
  

• Make sure all dependencies are covered.

  rvm requirements
  

• Install packages

  $ sudo apt-get install vagrant
  $ sudo gem install berkshelf bundler test-kitchen
  $ wget https://packages.chef.io/stable/ubuntu/12.04/chefdk_0.16.28-1_amd64.deb
  $ sudo dpkg -i chefdk_0.16.28-1_amd64.deb
  

• Install terraform

  $ wget https://releases.hashicorp.com/terraform/0.7.0/terraform_0.7.0_linux_amd64.zip
  $ gunzip terraform_0.7.0_linux_amd64.zip
  $ sudo mkdir -p /usr/local/terraform/bin ; sudo cp terraform /usr/local/terraform/bin
  $ echo PATH=/usr/local/terraform/bin:/home/your-user-name/terraform:$PATH >> .bashrc
  

Environment Setup (chef/knife)

1. On your local machine create a hidden directory .chef.

   $ mkdir ~/.chef/
   

2. Create a file named knife.rb and modify both the node_name and client_key with your information. For consistency we have decided to name both the same as your username to chef.io. Ensure your .pem is in the ~/.chef directory or else you won't be able to interact with the server.

    # See https://docs.chef.io/config_rb_knife.html for more information on knife configuration options
    current_dir = File.dirname(__FILE__)
   
    log_level               :info
    log_location            STDOUT
    node_name               "company_name_cwong"
    client_key              "#{current_dir}/company_name_cwong.pem"
    validation_client_name  "company_name-validator"
    validation_key          "#{current_dir}/company_name-validator.pem"
    chef_server_url         "https://api.opscode.com/organizations/company_name"
    cookbook_path           ["#{current_dir}/../cookbooks"]
   

3. Grab a copy of the file encrypted_data_bag_secret from secure:/home/secure/hosted_chef/encrypted_data_bag_secret and place the file inside ~/.chef/

   $ ls -l ~/.chef/
   total 24
   drwxr-xr-x  4 cwong  987599992   136 Jul 23  2015 cache/
   -rw-r-----@ 1 cwong  987599992  1678 Jul 20  2015 company_name_cwong.pem
   -rw-------  1 cwong  987599992   685 Aug  7  2015 encrypted_data_bag_secret
   -rw-r-----@ 1 cwong  987599992   556 Jul 22  2015 knife.rb
   

4. Test connection

   $ knife data bag list
   

5. Setup your environment EDITOR from the shell or add to your ~/.bash_profile

   $ export EDITOR=vim
   

Test Kitchen

1. Generating cookbook with berks cookbook <cookbookname> is deprecated? We can generate a generic cookbook using chef

   $ chef generate cookbook helloworld
   

2. Configure your Gemfile inside the cookbook (make a file called Gemfile in the helloworld directory, and paste the following into it)

   source 'https://rubygems.org'
   gem 'berkshelf'
   gem 'kitchen-vagrant'
   gem 'test-kitchen'
   gem 'chefspec'
   gem 'rspec'
   gem 'serverspec'
   gem 'rake'
   gem 'fauxhai'
   

3. Modify your recipes/defaults.rb to actually do something

   $ echo 'puts "This is my first recipe, Hello world"' >> recipes/default.rb
   

4. Modify your cookbook’s .kitchen.yml to include the default recipe ```bash — driver: name: vagrant provisioner: name: chef_zero platforms:
   • name: ubuntu-15.04 suites:
   • name: default run_list:
   • recipe[helloworld::default] ```
5. Build using the cookbook recipe we just created

   $ kitchen converge
   -----> Starting Kitchen (v1.7.2)
   -----> Converging <default-ubuntu-1504>...
       Preparing files for transfer
       Preparing dna.json
       Resolving cookbook dependencies with Berkshelf 4.3.2...
       Removing non-cookbook files before transfer
       Preparing validation.pem
       Preparing client.rb
   -----> Chef Omnibus installation detected (install only if missing)
       Transferring files to <default-ubuntu-1504>
       Starting Chef Client, version 12.9.38
       Creating a new client identity for default-ubuntu-1504 using
   the validator key.
       resolving cookbooks for run list: ["helloworld::default"]
       Synchronizing Cookbooks:
         - helloworld (0.1.0)
       Installing Cookbook Gems:
       Compiling Cookbooks...
       This is my frist recipe, Hello world
       Converging 0 resources
       Running handlers:
       Running handlers complete
       Chef Client finished, 0/0 resources updated in 01 seconds
       Finished converging <default-ubuntu-1504> (0m3.31s).
   -----> Kitchen is finished. (0m4.30s)
   

6. Vierify the kitchen converged by running:

   $ kitchen list
   

7. Verify by loggin in

   $ kitchen login
   

8. Once you have verified your work and want to clean up, exit out of the VM and terminate the instance

   $ kitchen destroy
   

company_name Cookbooks

Assuming you have access to the chef cookbooks on https://stash.company_nameng.com/projects/CHEF

Clone Cookbooks

Clone the cookbooks to your local machine

$ mkdir ~/git-company_name
$ cd ~/git-company_name
$ git clone
ssh://git@stash.company_nameng.com/chef/company_name-chef-splunk.git
Cloning into 'company_name-chef-splunk'...
Warning: Permanently added 'stash.company_nameng.com,54.183.118.40' (RSA) to
the list of known hosts.
remote: Counting objects: 337, done.
remote: Compressing objects: 100% (276/276), done.
remote: Total 337 (delta 121), reused 0 (delta 0)
Receiving objects: 100% (337/337), 54.27 KiB | 0 bytes/s, done.
Resolving deltas: 100% (121/121), done.
Checking connectivity... done.

Create New Cookbook

Stash

1. You have a new project (company_name-chef-kafka-consumer), you will need to create a git repo on stash.company_nameng.com
2. Once the repo is created, you will want to go to Settings, under WORKFLOW, click Hooks and enable “Stash Webhook to Jenkins”
   1. Jenkins URL: https://ci-dev.company_nameng.com Repo Clone URL: SSH
   2. Check: Skip SSL Certificate Validation

Jenkins

1. Create a new jenkins build pipeline from chef-cookbooks
   1. Click “New Item”
   2. Item Name: company_name-chef-kafka-consumer
   3. Copy Existing Item: company_name-chef-kafka-admin (and change the settings from admin to match your current cookbook/repo)
2. Once the pipeline is created, the next time you commit changes, your job will show up in the chef-global-pipeline

Git

When you’re ready to commit, just include a new tag to the cookbook and commit changes

$ git tag 0.0.1
$ git push --tags
$ git commit -am "cookbook changes with tags"
$ git push

At this point your job will show up in Jenkins chef-global-pipeline. During the git commit process, you have the option to “Skip Integration” testing by passing the following to the commit comment “"; this will save you some time during lower environment testing.

Misc Knife/Chef-Client Commands

Here are some useful knife/chef-client commands

Data Bag Commands

# list data bags
$ knife data bag list

# show/edit user detail
$ knife data bag show users rbowlby -Fj
$ knife data bag edit users rbowlby -Fj

# list items in a data bag
$ knife data bag show company_name-prayse-prediction

# edit individual items in a data bag
$ knife data bag edit company_name-prayse-prediction prd
$ knife data bag edit company_name-prayse-prediction prd-secrets -Fj --secret-file=/Users/cwong/.chef/encrypted_data_bag_secret

SSH Commands

# check uptime on all instance(s) that are in the stg ENV, and uses the company_name-prayse-prediction recipe
$ knife ssh 'chef_environment:stg AND (recipe:company_name-prayse-prediction)' -x cwong 'uptime'

# run chef-client on all instances that match the following criteria using the automation account (NOPASSWD for chef-client)
$ knife ssh '(chef_environment:stg AND recipe:company_name-prayse-prediction)' -x automation -i ~/.ssh/id_rsa_automation "sudo chef-client"

# run chef-client to ONLY apply one cookbook/recipe (useful for updating user
$ chef-client -o 'recipe[company_name-users]'

Node Commands

# list all nodes managed by chef
$ knife node list

Search Commands

# get info on the node
$ knife search node 'name:prediction01.company_name.com' -Fj
$ knife search node 'chef_environment:prd AND recipe:company_name-pitch' -Fj

# list available environment(s)
$ knife environment list

# list available cookbook(s)
$ knife cookbook list

# list all available nodes/clients
$ knife node list
$ knife client list

Knife Performance/Troubleshooting

Knife running slow

$ time knife null
real 0m18.482s

$ time knife rehash
Using knife-rehash will speed up knife's load time by caching the location
of subcommands on disk.
However, you will need to update the cache by running `knife rehash`
anytime you install a new knife plugin.
Knife subcommands are cached in /Users/cwong/.chef/plugin_manifest.json.
Delete this file to disable the caching.
real 0m17.515s

$ time knife null
real 0m7.186s

Cleaning up some Gem files

$ gem cleanup
$ time knife null
real 0m0.530s