Creating a Development Environment for Awestruct and Asciidoctor with Vagrant and Puppet
Introduction
When exploring creating Asciidoctor based sites with Awestruct, setting up the Ruby environment was quite a bit of work. Some of the gems that had to be installed were native, and weren’t easy to come by on Windows, let alone compile. Also, installing the required gems wasn’t always that straightforward, depending on the installation history of the host environment. The error messages were not always clear, and quite some searching on the internet was required. For each PC where I wanted to work with Awestruct and Asciidoctor, this work repeated itself, of course with variations.
In addition, as a Windows user, I missed the Linux command line, and a good Vim installation. I managed neither with Babun, cygwin directly, or gvim to get Vim to look and work the way I wanted to. One major issue for me was colouring in the terminal. After a few attempts I gave up, and decided to set up an environment using Vagrant and Puppet. I did explore docker, but again, couldn’t get colouring to work in a reasonable amount of time. Also, this environment should work on both Windows, Linux, and Mac: so Vagrant with Puppet appeared to be a good choice. Using Ansible was not an option, as it does not run on a Windows control machine.
This article is about setting up a Vagrant project, and provisioning the environment with Puppet. The complete code is available on github. This article is not a detailed explanation of all steps, but attempts to describe some of the questions that came up, and how a terminal based development environment could look like.
Make sure that the core.autocrlf for the repository containing the vagrant project is set to input, especially if
you also work on this on Windows. The reason is: if you check out the project on Windows with core.autocrlf=true, then provision a
Linux machine, files will be copied over with CRLF line-endings, and shell script will stop working. The visible effect
is ^M showing up in files and error messages.
|
Solution Description
The objectives for this development environment were initially to set up a virtual box with git, ruby, nginx, and vim (spf13-vim). Ruby should be set up and configured such that all gems required for an Awestruct/Asciidoctor project can be installed using bundler and the gemfile in the project.
Later, Java, Maven, and JBake were added to the mix, because it was so easy to do. Also, I looked at JBake as an alternative to Awestruct (and Jekyll), and needed a Java/Maven environment for other tasks as well.
The environment should be as simple to use and set up as possible. This resulted in the following additional points:
No special users, just the default vagrant user.
No special authentication / authorization.
No virus protection.
Default folder mapping just to the project directory, mapped by
/vagrant
.Documents to be worked on can be version controlled from within the box if desired.
Setup
Install Vagrant.
Install Puppet as Windows standalone node. This is needed if you want to use Puppet to install modules to the Vagrant project.
Install your favorite version control system client. We’ll use git throughout this tutorial.
Open a command line prompt and navigate to a folder of your choice.
Create a new subfolder that will contain the Vagrant configuration, and will be the version controlled project. We’ll call it the vagrant folder throughout this tutorial.
Folder structure
The folder structure in the vagrant folder looks like this:
. ├── puppet │ ├── manifests │ │ └── vagrant.pp (1) │ └── modules (2) │ ├── git │ ├── java │ ├── jbake │ ├── maven │ ├── nginx │ ├── shell │ ├── ssh_keygen │ ├── stdlib │ └── vim ├── src (3) ├── tmp (3) │ └── id_rsa.pub └── Vagrantfile (4)
1 | This file contains the provisioning definition. |
2 | Modules are used for provisioning. |
3 | Folders synchronized between the host and the guest system. |
4 | The file defining the Vagrant configuration. |
Vagrant File
The vagrant file, Vagrantfile
, is used to define the Vagrant configuration. The current version looks like this:
Vagrant.configure(2) do |config|
# Base the project on Ubuntu (Trusty):
config.vm.box = "ubuntu/trusty64"
config.vm.hostname = "adoc-devenv"
config.vm.network "private_network", ip: "192.168.33.10"
# Setting up the private network doesn't work on windows 10 with Virtualbox 5.0.10, unless you manually
# check the "VirtualBox NDIS6 Bridged Networking Driver" in the network adapters property page after the adapter has been created.
# In other words: running vagrant up for the first time will create the adapter, but fail to startup the
# virtual box. Find the created network adapter in the Windows 10 settings, select its properties, and select (check)
# "VirtualBox NDIS6 Bridged Networking Driver". Run vagrant up again, this time the box should start normally.
# As a workaround, a public network can be configured instead, using
# config.vm.network "public_network"
# or with a specific IP address:
# config.vm.network "public_network", ip: "192.168.1.166"
# Map the local folder containing possible sources to the folder /work/src in the box.
config.vm.synced_folder "src/", "/work/src"
# Installation of "VirtualBox Guest Additions" (https://github.com/dotless-de/vagrant-vbguest):
# Install vbguest plugin with:
# vagrant plugin install vagrant-vbguest
# Uncomment the following (change version if necessary)
# config.vbguest.auto_update = true
# config.vbguest.iso_path = 'http://download.virtualbox.org/virtualbox/5.0.18/VBoxGuestAdditions_5.0.18.iso'
# Configure provisioning with Puppet:
config.vm.provision :puppet do |puppet|
puppet.manifests_path = "puppet/manifests"
puppet.module_path = "puppet/modules"
puppet.manifest_file = "vagrant.pp"
end
end
Puppet Modules
Puppet modules implement functionality and/or provide files and directories to provision the target box with.
A module is just a directory in the modules folder. You can create your own module, e.g. to provide shell configuration, the module
shell
was created. Note its files
subfolder, which contains the files that can be copied over by explicit Puppet directives in a
manifest.
Existing modules can be used by copying them from the official distribution site to the modules folder, or by installing them using
a local (standalone) Puppet installation. For instance, the Java module can be installed by executing, in the puppet
folder,
puppet module install puppetlabs-java --modulepath .\modules
Puppet Manifest File
The Puppet manifest file, vagrant.pp
, declares the target state of the development environment. We will not discuss it in detail,
but mention a few points I used some time to figure out.
Executing Arbitrary Commands
Commands can be executed by using exec
. For instance, to update the Ubuntu package manager index files, use
exec { 'apt-update':
command => '/usr/bin/apt-get update'
}
Installing Software
In general, the package manager of the target box should be used to install software. This makes the puppet manifest more portable.
It is accomplished using package
, e.g. in order to install unzip
, add the following to the vagrant file:
# install unzip
package { 'unzip':
require => Exec['apt-update'],
ensure => installed,
}
Files and Directories
A file can be copied from the host to the box using:
# shell configuration
file { '/home/vagrant/.profile':
owner => vagrant,
group => vagrant,
mode => 664,
ensure => 'present',
source => 'puppet:///modules/shell/.profile',
}
The file .profile
has to be stored in folder called files
in the shell
module folder. Note that the name of the module,
shell
, is an arbitrary choice of name, but files
is not.
A directory can be created using:
file { '/home/vagrant/bin/':
owner => vagrant,
group => vagrant,
mode => 755,
ensure => 'directory',
}
A directory can be copied recursively using:
# install maven
file { '/home/vagrant/bin/maven-3.3.3':
owner => vagrant,
group => vagrant,
mode => 664,
ensure => 'present',
source => 'puppet:///modules/maven/maven-3.3.3/',
recurse => true,
require => File['/home/vagrant/bin/'],
}
Note the recurse ⇒ true
property and the requirement for the directory /home/vagrant/bin
to be present.
Installing Java
In order to install the Java OpenJDK, install the puppetlabs-java module, and add the following line to vagrant.pp
:
# install java
# this installs openjdk-7-jdk on Trusty
include java
Creating ssh-Keys
Install the ssh_keygen module, and add the following line to the vagrant file:
ssh_keygen { 'vagrant': }
This will create a private/public key pair in /home/vagrant/.ssh
.
Configuring git
Install the git module, and add the following to the vagrant file (make sure to use your name and email address!):
# install and configure git
include git
git::config { 'user.name':
value => 'First Last',
user => 'vagrant',
require => Class['git'],
}
git::config { 'user.email':
value => 'first@last.net',
user => 'vagrant',
require => Class['git'],
}
git::config { 'core.autocrlf':
value => 'input',
user => 'vagrant',
require => Class['git'],
}
git::config { 'core.editor':
value => 'vim',
user => 'vagrant',
require => Class['git'],
}
Shared Folders
Folders can be shared by the host and the box. By default, the vagrant folder on the host is mapped to /vagrant
in the box. Two
folder were set up for sharing (see Folder Structure): src
, which might contain version controlled code you want to work on,
and tmp
, the content of which should not be version controlled. The src
-folder is mapped to /work/src
in the box as defined
in the vagrant file. tmp
is available as /vagrant/tmp
in the box.
In the tmp
folder, create a .gitignore
file with the following content:
* !.gitignore
Using the Development Environment
Open a command line prompt, and navigate to the vagrant folder.
Tip: on Windows, use Babun as your shell.
Start your environment by typing
vagrant up
. This might take a while, especially if it is the first time you do this.Connect to your box by executing
vagrant ssh
. You need to have an ssh client installed in order for this to work.If you want to clone your site’s git repository in your box, you need to copy the public key that has been generated as part of provisioning of your box in the directory /home/vagrant/.ssh to the remote git server. Copy the public key to the /vagrant/tmp mapped folder, and copy it from there, on the host, to e.g. github.
Once finished working on your project, run
vagrant halt
on the host. Do not usevagrant destroy
, unless all your data is safely stored away in some other place.
Working With a Site
You can test the project using the guarding branch of https://github.com/mgfeller/aablog-foundation.
Clone into
/home/vagrant/awestruct-project
(this is defined as server root in the provided nginx configuration file).Run
bundle install
to install all required gems.Generate the site using
awestruct -g
, the result is generated in the _site folder.On your host, open a browser and go to the IP address specified in the Vagrant file, port 90 (as specified in the nginx default configuration file).
Tooling Tips
Using the Screen Tool
From the man-page: "Screen is a full-screen window manager that multiplexes a physical terminal between several processes (typically interactive shells)."
See also the Linux Screen Tutorial.
On your box, in the site folder, start screen by executing screen
.
Help is available with screen --help
and man screen
.
The one command to remember is C-a ?
. This will display a list of the available screen commands and their bindings.
Assume you want to run guard
in one screen, and edit your files with vim in another:
Start
screen
if you haven’t done so already.In the site directory, start guard using
bundle exec guard
, you should see some information that guard has been started.Create another screen window by typing
C-a c
.Start
vim
and edit your files.Switch between windows by using
C-a n
(next) orC-a p
(previous).Close (aka kill) a window by
C-a k
and confirming.Closing all windows closes the screen as well.
Useful Vim Commands
Some commands I found useful:
Setting current language to asciidoc for syntax colouring:
:set syntax=asciidoc
Open file explorer on current folder:
:e .
Open file explorer in a vertical split:
:vs.
Swap splits, left-right:
Ctrl-W r
Goto next tab (normal mode):
gt
Goto buffer by number (e.g. 3):
:b 3
Goto buffer by name (e.g. doc), supports tab-completion:
:b doc
Exit insert mode:
Ctrl-C
Save all open buffers at once:
:wa
Wrap line (normal mode):
gq
Setting line width:
:set textwidth=132
Copy/paste (normal mode): mark the area using
v
and the cursor, theny
to copy. Insert before cursor withP
, after cursor withp
Conclusion
Using Vagrant and Puppet makes it easy to create a Linux-based development environment that can be used to provide the tooling necessary for Awestruct/Asciidoctor based sites. At the same time, it serves also as documentation for setting up and configuring an environment on another (Ubuntu-based) machine, if so desired. The possibility to use the proper Linux command line and Vim on a Windows box is a big plus.