CODIEZ

second-person plural present subjunctive of coder

Install Oracle on CentOS 6.4 With Vagrant and Ansible

This post will cover installing oracle xe on CentOS 6.4 using vagrant and ansible. If you have not already, read the first post on creating a vagrant centos base box

The source code is available at: https://github.com/ismaild/vagrant-centos-oracle

About Ansible

Ansible is an IT orchestration engine/configuration management system, that lets you easily describe how you would like your servers to look and then automate it. It differs from Chef and Puppet, as it requires nothing to be installed on the server, it uses ssh. Tasks are described in yml, which can be written and read even by non programmers.

Vagrant init

Lets make sure we have our CentOS 6.4 base box

1
2

$ vagrant box list
centos-64-x86_64 (virtualbox)

Vagrant uses a Vagrantfile to describe the type of machine you would like to build, and the file should be stored in the root directory of your source code. Next we create the Vagrantfile

1
2
3
4
5
6
7

$ mkdir ../vagrant-centos-oracle
$ cd ../vagrant-centos-oracle
$ vagrant init
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.

Edit the Vagrantfile and change config.vm.box to:

1

  config.vm.box = "centos-64-x86_64"

Now we run:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
[default] Importing base box 'centos-64-x86_64'...
[default] Matching MAC address for NAT networking...
[default] Setting the name of the VM...
[default] Clearing any previously set forwarded ports...
[default] Creating shared folders metadata...
[default] Clearing any previously set network interfaces...
[default] Preparing network interfaces based on configuration...
[default] Forwarding ports...
[default] -- 22 => 2222 (adapter 1)
[default] Booting VM...
[default] Waiting for machine to boot. This may take a few minutes...
[default] Machine booted and ready!
[default] Mounting shared folders...
[default] -- /vagrant
$ vagrant ssh
Welcome to your Vagrant-built virtual machine.
[vagrant@localhost ~]$

Great it works, we have a basic CentOS 6.4 Minimal install working, this is exactly like the box we created with veewee. Not too interesting just yet. We can add other customizations, like increase the memory size by adding this to the Vagrantfile

1
2
3

  config.vm.provider :virtualbox do |vb|
    vb.customize ["modifyvm", :id, "--memory", "1024"]
  end

You will also notice, that /vagrant folder is a shared folder with your host machine, the root of the directory where the Vagrantfile is stored.

1
2
3
4
5
6
7
8
9
10

[vagrant@localhost ~]$ ls /vagrant/
Vagrantfile
[vagrant@localhost ~]$ touch /vagrant/abd.txt
[vagrant@localhost ~]$ ls /vagrant/
abd.txt  Vagrantfile
[vagrant@localhost ~]$ exit
logout
Connection to 127.0.0.1 closed.
$ ls
Vagrantfile abd.txt

Creating the playbook

Ansible has playbooks, which basically describe all the steps ansible needs to execute to get your system to the required state. All the steps are described in a yml file, with specific keywords for each task. You can read more at: http://www.ansibleworks.com/docs/playbooks.html

Before we start with our oracle playbook, we need to create a few directories.

1
2
3
4

$ mkdir provisioning
$ touch provisioning/oracle-xe.yml
$ mkdir oracle
$ touch oracle/xe.rsp

Before installing oracle, a few base packages are needed. Lets ensure our playbook caters for these:

oracle-xe.yml 
1
2
3
4
5
6
7
8
9
10
11

---
- hosts: all
  sudo: yes
  tasks:
    - name: ensure packages required are installed
      yum: pkg=$item state=latest
      with_items:
        - libaio
        - bc
        - flex
        - unzip
  • hosts: specifies which hosts to run the playbook on
  • Then we tell ansible to use sudo to run the commands
  • We then have the list of tasks we need to run
  • The yum command can be used to install packages, specifying them with pkg=
  • with_items lets you specify multiple items and ansible will loop through all of the items on the list and run yum for each package
  • For the name: you can use anything that describes the task.

Unfortunately due to oracle licensing, you will need to accept the license agreement and download the oracle rpm from:

http://www.oracle.com/technetwork/products/express-edition/downloads/index.html

Save the zip file to the oracle directory.

The next thing we need to include in our playbook, is unzipping and installing oracle:

oracle-xe.yml 
1
2
3
4

    - name: unzip oracle rpm
      command: /usr/bin/unzip -q /vagrant/oracle/oracle*.rpm.zip -d /vagrant/oracle creates=/vagrant/oracle/Disk1
    - name: install oracle
      shell: /bin/rpm -ivh /vagrant/oracle/Disk1/oracle-xe-11.2.0-1.0.x86_64.rpm creates=/u01
  • creates= defines a directory that is created when the task runs, if the box is reloaded and the directory exists, the task will be skipped.
  • You will need this or to ignore errors as the oracle installation returns an error if it is already installed.

Then we configure oracle and the vagrant user environment

oracle-xe.yml 
1
2
3
4
5

    - name: configure oracle
      shell: /etc/init.d/oracle-xe configure responseFile=/vagrant/oracle/xe.rsp
      ignore_errors: True
    - name: setup oracle environment
      shell: /bin/echo 'source /u01/app/oracle/product/11.2.0/xe/bin/oracle_env.sh' >> /home/vagrant/.bash_profile
  • shell does what it says on the box
  • We need to pass the oracle configure script a response file xe.rsp so it does not wait for input
  • Then we setup the vagrant users environment

Add the following to oracle/xe.rsp

text 
1
2
3
4
5

ORACLE_HTTP_PORT=8080
ORACLE_LISTENER_PORT=1521
ORACLE_PASSWORD=manager
ORACLE_CONFIRM_PASSWORD=manager
ORACLE_DBENABLE=y

We need to tell Vagrant to use the ansible playbook, by adding the following to the Vagrantfile. We also set the ansible verbose setting to extra, so we can see what is going on.

1
2
3
4

  config.vm.provision "ansible" do |ansible|
    ansible.playbook = "provisioning/oracle-xe.yml"
    ansible.verbose = "extra"
  end

We can now test that oracle works from inside the VM:

1
2
3
4
5
6
7
8
9
10
11
12
13

$ vagrant up
$ vagrant ssh
Last login: Sun Nov 10 11:35:22 2013 from 10.0.2.2
Welcome to your Vagrant-built virtual machine.
[vagrant@localhost ~]$ sqlplus system/manager@localhost

SQL*Plus: Release 11.2.0.2.0 Production on Sun Nov 10 12:37:23 2013

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production

Vagrant port fowarding – connect from your host

If we try an connect from the host to the VM, it wont work. To get this to work we will use vagrant port forwarding by adding the following to our Vagrantfile.

1
2

  config.vm.network "forwarded_port", guest: 8080, host: 8080, auto_correct: true
  config.vm.network "forwarded_port", guest: 1521, host: 1521, auto_correct: true

We also need to modify our playbook, to disable iptables and to configure oracle to accept remote connections, from outside the VM.

oracle-xe.yml 
1
2
3
4
5
6

    - name: stop ip tables
      shell: service iptables stop
    - name: set oracle listener
      sudo: False
      shell: ORACLE_HOME=/u01/app/oracle/product/11.2.0/xe /u01/app/oracle/product/11.2.0/xe/bin/sqlplus
        system/manager@localhost < /vagrant/oracle/set_listener.sql

Also create a new file: oracle/set_listener.sql

set_listener.sql 
1
2
3

EXEC DBMS_XDB.SETLISTENERLOCALACCESS(FALSE);
quit;
/

Lets see if we can connect from our host to the VM:

1
2
3
4
5
6
7
8
9
10
11
12
13

$ vagrant provision
$ sqlplus system/manager@localhost

SQL*Plus: Release 11.2.0.3.0 Production on Sun Nov 10 15:19:50 2013

Copyright (c) 1982, 2012, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production

SQL> exit
$ curl -v http://localhost:8080/apex/f?p=4950:1

Great, it works. This may seem like an awful amount of work just to setup a centos vm with oracle, though remember, Once you have done this once, no one else needs to follow the same steps again. All they need to do is issue a vagrant up from your source code repo.

You can view the full source for this tutorial at: https://github.com/ismaild/vagrant-centos-oracle

The full Vagrantfile and oracle-xe.yml are below.

Vagrantfile

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

# -*- mode: ruby -*-
# vi: set ft=ruby :

# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "centos-64-x86_64"

  config.vm.provider :virtualbox do |vb|
    vb.customize ["modifyvm", :id, "--memory", "1024"]
  end

  config.vm.provision "ansible" do |ansible|
    ansible.playbook = "provisioning/oracle-xe.yml"
    ansible.verbose = "extra"
  end

  config.vm.network "forwarded_port", guest: 8080, host: 8080, auto_correct: true
  config.vm.network "forwarded_port", guest: 1521, host: 1521, auto_correct: true
end

Ansible Oracle Playbook

oracle-xe.yml 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

---
- hosts: all
  sudo: yes
  tasks:
    - name: ensure packages required are installed
      yum: pkg=$item state=latest
      with_items:
        - libaio
        - bc
        - flex
        - unzip
    - name: unzip oracle rpm
      command: /usr/bin/unzip -q /vagrant/oracle/oracle*.rpm.zip -d /vagrant/oracle creates=/vagrant/oracle/Disk1
    - name: install oracle
      shell: /bin/rpm -ivh /vagrant/oracle/Disk1/oracle-xe-11.2.0-1.0.x86_64.rpm creates=/u01
    - name: configure oracle
      shell: /etc/init.d/oracle-xe configure responseFile=/vagrant/oracle/xe.rsp
      ignore_errors: True
    - name: setup oracle environment
      shell: /bin/echo 'source /u01/app/oracle/product/11.2.0/xe/bin/oracle_env.sh' >> /home/vagrant/.bash_profile
    - name: stop ip tables
      shell: service iptables stop
    - name: set oracle listener
      shell: ORACLE_HOME=/u01/app/oracle/product/11.2.0/xe /u01/app/oracle/product/11.2.0/xe/bin/sqlplus
        system/manager@localhost < /vagrant/oracle/set_listener.sql

Build a Vagrant CentOS 6.4 Base Box

This post will discuss building a CentOs 6.4 vagrant box, and in later posts we will use this box and ansible to further define our machines, such as installing oracle xe.

About Vagrant

Vagrant lets you create and configure lightweight, reproducible, and portable development environments. All too often you will find teams developing off a central server, purely because of the pain of setting up a dev environment. Nothing is automated. There are many challenges with this approach. IMHO this slows down a team, and experimentation is discouraged and you end up “stepping on each others code”.

Vagrant can solve some these challenges, ever started a new project and spent a few days just getting your environment setup? Well with vagrant this should be a thing of the past.

Before you start

You will need some baseline software before we begin. Make sure you have the following installed.

Lets check we have everything installed

1
2
3
4
5
6
7
8

$ git version
git version 1.8.3.4 (Apple Git-47)
$ vagrant --version
Vagrant 1.3.5
$ VBoxManage --version
4.3.2r90405
$ veewee version
Version : 0.3.12 - use at your own risk

Build a CentOS 6.4 base box

You can find a number of vagrant base boxes online, but i prefer creating one from scratch. We can do that easily with veewee.

First we clone the repo: git clone https://github.com/ismaild/vagrant-boxes.git. I have disabled chef and puppet since we will be using ansible. Feel free to add it back in if you want it, which you can by editing: definitions/CentOS-6.4-x86_64/definition.rb

Veewee will automatically try and download the required ISO files. If you already have them, you can put them in the iso directory and it will not need to download.

Just make sure the version of the virtualbox guest additions matches your installed version of virtualbox.

Then run the following commands:

1
2
3
4
5
6
7
8
9
10

$ cd vagrant-boxes
$ veewee vbox list
# if no boxes defined
$ veewee vbox define CentOS-6.4-x86_64 CentOS-6.4-x86_64-minimal
# Build the box
$ veewee vbox build CentOS-6.4-x86_64
# Eject the disks from the running VM and shutdown.
# Package the box
$ vagrant package --base CentOS-6.4-x86_64 --output CentOS-6.4-x86_64.box
$ vagrant box add centos-64-x86_64 CentOS-6.4-x86_64.box

Now we have a base CentOS 6.4 box to use. In the next post we will go through setting up oracle with ansible.

Setup Oracle Instant Client and Ruby Oci8 Gem on Mac

Recently i have had to get my dev environment setup to connect to oracle for a project.

Getting the client setup can sometimes be a real pain with a Mac. Oracle does provide client libraries but they are painfully slow in updating them and fixing bugs. There was an outstanding segmentation fault bug for over 2 years.

Here are the steps to get it setup on Mac, and it would probably work for ubuntu etc, just have not tested it.

First grab the 64 bit client from:

http://www.oracle.com/technetwork/topics/intel-macsoft-096467.html

You need:

  • instantclient-basic-macos.x64-11.2.0.3.0.zip
  • instantclient-sqlplus-macos.x64-11.2.0.3.0.zip
  • instantclient-sdk-macos.x64-11.2.0.3.0.zip

You can also grab the lite version, if you do not need any translations.

Unzip the files:

1
2
3
4
5
6
7
8
9
10

cd ~/Downloads
#basic
unzip -qq instantclient-basic-macos.x64-11.2.0.3.0.zip

#lite
unzip -qq instantclient-basiclite-macos.x64-11.2.0.3.0.zip

unzip -qq instantclient-sqlplus-macos.x64-11.2.0.3.0.zip

unzip -qq instantclient-sdk-macos.x64-11.2.0.3.0.zip

Move files

1
2
3
4
5
6

cd instantclient_11_2
mkdir -p /usr/local/oracle/product/instantclient_64/11.2.0.3.0/bin
mkdir -p /usr/local/oracle/product/instantclient_64/11.2.0.3.0/lib
mkdir -p /usr/local/oracle/product/instantclient_64/11.2.0.3.0/jdbc/lib
mkdir -p /usr/local/oracle/product/instantclient_64/11.2.0.3.0/rdbms/jlib
mkdir -p /usr/local/oracle/product/instantclient_64/11.2.0.3.0/sqlplus/admin

1
2
3
4
5
6
7
8
9
10
11
12

mv ojdbc* /usr/local/oracle/product/instantclient_64/11.2.0.3.0/jdbc/lib/
mv x*.jar /usr/local/oracle/product/instantclient_64/11.2.0.3.0/rdbms/jlib/

# rename glogin.sql to login.sql
mv glogin.sql /usr/local/oracle/product/instantclient_64/11.2.0.3.0/sqlplus/admin/login.sql

# Move lib & sdk
mv *dylib* /usr/local/oracle/product/instantclient_64/11.2.0.3.0/lib/
mv sdk /usr/local/oracle/product/instantclient_64/11.2.0.3.0/lib/sdk

mv *README /usr/local/oracle/product/instantclient_64/11.2.0.3.0/
mv * /usr/local/oracle/product/instantclient_64/11.2.0.3.0/bin/

Setup TNS Names

1
2

mkdir -p /usr/local/oracle/admin/network
touch /usr/local/oracle/admin/network/tnsnames.ora

Put in your tnsnames, example:

tnsnames.ora 
1
2
3
4
5
6
7
8
9

 ORADEMO=
 (description=
   (address_list=
     (address = (protocol = TCP)(host = 127.0.0.1)(port = 1521))
   )
 (connect_data =
   (service_name=orademo)
 )
)

Setup your environment

Create a file to store your oracle client environment variables with touch ~/.oracle_client

Add the following to it:

.oracle_client 
1
2
3
4
5
6

export ORACLE_BASE=/usr/local/oracle
export ORACLE_HOME=$ORACLE_BASE/product/instantclient_64/11.2.0.3.0
export PATH=$ORACLE_HOME/bin:$PATH
export DYLD_LIBRARY_PATH=$ORACLE_HOME/lib:$DYLD_LIBRARY_PATH
export TNS_ADMIN=$ORACLE_BASE/admin/network
export SQLPATH=$ORACLE_HOME/sqlplus/admin

Then run:

1
2

echo "source ~/.oracle_client" >> ~/.bash_profile
source ~/.bash_profile

Which will add the environment variables to your .bash_profile, You can also find this in my dotfiles

Test Sql*Plus works

1
2
3
4
5
6
7
8
9
10
11
12

sqlplus user/pass@orademo

SQL*Plus: Release 11.2.0.3.0 Production on Thu Sep 12 09:19:55 2013

Copyright (c) 1982, 2012, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> select table_name from user_tables;

You could also store multiple versions of the client, with different .oracle_client files, with a small shell script you could switch between different versions (i.e like when oracle does not fix a bug for 2 years! )

Install Ruby oci gem

1
2
3
4
5
6

cd /usr/local/oracle/product/instantclient_64/11.2.0.3.0/lib
ln -s libclntsh.dylib.11.1 libclntsh.dylib
ln -s libocci.dylib.11.1 libocci.dylib

# Make sure sdk directory is in /usr/local/oracle/product/instantclient_64/11.2.0.3.0/lib
gem install ruby-oci8

Test that it works…

1
2
3
4
5
6
7
8

irb
irb(main):001:0> require 'oci8'
irb(main):006:0> o = OCI8.new('user','pass','127.0.0.1/orademo')
=> #<OCI8:user>
irb(main):011:0> o.exec('select * from dual') do |r| puts r.join(','); end
X
=> 1
irb(main):012:0> exit

If you have issues connecting with an SID or Service Name, try using the IP.

You now have a working oracle database client, and can connect to it from ruby.

Setup a MongoDB Container With a Docker File

In our previous post we went through the steps to Setup a Docker Container With MongoDB manually.

Docker has a simple DSL that lets you automate all of these steps to make a conainer.

Docker file syntax

Every line in a docker file has the following structure: INSTRUCTION arguments

Comments are ignored, and the first line in the docker file should contain the command FROM <image

Commands available full details

  • FROM (select the base image)
  • MAINTAINER (Set the author field for images)
  • RUN (run a command, and commit)
  • CMD (the default execution command for the container)
  • EXPOSE (set the port to be publicly exposed)
  • ENV (set environment variables)
  • ADD (add files from source and copy them to the container)
  • ENTRYPOINT (configure the container to run as an executable)
  • VOLUME (add a volume)
  • USER (set the user)
  • WORKDIR (set working directory)

MongoDB dockerfile

We create a dockerfile, and just use all the same commands we used previously touch Dockerfile

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

FROM ubuntu:latest

RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
RUN echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | tee /etc/apt/sources.list.d/10gen.list

RUN dpkg-divert --local --rename --add /sbin/initctl
RUN ln -s /bin/true /sbin/initctl

RUN apt-get update
RUN apt-get install mongodb-10gen

RUN mkdir -p /data/db

EXPOSE 27017
CMD ["usr/bin/mongod", "--smallfiles"]

Then we issue:

sudo docker build -t codiez/mongodb .

and start it up with…

sudo docker run -d codiez/mongodb

In the next post, we will discuss creating a SAAS database service using docker and mongodb.

Setup a Docker Container With MongoDB

If you have not yet setup docker, follow the install docker post before this.

First, lets take a look at what docker images we have.

1
2
3
4
5
6
7

dock@saas:~$ sudo docker images
REPOSITORY          TAG                 ID                  CREATED             SIZE
ubuntu              12.04               8dbd9e392a96        4 months ago        131.5 MB (virtual 131.5 MB)
ubuntu              12.10               b750fe79269d        5 months ago        24.65 kB (virtual 180.1 MB)
ubuntu              latest              8dbd9e392a96        4 months ago        131.5 MB (virtual 131.5 MB)
ubuntu              precise             8dbd9e392a96        4 months ago        131.5 MB (virtual 131.5 MB)
ubuntu              quantal             b750fe79269d        5 months ago        24.65 kB (virtual 180.1 MB)

Notice we have the images locally, so if we run the hello world the command runs instantly, without the need to download images again.

Now, lets check if we have and docker containers running:

1
2

dock@saas:~$ sudo docker ps
ID                  IMAGE               COMMAND             CREATED             STATUS              PORTS

As expected, we do not have any containers currently running.

Install MongoDB

Start an interactive shell in a container with: sudo docker run -i -t ubuntu /bin/bash

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

root@f9d75a04ce10:/# apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10

root@f9d75a04ce10:/# echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | tee /etc/apt/sources.list.d/10gen.list

root@f9d75a04ce10:/# apt-get update

# initctl fix
root@a1f0680f8458:/# dpkg-divert --local --rename --add /sbin/initctl

root@a1f0680f8458:/# ln -s /bin/true /sbin/initctl

root@a1f0680f8458:/# apt-get install mongodb-10gen

root@a1f0680f8458:/# mkdir -p /data/db

root@a1f0680f8458:/# mongod
mongod --help for help and startup options
Tue Sep  3 14:10:36.469 [initandlisten] MongoDB starting : pid=125 port=27017 dbpath=/data/db/ 64-bit host=a1f0680f8458

exit

MongoDB starts succesfully in the container. Now we need to commit the container, to save the state and push to the docker index.

sudo docker commit a1f0680f8458 codiez/mongodb

Remember to replace your container id, mine was a1f0680f8458 which you can see at the prompt, and your <username>/<container_name>

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

dock@saas:~$ sudo docker commit a1f0680f8458 codiez/mongodb

dock@saas:~$ sudo docker login
Username: codiez
Password:
Email: ismail@codiez.co.za
Login Succeeded
dock@saas:~$ sudo docker push codiez/mongodb
The push refers to a repository [codiez/mongodb] (len: 1)
Processing checksums
Sending image list
Pushing repository codiez/mongodb (1 tags)
Pushing 8dbd9e392a964056420e5d58ca5cc376ef18e2de93b5cc90e868a1bbc8318c1c
Image 8dbd9e392a964056420e5d58ca5cc376ef18e2de93b5cc90e868a1bbc8318c1c already pushed, skipping
Pushing tags for rev [8dbd9e392a964056420e5d58ca5cc376ef18e2de93b5cc90e868a1bbc8318c1c] on {https://registry-1.docker.io/v1/repositories/codiez/mongodb/tags/latest}
Pushing f0ab8043e4e8135379d35410a4847769efb9245d8d4817cb24a2196c434c8506

Once that completes, we can start MongoDB in the container, and map it to a port by running:

sudo docker run -d -p 27017 codiez/mongodb /usr/bin/mongod --smallfiles

This basically runs the container, based on the codiez/mongodb image, with the command /usr/bin/mongod and maps the default mongodb port 27017 to an external port.

Check that the container is running and get the port:

1
2
3

dock@saas:~$ sudo docker ps
ID                  IMAGE                   COMMAND             CREATED             STATUS              PORTS
a1f0680f8458        codiez/mongodb:latest   /usr/bin/mongod     5 seconds ago       Up 4 seconds        49157->27017

You can also inspect the image to grab the port by running sudo docker inspect container_id

Now, we can test an external connection to MongoDB, test from your local machine and connect to the VM IP with the port for your container:

1
2
3
4
5
6
7
8
9

$ mongo 192.168.0.21:49157
MongoDB shell version: 2.4.5
connecting to: 192.168.0.21:49157/test
> db
test
> db.posts.insert({title:"Hello MongoDB in Docker"})
> db.posts.find()
{ "_id" : ObjectId("5227058d112c68baaa3b94d9"), "title" : "Hello MongoDB in Docker" }
>

Update: Stop the container and restart it since we do not want to commit with the test data.

1
2

dock@saas:~$ sudo docker stop a1f0680f8458
dock@saas:~$ sudo docker run -d -p 27017 codiez/mongodb /usr/bin/mongod --smallfiles

One final commit to save the command and port mapping. Also notice you do not have to enter in the entire container id when running commands, just the first few characters.

1
2
3
4

dock@saas:~$ sudo docker ps
ID                  IMAGE                   COMMAND                CREATED             STATUS              PORTS
298a43e2f98e        codiez/mongodb:latest   /usr/bin/mongod --sm   35 seconds ago      Up 34 seconds       49164->27017
dock@saas:~$ sudo docker commit -run '{"Cmd": ["/usr/bin/mongod", "--smallfiles"], "PortSpecs": [":27017"]}' 298a4 codiez/mongodb

Now that we have an image, we can run docker pull codiez/mongodb to grab the container and run it with docker run -d codiez/mongodb

The next post will discuss automating the creation of a container with a dockerfile.

Hello Docker

I have been playing around recently with Docker. It could really simplify deployments, creating SAAS services or even creating your own personal PAAS.

What is docker?

Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.

Containers vs Virtual Machines

Virtual Machines: require a complete operating system image, with allocated resources to run. They take a long time to bootup, and have quite a bit of overhead.

Containers: are much more lightweight, since there is no overhead of a complete virtual environment, with the kernel managing the memory and access to the file system. This also means you can bootup an application in seconds.

Install docker

The simplest method would be to use vagrant to set it up on Mac or Linux. (http://docs.docker.io/en/latest/installation/vagrant/)

Setup with vagrant

1
2
3
4

git clone https://github.com/dotcloud/docker.git
cd docker
vagrant up
vagrant ssh

Manual Setup

Docker works best with the 3.8 kernel due to a bug in lxc which can cause some issues if you are on 3.2. run uname -r to check which version you are on, and run the following if you are not on 3.8

1
2
3
4
5

sudo apt-get update

sudo apt-get install linux-image-generic-lts-raring linux-headers-generic-lts-raring

sudo reboot

Then run…

1
2
3
4
5
6
7
8
9
10
11

sudo apt-get update

sudo apt-get upgrade

sudo apt-get install python-software-properties git-core build-essentials ssh

sudo apt-get update

sudo add-apt-repository ppa:dotcloud/lxc-docker

sudo apt-get install lxc-docker

Now, hello world… docker style.

1
2
3
4
5
6
7
8
9

dock@saas:~$ sudo docker run ubuntu /bin/echo hello world
[sudo] password for dock:
Pulling repository ubuntu
Pulling image 8dbd9e392a964056420e5d58ca5cc376ef18e2de93b5cc90e868a1bbc8318c1c (precise) from ubuntu
Pulling image b750fe79269d2ec9a3c593ef05b4332b1d1a02a62b4accb2c21d589ff2f5f2dc (quantal) from ubuntu
Pulling 27cf784147099545 metadata
Pulling 27cf784147099545 fs layer
Downloading 94.86 MB/94.86 MB (100%)
hello world

What just happened?

  • docker downloaded the base image from the docker index
  • it created a new LXC container
  • It allocated a filesystem for it
  • Mounted a read-write layer
  • Allocated a network interface
  • Setup an IP for it, with network address translation
  • And then executed a process in there
  • Captured its output and printed it to you

In the next post, we will discuss setting up a docker container with mongodb.

Uninstall MySQL & MacPorts, Reinstall MySQL With Homebrew on Mac OSX

I have recently have to set up my dev environment once again, which can be quite a bit of a pain, especially if you do not have something like boxen.

This time however i decided to avoid macports all together, one of the “benefits” of macports over homebrew is the ability to run different versions since macports will install into /opt , though its a feature i never used, and i have had a bunch of strange build incompatabilities that has taken me quite a bit of time to debug.

Here are the steps

Uninstall macports

1

sudo port -fp uninstall --follow-dependents installed

1
2
3
4
5
6
7
8
9
10

sudo rm -rf /opt/local
sudo rm -rf /Applications/DarwinPorts
sudo rm -rf /Applications/MacPorts
sudo rm -rf /Library/LaunchDaemons/org.macports.*
sudo rm -rf /Library/Receipts/DarwinPorts*.pkg
sudo rm -rf /Library/Receipts/MacPorts*.pkg
sudo rm -rf /Library/StartupItems/DarwinPortsStartup
sudo rm -rf /Library/Tcl/darwinports1.0
sudo rm -rf /Library/Tcl/macports1.0
sudo rm -rf ~/.macports

Uninstall MySQL

  1. Backup any databases you have using mysqldump
  2. Stop MySQL if it is currently running, under settings > mysql

and then run:

1
2
3
4
5
6
7
8

sudo rm /usr/local/mysql
sudo rm -rf /usr/local/mysql*
sudo rm -rf /Library/StartupItems/MySQLCOM
sudo rm -rf /Library/PreferencePanes/My*
sudo rm -rf /Library/Receipts/mysql*
sudo rm -rf /Library/Receipts/MySQL*
sudo rm -rf /private/var/db/receipts/*mysql*
rm -rf ~/Library/PreferencePanes/My*

You may also need to edit /etc/hostconfig and remove the line MYSQLCOM=-YES-

Install Homebrew & MySQL

1
2

ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"
brew install mysql

Welcome to nirvana

:)

Seriously, i have not yet had any compatibility or weird build errors with homebrew.

Editing MongoDB Arrays in Rails

One of the awesome features of MongoDB is the ability to store arrays directly in your documents, which maps directly with ruby/python/js arrays etc.

An example Post model:

data.json 
1
2
3
4

{
  title: 'Editing MongoDB Arrays in Rails'
  tags: ['ruby', 'rails', 'mongodb']
}

Though when working with MongoDB arrays in rails and forms to edit the data, you could end up doing a bunch of controller logic and/or js logic just to get the data in.

This is especially true if you want to just store a bunch of strings in the array. I tried many different ways of doing this, but the simplest solution with out any changes to your controller or forms is to just use a virtual attribute on your model.

Post.rb 
1
2
3
4
5
6
7
8
9
10
11
12
13
14

class Post
  include Mongoid::Document

  field :title, type: String
  field :tags, type: Array, default: []

  def tags_list=(arg)
    self.tags = arg.split(',').map { |v| v.strip }
  end

  def tags_list
    self.tags.join(', ')
  end
end

Then in your view you just use tags_list:

_form.html.erb 
1

<%= f.text_field :tags_list %>

In the shell:

1
2
3
4
5
6

post = Post.new
post.tags_list = 'tag1, tag2, tag3'
=> "tag1, tag2, tag3"

post.tags
=> ["Tag1", "Tag2", "Tag3"]

Any comma seperated values entered will be stored correctly in MongoDB, you could also use this to apply any transformations to the data before it saved, i.e captilizing each value.

We ended up using this bit of code quite often, so i extracted it out to lib, and then packaged it into a gem.