New OpenShift Cartridge Format - Part 1

New Cartridge Format?

This is the first of a multi-part blog series to discuss, in depth, what cartridges are and how you can create them. This post focuses on high level concepts to get the user thinking about cartridges. As well as some of the differences between the old and new cartridge format.

First, lets look at what a cartridge is and some history behind how we got to where we are. In OpenShift you create an application and those applications consist of gears. For simplicity sake, think of a gear as a user with a home directory that's in a jail. The jail uses a lot of fancy selinux, cgroups and other constraints but you don't really need to be aware of those concepts to create a basic cartridge. You just need to understand there's some places you can write to, some IP's you can bind to and others you can't.

Cartridges are what we put inside your gear to make them useful. If you want to use PHP, you'll need a PHP cartridge. Want to use Jboss you'll need the Jboss cartridge.

How did we get here?

The original cartridge system was designed and brought to us by the team that joined us from Makara acquisition a couple of years ago. The integration of these two systems resulted in the format we're using in OpenShift today. It works great but creating new cartridges is a bit complicated. The new cartridge format builds off of the old cartridge infrastructure but greatly simplifies the requirements to get started.

Those of you that have already been using OpenShift DIY applications will feel right at home with the new format or "v2" as we sometimes call it. That's because the new format was heavily modeled after the DIY setup.

If you've created a cartridge under the "v1" format you're familiar with how complicated the “configure” script is. In the "v1" cartridge format it actually behaves like an installer. So to create a new cartridge typically a user would create an empty gear. Put some service, log directories and config files in the gear. Then write a script to automate everything the cartridge writer just did manually. That script is the configure script.

We realized this was a bit painful for people, it was for us. Why bother writing an installer to create what I just created? Why not just zip it up and re-use it? That was the starting point for the new cartridge format.

First steps

So, what are we really talking about here? What do we really need to do to create an new cartridge? Step one is to pick something to create a cartridge for. In this case we're going to discuss the simplest topic possible, a static Apache web server. For the rest of the series we'll be basing off of this basic static Apache httpd template.

Apache httpd normally has one root process that listens on port 80 and spawns additional processes that run as the apache user to actually service requests. On a RHEL box it has startup scripts in /etc/init.d/, logs in /var/log, binaries in /usr/bin and other misc files all over the filesystem.

Our goal for this cartridge is to make httpd run as a regular Unix user (no root) in a home directory. So lets look at doing this. For the rest of this post lets forget about OpenShift, forget about cartridges. Lets just try to make httpd run inside a home directory as a regular user.

Lucky for us, httpd runs just fine with regular user permissions provided you don't need to bind to port 80. So for our purposes we'll bind to port 8080 instead. How can httpd be setup this way? Going through the httpd docs we see that there's a few config options we'll need to change. Keep in mind at this point we're not trying to create a cartridge, we're just trying to get apache to run, as a regular user on a regular Unix system (in this case RHEL6). In the next blog post we'll turn this work, into a cartridge.

Step 1: Ensure httpd is installed

$ sudo yum install httpd

Step 2: Create a little environment for apache to run in as your user: My user is mmcgrath so we're actually creating a /home/mmcgrath/httpd-example/ directory.

$ mkdir ~/httpd-example/
$ cd ~/httpd-example/
$ mkdir logs run html etc conf.d
  • log/ for logs
  • run/ for pid file
  • html/ for our static pages
  • etc/ for configuration.
  • conf.d for any additional config changes to apache

Next lets populate the config directory. We'll use the system httpd config as an example.

$ cp /etc/httpd/conf/httpd.conf etc/

Now we need to change the system httpd config to suit our needs. We already know that the system httpd binds to port 80. Lets change that to port 8080 on some specific IP:

Listen 80

And replace it with

Listen 127.1.2.3:8080

You may be wondering why 127.1.2.3 instead of 127.0.0.1. This is because in OpenShift the standard loopback address of 127.0.0.1 is restricted. When you get a gear during creation time, we assign you half of a class C in the loopback range. Port 80 is changed to port 8080 for reasons described above, a regular unix user can't bind to port 80 and we're expecting port 8080 anyway. This IP is assigned to you during cartridge creation but we won't get to that until the next blog post.

Now we have to update some paths. First lets update the ServerRoot directory. This tells httpd where to look for things. Since we're running everything out of our home directory, lets specify the full path to our new httpd-examplej.

Find the line that says:

ServerRoot “/etc/httpd”

Replace it with:

ServerRoot “/home/$YOUR_USERNAME_GOES_HERE/httpd-example/”

In my case I replaced $YOUR_USERNAME_GOES_HERE with 'mmcgrath'. This is the user you're logged in as, it'd be whatever shows up with the "whoami" command. The next path change is to our html directory. Find:

DocumentRoot “/var/www/html”

And replace it with

DocumentRoot “html/”

Save that file and we're good to go. The last thing apache needs is the location of all those modules listed in the config file. Before we configure that, lets take a look at what happens when we try to start apache. To do this we'll just call httpd with the -f command and specify the full path to this config file.

$ httpd -f ~/httpd-example/etc/httpd.conf
httpd: Syntax error on line 150 of /home/mmcgrath/httpd-example/etc/httpd.conf: Cannot load /home/mmcgrath/httpd-example/modules/mod_auth_basic.so into server: /home/mmcgrath/httpd-example/modules/mod_auth_basic.so: cannot open shared object file: No such file or directory

This is the kind of error you get when trying to run things from a non-typical directory. It is very common so know what to look for. Sometimes it may be a permission denied trying to write to a bad directory or in some cases the directory could not be found. In this case it's looking for the modules directory and it's just not there. Take a moment to think on why and how you could correct it before moving on.

It turns out there's a couple of things we could do. One is to edit the httpd.conf file again and tell apache where the modules are. We could also make a directory and copy the modules in it. I prefer the symlink method because like httpd on the system, a symlink to the modules ensures that everything stays up to date whenever the httpd package is updated. So, lets create a symlink:

$ ln -s /usr/lib64/httpd/modules/ modules

Note: This assumes you're using 64bit httpd otherwise use /lib/ instead of /lib64/

That's it! Everything should work so lets try this again:

$ httpd -f ~/httpd-example/etc/httpd.conf

No output is a good thing. So, how can we test? If you recall the Listen address we used earlier, we can browse right to it with wget, curl or your favorite web browser:

$ wget -O- http://127.1.2.3:8080/
Connecting to 127.1.2.3:8080... connected.
HTTP request sent, awaiting response... 403 Forbidden
2013-04-01 16:39:36 ERROR 403: Forbidden.

403 error? What and why? Well, this is where it comes in handy to have some expertise in the cartridge you're creating. In this case we'll have to think about what httpd is doing. We can clearly see it responded on the port and IP we were expecting. That's great news, it means things are up and running.

So the next step is to look through the logs:

$ cat logs/error_log
[Mon Apr 01 16:39:36 2013] [error] [client 127.1.2.3] Directory index forbidden by Options directive: /home/mmcgrath/httpd-example/html/

This means that something in the apache configuration isn't allowing the directory index to be displayed. Again like above, we have a couple of options and as the cartridge writer it's your responsibility to decide what experience you want your users to have. We could edit the httpd.conf file and have it allow indexes. Or we could give the html/ directory a proper landing page. Lets do the latter:

$ echo “hello world” > html/index.html

Now instead of a 403 error, we should get a our hello world message:

$ wget -O- http://127.1.2.3:8080/
... snip ...
hello world
2013-04-01 16:54:59 (1.93 MB/s) - written to stdout [12/12]

Now we're talking. Our next step is to figure out how to do this same thing inside OpenShift. It will involve some pre-defined locations for scripts, directory structures and some erb templates. See part 2 to see what that's all about.

If you can't wait and just want to jump right into the documentation. Go ahead, we keep it in our git repo: https://github.com/openshift/origin-server/blob/master/node/README.writing_cartridges.md It will help to have a copy of OpenShift origin or OpenShift enterprise installed.

Link to part 2: https://www.openshift.com/blogs/new-openshift-cartridge-format-part-2

What's Next?

Hi, we need tutorials like this, i'm impatient about next part

Good post!

Hello Mike,

Did I miss the second part? If so, could you please post a link to it on a bottom of Part 1?

Thanks, Boris

Great idea Boris. I've added it and here as well

So what environment are you using to build this in? RHCE, Fedora? I'm using Arch so I don't really have access to Yum. I could probably build this in a VM though I'm wondering if that is even necessary.

Great tutorial, I specially like that you clearly explain how to trouble shoot problems, instead of just giving th solution

You forgot the part where you need to update element in your http.conf to allow access to the html/ directory.

The big issue is as standard user I can't do sudo even yum. All ended up with permission error. If this post is for Fedora only, please say somewhere. I'm not going to wget and unpack different php packages just to get the yum functionality. OpenShift is quite handy, but this is a serious limitation for not Fedora users.

YOu don't need to yum install. What specifically are you running into issues with? Are you just trying to run an application or build a cartridge?

More from this author