Lightweight HTTP serving using nginx on OpenShift

Lightweight HTTP serving using nginx on OpenShift

Pre-requisites

I was trying to set up nginx on OpenShift and I'd like to share my experience here. To get started, you need to have an OpenShift account. You can register for a free OpenShift account. I recommend you to install the command line client. Follow the getting started guide to install the client tools and to setup the environment on your machine.

Create OpenShift application

To create an OpenShift application, you can use the client tools that you just installed. Let's name the application as mysite and with diy-0.1 type.

rhc app create mysite diy-0.1

Show the newly created application info:

rhc app show -a mysite

Use the SSH information from the Git URL to SSH into your application.

ssh <random-strings>@mysite-<your-namespace>.rhcloud.com

The environment variables that are available to you in your OpenShift application can be found here. Those environment variables are very useful.

Install nginx

Now we can start the nginx installation. Navigate to the tmp dir and download the nginx source.

cd $OPENSHIFT_TMP_DIR
wget http://nginx.org/download/nginx-1.2.2.tar.gz
tar zxf nginx-1.2.2.tar.gz
cd nginx-1.2.2

If you run

./configure --prefix=$OPENSHIFT_DATA_DIR

directly, you will get the following errors:

checking for PCRE library ... not found
checking for PCRE library in /usr/local/ ... not found
checking for PCRE library in /usr/include/pcre/ ... not found
checking for PCRE library in /usr/pkg/ ... not found
checking for PCRE library in /opt/local/ ... not found
 
./configure: error: the HTTP rewrite module requires the PCRE library.
You can either disable the module by using --without-http_rewrite_module
option, or install the PCRE library into the system, or build the PCRE library
statically from the source with nginx by using --with-pcre=<path> option.

There is no way for us to install the PCRE lib into the system, so we are left with second option which is to build it from the source directly.

cd $OPENSHIFT_TMP_DIR
wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.31.tar.bz2
tar jxf pcre-8.31.tar.bz2

You can check the standard and optional HTTP modules that are supported by nginx here so that you can enable them when configuring the makefile.

cd nginx-1.2.2
./configure --prefix=$OPENSHIFT_DATA_DIR --with-pcre=$OPENSHIFT_TMP_DIR/pcre-8.31

You should be able to see the following output if it's successful. Those information will be needed for you to configure nginx later on.

Configuration summary
  + using PCRE library: /tmp//pcre-8.31
  + OpenSSL library is not used
  + md5: using system crypto library
  + sha1: using system crypto library
  + using system zlib library
 
  nginx path prefix: "/var/lib/stickshift/c45cdc9a27944dc5b1cd7cb9e5c9f8c7/mysite/runtime/"
  nginx binary file: "/var/lib/stickshift/c45cdc9a27944dc5b1cd7cb9e5c9f8c7/mysite/runtime//sbin/nginx"
  nginx configuration prefix: "/var/lib/stickshift/c45cdc9a27944dc5b1cd7cb9e5c9f8c7/mysite/runtime//conf"
  nginx configuration file: "/var/lib/stickshift/c45cdc9a27944dc5b1cd7cb9e5c9f8c7/mysite/runtime//conf/nginx.conf"
  nginx pid file: "/var/lib/stickshift/c45cdc9a27944dc5b1cd7cb9e5c9f8c7/mysite/runtime//logs/nginx.pid"
  nginx error log file: "/var/lib/stickshift/c45cdc9a27944dc5b1cd7cb9e5c9f8c7/mysite/runtime//logs/error.log"
  nginx http access log file: "/var/lib/stickshift/c45cdc9a27944dc5b1cd7cb9e5c9f8c7/mysite/runtime//logs/access.log"
  nginx http client request body temporary files: "client_body_temp"
  nginx http proxy temporary files: "proxy_temp"
  nginx http fastcgi temporary files: "fastcgi_temp"
  nginx http uwsgi temporary files: "uwsgi_temp"
  nginx http scgi temporary files: "scgi_temp"

Now we can compile and install the nginx.

make install

Once it's done, you can navigate to $OPENSHIFT_DATA_DIR where your nginx is installed.

Configure nginx

OpenShift only allow an internal IP address and port for your application which are available through $OPENSHIFT_INTERNAL_IP and $OPENSHIFT_INTERNAL_PORT enviroment variables. And these values may change.

It will be easy if we can simply specify these environment variables directly to nginx.conf by using the env directive. But those env variables can only be referred in the main block of the config, not the http, server or location blocks.

So, we need to do a little trick here to bind the internal IP address and port in nginx configuration dynamically. Let's edit the nginx configuration file:

vi $OPENSHIFT_DATA_DIR/conf/nginx.conf

Modify the listen value to:

http {
    …
    server {
        listen       $OPENSHIFT_IP:$OPENSHIFT_PORT;
        server_name  localhost;
        … 
        }
    …
    }

Rename the modified configuration file.

mv $OPENSHIFT_DATA_DIR/conf/nginx.conf $OPENSHIFT_DATA_DIR/conf/nginx.conf.template

We will modify the $OPENSHIFT_IP and $OPENSHIFT_PORT values when the start action hook is called. I am going to show it to you in the next section.

Final touch up

To start up your application automatically, edit the .openshift/action_hooks/start file. Exit from your ssh session. On your machine,

cd mysite
vi .openshift/action_hooks/start
#!/bin/bash
# The logic to start up your application should be put in this
# script. The application will work only if it binds to
# $OPENSHIFT_INTERNAL_IP:8080
# nohup $OPENSHIFT_REPO_DIR/diy/testrubyserver.rb $OPENSHIFT_INTERNAL_IP $OPENSHIFT_REPO_DIR/diy > $OPENSHIFT_DIY_LOG_DIR/server.log 2>&1 &
# replace the $OPENSHIFT_INTERNAL_IP and $OPENSHIFT_INTERNAL_PORT before starting up the server
sed -e "s/`echo '$OPENSHIFT_IP:$OPENSHIFT_PORT'`/`echo $OPENSHIFT_INTERNAL_IP:$OPENSHIFT_INTERNAL_PORT`/" $OPENSHIFT_DATA_DIR/conf/nginx.conf.template > $OPENSHIFT_DATA_DIR/conf/nginx.conf
nohup $OPENSHIFT_DATA_DIR/sbin/nginx > $OPENSHIFT_DIY_LOG_DIR/server.log 2>&1 &
git commit -a -m "start nginx when starting up the app"
git push

Finally, use your browser again to navigate to http://mysite-.rhcloud.com. The same welcome page earlier will be displayed. And we are done.

Note: Use rhc tail -a mysite command to troubleshoot the problem if you are having problem with the start script.

Tags:

Hi, It's a great tutorial, i've follow step by step based on your tutorial i had problem when git push on the last. here is the error sed: -e expression #1, char 35: unknown option to `s' i have change with single quote and still error, so my web showing 503 error code. please help

thank's

Try using this:
sed -e "s/`echo '$OPENSHIFT_IP:$OPENSHIFT_PORT'`/`echo

HTHs;
~Nam

Sorry but still error, i don't know about sed command. complete script for.openshift/action_hooks/start please.

Thank's

Jefri:

Cut and paste the below into .openshift/action_hooks/start

sed -e "s/`echo '$OPENSHIFT_IP:$OPENSHIFT_PORT'`/`echo $OPENSHIFT_INTERNAL_IP:$OPENSHIFT_INTERNAL_PORT`/" $OPENSHIFT_RUNTIME_DIR/conf/nginx.conf.template > $OPENSHIFT_RUNTIME_DIR/conf/nginx.conf
nohup $OPENSHIFT_RUNTIME_DIR/sbin/nginx > $OPENSHIFT_LOG_DIR/server.log 2>&1 &

Thanks
Sumana

Hi jefri,

Sorry for the encoded characters in the bash script. I didn't escape the backtick. Nam and Sumana have provided the correct sed command. Thanks to Nam and Sumana :)

I simplified the script for for .openshift/action_hooks/start. See below. Hope it helps.

#!/bin/bash
 
TEMPLATE_FILE=$OPENSHIFT_RUNTIME_DIR/conf/nginx.conf.template
CONF_FILE=$OPENSHIFT_RUNTIME_DIR/conf/nginx.conf
IP_PORT=\`echo $OPENSHIFT_INTERNAL_IP:$OPENSHIFT_INTERNAL_PORT\`
 
sed -e "s/\`echo '$OPENSHIFT_IP:$OPENSHIFT_PORT'\`/$IP_PORT/" $TEMPLATE_FILE > $CONF_FILE
nohup $OPENSHIFT_RUNTIME_DIR/sbin/nginx > $OPENSHIFT_LOG_DIR/server.log 2>&1 &

Thank you for all your reply It's works :D

Hi,

Great post! The steps worked very well for me and now I have nginx running :)

The only problem that I encountered was that when nginx redirects requests with 3xx codes, it points the client to appname.rhcloud.com:8080 and redirected browser timeouts waiting for the reply from externally not accessible 8080 port.

To fix this, a following directive needs to be placed in http section of the nginx config file:

port_in_redirect off;

Thanks, Jan

Updates to this blog 11/12/2012:
Updated all references of $OPENSHIFT_RUNTIME_DIR to $OPENSHIFT_DATA_DIR.
Updated all references of $OPENSHIFT_LOG_DIR to $OPENSHIFT_DIY_LOG_DIR

This is due to the typeless gears changes made in relationship to changes in {application name} directory to {cartridge ID} directory. Logs and runtime directories are now tied to {cartridge ID}.

For others that might come across this, I've created a simple openshift repo that downloads, compiles and installs nginx onto a gear. You should be able to include it with an existing repo to get nginx running on the same gear. Lastly, the nginx config file is in .openshift/templates and gets processed by erb everytime the server starts up.

https://github.com/gsterjov/openshift-nginx

I would love to get nginx as a cartridge so I can serve static content out while using gevent/gunicorn to handle app requests. But for the time being this works well enough.

Thanks for posting your quickstart!!

How do I remove or disable nginx?

Erivando Ramos RWD

We are working on making cartridge creations easy; I am sure you may have already read about it, if not, here are the links:

https://www.openshift.com/blogs/new-openshift-cartridge-format-part-1 https://www.openshift.com/blogs/new-openshift-cartridge-format-part-2

Hi

How do I remove or disable nginx?

Erivando Baturité/Brazil

Take a look at the quickstart's build, start and stop hooks. Assuming you added this on top of your existing app, you can disable or remove nginx by modifying build/start/stop and deleting the nginx dir (${OPENSHIFT_HOMEDIR}/app-root/runtime/nginx)

Impressive work, thank you.

More from this author