Manual Scaling on OpenShift using Marker Files

Autoscaling

Gondola scaling the mountain as an analogy to autoscaling

VS

Manual Scaling

Scaling the moutain as an analogy for manual scaling

While OpenShift supports auto-scaling, which is awesome in a lot of use cases, there are times when you might want manual control over scaling. Say for example, you knew there was a big rush coming to your site (a sale or a promotion), you could manually scale up a bunch of gears so they would be ready and waiting when the crowds come to your site. From there, as you look at the logs you can monitor the decline in traffic and manually scale down your gears.

To do this it is actually quite simple with OpenShift. The basic idea is remove a marker in the git repo for your scalable application and then issue scaling commands to your application from the command line.

In practice

To begin with, let's make a scalable application. Today I am going to use Python, because, who doesn't love snakes...

rhc app create manualscale python-2.6 -s

Now we have a scalable app running on OpenShift and a local git repo named manualscaling. By default this application will automatically scale up when there are > 16 connections to an individual Python application gear. To read more about how autoscaling works check out Scale or Fail blog post

Turning off autoscaling

Now to prevent OpenShift from scaling based on it's algorithm. Here are the steps:

  1. Go into your git repository named manualscale and then into the .openshift dir and then into the markers directory cd manualscale/.openshift/markers
  2. Next add a marker files named disable_auto_scaling which has no contents. On Mac and Linux:
    touch disable_auto_scaling

On windows you can just create a new file named disable_auto_scaling, by right clicking in explorer and saying new text file, then remove the .txt extension, and you are all set

Now do the following git commands to make sure the settings file is sent.

git add .
git commit -am "disabling autoscaling"
git push

You should see all the normal git and build feedback from your gear. You have just turned off autoscaling for your scalable application.

Command for manual scaling

To manually autoscale you have two options, you can either use the OpenShift Broker REST API or SSH into your gear and execute commands directly on the gear. Today we are going to use the SSH route. Perhaps, if people show interest, I can show how to accomplish the same tasks using the REST API in a future blog post.

SSH'ing into your gear

The simplest way to SSH into your gear is

  1. You are inside the local git repo for your application
  2. Execute this command:  rhc ssh

Voilà, the rhc tool knows about your ssh application and then uses the git information to figure out your account and gear address.

If this doesn't work (usually because you did a manual git clone rather than getting the git repo from the rhc create command), then we need to get the user and url for our gear.

  1. Execute the following command:  rhc app show manualscale

This will produce output that looks like:

manualscale @ http://manualscale-spdemo.rhcloud.com/
  (uuid: 51aba52e4382ec4dce006e6a)
----------------------------------------------------
  Created: Jun 02  1:03 PM
  Gears:   1 (defaults to small)
  Git URL: ssh://51aba52e4382ec4dce006e6a@manualscale-spdemo.rhcloud.com/~/git/manualscale.git/
  SSH:     51aba52e4382ec4dce006e6a@manualscale-spdemo.rhcloud.com
 
  python-2.6 (Python 2.6)
  -----------------------
    Scaling: x1 (minimum: 1, maximum: available) on small gears
 
  haproxy-1.4 (OpenShift Web Balancer)
  ------------------------------------
    Gears: Located with python-2.6
  1. So now just take the id@url from the SSH: line and add it to your ssh command. Please be aware Putty uses a different key system so you will need to convert your private ssh key to a Putty type.

 ssh 51aba52e4382ec4dce006e6a@manualscale-spdemo.rhcloud.com

Now executing scaling command on the gear

Now that we have SSH'ed into our main gear for our application we can use two simple commands to carry out scaling:

  1. To scale up:

 add-gear -a AppName -u UUID -n NameSpace

Which in our case would look like:

 add-gear -a manualscale -u 51aba52e4382ec4dce006e6a -n spdemo

  1. To scale down:

 remove-gear -a AppName -u UUID -n NameSpace

Which in our case would look like:

 remove-gear -a manualscale -u 51aba52e4382ec4dce006e6a -n spdemo

How to see the impact of your commands

If you want to see evidence of the scaling there are two ways to see it. The first is to use the command I showed above from your local machine.

 rhc app show manualscale

After scaling up, the output from this command should show a line that says:

 Gears:   2 (defaults to small)

which shows that we are now consuming two gears. And then after a remove-gear command you should see:

 Gears:   1 (defaults to small)

You can also look at the HAProxy page, since we use HAProxy to handle load balancing and routing for scale apps. All you have to do is go:

 http://yourapp-yournamespace.rhcloud.com/haproxy-status

which in our case would be:

 http://manualscale-spdemo.rhcloud.com/haproxy-status

And with that we are all set to do manual scaling on OpenShift. As always leave any questions or comments down below.

What's Next?

Tags:

You can also use

rhc scale-cartridge python -a manualscale 2

To set an arbitrary scale.

also to clone properly one can use at a later stage rhc git-clone manualscale and ssh can be done like rhc ssh manualscale outside git repo.