Build Cloud Enabled Java Redis Applications with Spring on OpenShift

In early 2011, I was thinking about getting started with NoSQL databases but wasn't sure which NoSQL data store I should start with. I wanted to learn a NoSQL database that didn't have a steep learning curve and was generic enough to solve more than one problem. I zeroed down on Redis and started my NoSQL journey. In the last 2.5 years I have worked with many NoSQL databases including MongoDB , HBase, and Cassandra but haven't spent much time with Redis . Recently, OpenShift announced support for community-contributed cartridges. Even though Redis is still not "officially" supported by OpenShift, you can easily use it with the community contributed cartridge. In this blog post, we will start with Redis basics, and then build a Spring Data Redis application, then finally deploy the application on OpenShift.

What is Redis?

Redis(REmote DIctionary Server) is an open source, advanced, NoSQL key value datastore written in the ANSI C programming language. It is an in-memory datastore but writes to disk as well for durability. There are two ways Redis can persist the data – RDB and AOF. RDB persistence performs point-in-time snapshots of your dataset at specified intervals. It is not very durable and you may loose some data but it is very fast. On the other hand, AOF persistence is much more durable and logs every write operation received by the server, that will be played again at server startup, reconstructing the original dataset. Whenever you query the Redis, data is served from in-memory and never from the disk. All operations performed by Redis on keys and values are stored in memory. It follows the server client model where it listens to TCP port and accepts commands. All the commands in Redis are atomic so there are no race conditions. You can work with the same key from different clients and there will be no race condition. If you have worked with Memcached (an in-memory object caching system) you will find that it is very similar, but Redis is Memcached++. Redis servers can also be clustered together to provide for very flexible deployment.

Redis Data Types

The Redis data model is very different from RDBMS and any plain NoSQL key-value data store for that matter. Redis datatypes resemble foundational datatypes of programming languages and feel natural to developers. Every data type supports operations applicable to its type. The supported data types are :

  1. Strings
  2. Lists
  3. Sets
  4. Sorted Sets
  5. Hashes

For more information into Redis data types you can refer to Redis documentation.

Getting Started with Redis

Getting started with Redis is as simple as downloading the tarball, extracting it, and running the make command as shown below.

$ wget http://redis.googlecode.com/files/redis-2.6.7.tar.gz
$ tar xzf redis-2.6.7.tar.gz
$ cd redis-2.6.7
$ make

The binaries that are now compiled are available in the src directory. Run Redis with:

$ src/redis-server
[894] 11 Sep 12:35:22.281 # Warning: no config file specified, using the default config. In order to specify a config file use ./src/redis-server /path/to/redis.conf
[894] 11 Sep 12:35:22.283 * Max number of open files set to 10032
 
[894] 11 Sep 12:35:22.289 # Server started, Redis version 2.6.7
[894] 11 Sep 12:35:22.290 * DB loaded from disk: 0.000 seconds
[894] 11 Sep 12:35:22.290 * The server is now ready to accept connections on port 6379

You can interact with Redis using the built-in client:

$ src/redis-cli
 
redis 127.0.0.1:6379> ping
PONG
redis 127.0.0.1:6379> 
redis 127.0.0.1:6379> 
redis 127.0.0.1:6379> INFO
# Server
redis_version:2.6.7
redis_git_sha1:00000000
redis_git_dirty:0
redis_mode:standalone
os:Darwin 12.0.0 x86_64
arch_bits:64
multiplexing_api:kqueue
gcc_version:4.2.1
process_id:3449
run_id:270454ebad19fbc851194548569efca6ac63e00a
tcp_port:6379
uptime_in_seconds:95
uptime_in_days:0
lru_clock:1407736
....

Developing Spring Redis Applications

Now we are going to discuss how a Java developer can build applications using the Spring Data Redis project. Spring Data for Redis is part of the umbrella Spring Data project which provides support for writing Redis applications. The Spring framework has always promoted a POJO based programming model with a strong emphasis on productivity, consistency, and portability. These values are carried over to Spring Data Redis project as well. It provides an abstraction over the existing Redis client libraries like Jedis, JRedis, Lettuce , SRP, and RJC. Which you choose doesn’t make any difference to your use of the Spring Data Redis library. The differences between the drivers have been abstracted out into a common set of APIs and template-style helpers. In this blog, we will use Jedis driver.

The Spring Data Redis project makes it very easy to work with a Redis key-value datastore by eliminating the boiler plate code required for interacting with Redis. This relieves developers from having to learn the low-level API. It also provides a generified template class named RedisTemplate which is very similar to JDBCTemplate or HibernateTemplate to interact with Redis. RedisTemplate is the main class for object oriented interaction with Redis. It handles object serialization and type conversion so that as a developer you work with Objects and don't not have to worry about serialization and type conversion.

Prerequisite

  1. Basic Java knowledge is required.

  2. Redis 2.6.7 or above installed.

  3. Install the latest Java Development Kit (JDK) on your operating system. You can either install OpenJDK 7 or Oracle JDK 7. OpenShift support OpenJDK 6 and 7.

  4. Download the latest Eclipse package for your operating system from the official Eclipse website.

  5. Sign up for an OpenShift Account. It is free and instant. Red Hat gives every user three free Gears on which to run your applications. At the time of this writing, the combined resources allocated for each user is 1.5 GB of memory and 3 GB of disk space.

Note : Source code of the application is available on github.

Step 1 : Create a Maven Project

After you have installed everything mentioned in the prerequisite section we can start with creating a Maven project using Eclipse. Open the Eclipse application, and go to File > New > Maven Project as shown below.

New Maven Project on OpenShift screenshot

Next it will ask you to select a project name and location. Go with the default settings. After pressing the Next button you will be asked to select Maven archetype. Select "maven-archetype-quickstart" as shown below.

Choose Maven Quickstart Archetype on OpenShift screenshot

Next it will ask you enter the Maven project details as shown below, then press the Finish button.

Define New Maven Project on OpenShift screenshot

Eclipse Maven tooling will import the project into your Eclipse workspace.

Step 2 : Upgrade to JDK 7

The template project created by Maven uses JDK 5 but in our application we will be using JDK 7. So in pom.xml add the Maven compiler plugin and use source and target version as 1.7. Also, in the pom.xml shown below I have updated JUnit dependency to the latest 4.11 version.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
 
  <groupId>com.openshift</groupId>
  <artifactId>blog</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>jar</packaging>
 
  <name>blog</name>
  <url>http://maven.apache.org</url>
 
  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>
 
  <dependencies>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.11</version>
        <scope>test</scope>
    </dependency>
  </dependencies>
 
  <build>
    <plugins>
        <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.1</version>
        <configuration>
          <!-- http://maven.apache.org/plugins/maven-compiler-plugin/ -->
          <source>1.7</source>
          <target>1.7</target>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

After making these changes you have to update the project. To update the maven project, right click on your project and then do Maven > Update Project.

Step 3 : Upgrade to WAR Project

We will be developing a Spring MVC web application so we have to upgrade our project from jar to war project. To do that, we will add the Maven war plugin and change packaging to war as shown below in pom.xml. I have removed some sections from pom.xml for brevity.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
 
  <groupId>com.openshift</groupId>
  <artifactId>blog</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>war</packaging>
 
  <name>blog</name>
  <url>http://maven.apache.org</url>
 
  <dependencies>
    ...
  </dependencies>
 
  <build>
    <plugins>
        ...      
 
        <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-war-plugin</artifactId>
         <version>2.3</version>
         <configuration>
           <failOnMissingWebXml>false</failOnMissingWebXml>
         </configuration>
      </plugin>
    </plugins>
  </build>
</project>

Adding "failOnMissingWebXml" ensures that Maven packages the WAR file even though no "web.xml" is present. From Java EE 6 onwards , web.xml is optional.

Step 4 : Adding Spring MVC Dependencies

Add the following dependencies to the pom.xml dependencies section.

<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-webmvc</artifactId>
    <version>3.2.4.RELEASE</version>
</dependency>
 
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-tx</artifactId>
    <version>3.2.4.RELEASE</version>
</dependency>
 
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-context-support</artifactId>
    <version>3.2.4.RELEASE</version>
</dependency>
 
<dependency>
    <groupId>org.codehaus.jackson</groupId>
    <artifactId>jackson-mapper-asl</artifactId>
    <version>1.9.13</version>
</dependency>
 
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>javax.servlet-api</artifactId>
    <version>3.1.0</version>
    <scope>provided</scope>
</dependency>
 
<dependency>
    <groupId>org.apache.velocity</groupId>
    <artifactId>velocity</artifactId>
    <version>1.7</version>
</dependency>

Step 5 : Configuring Spring Web MVC

Create a new package com.openshift.blog.config and add a new class called WebappConfig as shown below.

package com.openshift.blog.config;
 
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.servlet.config.annotation.EnableWebMvc;
import org.springframework.web.servlet.view.velocity.VelocityConfigurer;
import org.springframework.web.servlet.view.velocity.VelocityViewResolver;
 
@Configuration
@EnableWebMvc
public class WebappConfig {
 
    @Bean
    public VelocityConfigurer velocityConfigurer() {
        VelocityConfigurer velocityConfigurer = new VelocityConfigurer();
        velocityConfigurer.setResourceLoaderPath("/WEB-INF/velocity");
        return velocityConfigurer;
    }
 
    @Bean
    public VelocityViewResolver velocityViewResolver() {
        VelocityViewResolver viewResolver = new VelocityViewResolver();
        viewResolver.setCache(true);
        viewResolver.setPrefix("");
        viewResolver.setSuffix(".vm");
        return viewResolver;
    }
}

In the code shown above :

  1. We annotated the class with @Configuration annotation. @Configuration annotation indicates that class will declare one or more @Bean methods and the Spring container will process the class to generate bean definitions.

  2. We annotated the class with @EnableWebMvc annotation which configured Spring MVC.

  3. We defined couple of beans to use Apache Velocity as our view technology.

Since we are not using web.xml, we have to implement org.springframework.web.WebApplicationInitializer interface. This interface needs to be implemented in Servlet 3.0+ environments in order to configure the ServletContext programmatically -- as opposed to (or possibly in conjunction with) the traditional web.xml-based approach. Create BlogAppInitializer class as shown below.

public class BlogAppInitializer implements WebApplicationInitializer {
 
    @Override
    public void onStartup(ServletContext servletContext) throws ServletException {
        AnnotationConfigWebApplicationContext webApplicationContext = new AnnotationConfigWebApplicationContext();
        webApplicationContext.register(WebappConfig.class, AppConfig.class);
 
        webApplicationContext.getEnvironment().setActiveProfiles("openshift");
 
        Dynamic dynamic = servletContext.addServlet("blogApp", new DispatcherServlet(webApplicationContext));
        dynamic.setLoadOnStartup(1);
        dynamic.addMapping("/");
 
    }
 
}

Step 6 : Configuring RedisConnectionFactory

Before we can configure RedisConnectionFactory we need to add Spring Data Redis dependencies. In your application pom.xml file add the following dependencies.

<dependency>
    <groupId>org.springframework.data</groupId>
    <artifactId>spring-data-redis</artifactId>
    <version>1.1.0.RC1</version>
</dependency>
 
<dependency>
    <groupId>redis.clients</groupId>
    <artifactId>jedis</artifactId>
    <version>2.1.0</version>
</dependency>

RedisConnectionFactory is a thread-safe connection factory for Redis connections. RedisConnection is a short-lived, non-thread safe connection to Redis server. RedisConnection provides one-to-one mapping with Redis commands, whereas RedisConnectionFactory provides convenience methods that help eliminate boilerplate code. RedisConnectionFactory makes switching among the different Redis client APIs as easy as defining a bean. To connect to Redis using Jedis, you need to create an instance of org.springframe work.data.redis.connection.jedis.JedisConnectionFactory.

In this blog we'll use JedisConnectionFactory for the sample application, but you can use any of the other ConnectionFactory variants.

@Configuration
@Profile("openshift")
public class OpenShiftJedisRedisConfig implements RedisConfig {
 
    @Bean
    public RedisConnectionFactory redisConnectionFactory() {
        JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory();
        jedisConnectionFactory.setHostName(System.getenv("OPENSHIFT_REDIS_HOST"));
        jedisConnectionFactory.setPort(
                Integer.valueOf(System.getenv("OPENSHIFT_REDIS_PORT")));
        jedisConnectionFactory.setPassword(System.getenv("REDIS_PASSWORD"));
        jedisConnectionFactory.setUsePool(true);
        return jedisConnectionFactory;
    }
 
}

Step 7 : Configuring RedisTemplate

RedisTemplate provides high level abstraction for performing various redis operations, exception translation and serialization support. It is very similar to the JDBC template or JPA template. This relieves developers from learning low-level API. It is the central abstraction you’re likely to use when accessing Redis via Spring Data Redis. The various operations on data are split up into separate Operations classes.

@Configuration
@ComponentScan(basePackageClasses = { AppConfig.class, BlogRepository.class }, excludeFilters = { @Filter(EnableWebMvc.class) })
public class AppConfig {
 
    @Inject
    private RedisConfig redisConfig;
 
    @Bean
    public RedisTemplate<String, Blog> redisTemplate() {
        RedisTemplate<String, Blog> template = new RedisTemplate<>();
        template.setConnectionFactory(redisConfig.redisConnectionFactory());
        template.setKeySerializer(new StringRedisSerializer());
        template.setValueSerializer(new JacksonJsonRedisSerializer<>(Blog.class));
        template.setHashKeySerializer(new StringRedisSerializer());
        template.setHashValueSerializer(new JacksonJsonRedisSerializer<>(Blog.class));
        return template;
    }
}

In the code shown above, we are setting various serializers because Redis deals directly with byte arrays and doesn’t natively perform Object to byte[] translation. The Spring Data Redis project provides some helper utilities to make it easier to read and write data from Java code. The StringRedisSerializer utility makes our keys and values human-readable from the Redis command-line interface. The JacksonJsonRedisSerializer converts keys and values to human readable JSON format. This is very useful when we want to store an object in redis.

Step 8 : Save Blog to Redis Datastore

Now create a BlogRepository class, as shown below.

@Repository
public class BlogRepository {
 
    @Inject
    private RedisTemplate<String, Object> redisTemplate;
 
    public String save(Blog blog) {
        HashOperations<String, String, Blog> opsForHash = redisTemplate.opsForHash();
        opsForHash.put("blogs", blog.getId(), blog);
 
        return blog.getId();
 
    }
}

As you can see above, you inject RedisTemplate, the core class in the Spring Data Redis project, into BlogRepository.

RedisTemplate provides key-type operations such as ValueOperations, ListOperations, SetOperations, HashOperations, and ZSetOperations. In the code shown above we used HashOperations to store a new blog in the Redis datastore. The put() operation sets field in the hash stored at key to value.

Step 9 : View Blog from Redis Datastore

The next function that we will implement is getting blog from the hash. We will use HashOperations get() method to get the blog for the given id.

 public Blog get(String id) {
        return (Blog) redisTemplate.opsForHash().get("blogs", id);
    }

Step 10 : View Ten Latest Blogs

The last functionality that we need to add to BlogRepository is to view ten latest blogs. To achieve this, we will store the created blog in a list and trim that list to 10 elements.

    public String save(Blog blog) {
        HashOperations<String, String, Blog> opsForHash = redisTemplate.opsForHash();
        opsForHash.put("blogs", blog.getId(), blog);
 
        ListOperations<String, Object> opsForList = redisTemplate.opsForList();
 
 
        opsForList.leftPush("latestBlogs", blog);
        opsForList.trim("latestBlogs", 0, 9);
        return blog.getId();
 
 

Next we will implement latestBlogs() method which will just return all the elements from "latestBlogs" list.

 public List<Blog> latestBlogs(){
        List<Object> objs = redisTemplate.opsForList().range("latestBlogs", 0, -1);
        List<Blog> blogs = new ArrayList<>();
 
        for (Object blog : objs) {
            blogs.add((Blog)blog);
        }
        return blogs;
    }

Step 11 : Add IndexController

The application will render the latest ten blogs when we will go to the index("/") page. Create a new class IndexController as shown below.

package com.openshift.blog.controllers;
 
import java.util.List;
 
import javax.inject.Inject;
 
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.servlet.ModelAndView;
 
import com.openshift.blog.domain.Blog;
import com.openshift.blog.repository.BlogRepository;
 
@Controller
@RequestMapping("/")
public class IndexController {
 
    @Inject
    private BlogRepository blogRepository;
 
    @RequestMapping
    public ModelAndView index(){
        List<Blog> blogs = blogRepository.latestBlogs();
        ModelAndView modelAndView = new ModelAndView();
        modelAndView.addObject("blogs", blogs);
        modelAndView.setViewName("index");
        return modelAndView;
    }
}

The code shown above will fetch the 10 latest blogs from the Redis and add them to ModelAndView. Index velocity template is shown below.

<h2>List of Blogs</h2>
<ul>
    #foreach($blog in $blogs)
        <li><a href="blogs/${blog.id}">${blog.title}</a></li>
    #end
</ul>

Index template extends Layout template shown below.

<!doctype HTML>
<html>
    <head>
        <title>Welcome to Blog Application</title>
    </head>
    <body>
 
        $screen_content
 
        <hr />
        Copyright &copy 2013 Shekhar Gulati
    </body>
</html>

Step 12 : Add BlogController

Next create a new class called BlogController as shown below.

package com.openshift.blog.controllers;
 
import java.util.HashMap;
import java.util.Map;
 
import javax.inject.Inject;
 
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.servlet.ModelAndView;
import org.springframework.web.servlet.view.RedirectView;
 
import com.openshift.blog.domain.Blog;
import com.openshift.blog.repository.BlogRepository;
 
@Controller
@RequestMapping("/blogs")
public class BlogController {
 
    @Inject
    private BlogRepository blogRepository;
 
    @RequestMapping(method = RequestMethod.GET)
    public ModelAndView blogForm() {
        ModelAndView modelAndView = new ModelAndView("create");
        return modelAndView;
    }
 
 
    @RequestMapping(method = RequestMethod.POST)
    public RedirectView createBlog(Blog blog) {
        String blogId = blogRepository.save(blog);
        return new RedirectView("/blogs/" + blogId, true);
    }
 
    @RequestMapping(value="/{blogId}")
    public ModelAndView showBlog(@PathVariable("blogId") String blogId){
        Blog blog = blogRepository.get(blogId);
        Map<String, Object> model = new HashMap<String, Object>();
        model.put("blog", blog);
        return new ModelAndView("show", model);
    }
}

The code shown above does the following :

  1. When a user makes GET request to "/blogs" then it will render form to create a blog.

  2. When a user submits the form, a post request will be made and blog will be saved to Redis using BlogRepository save method. After save, user will be redirected to the blog.

  3. When user makes GET request to view the blog, blog is fetched from BlogRepository and shown to user using show velocity template.

Step 13 : Deploy to OpenShift

Install JBoss Tools OpenShift plugin as mentioned my earlier blog.

Create a new OpenShift application by going to File > New > Other > OpenShift Application and entering details as shown below.

New OpenShift Application screenshot

Next you will be asked to set up a blog and configure server adapter settings. Choose the default and click Next as shown below. We already have a project that we want to use so we will use existing project as shown below.

Setup Existing Project on OpenShift Screenshot

The next screen will ask you to specify the location where you want to clone the git repository and name of the git remote.Choose the default options.

Import Existing Application to OpenShift screenshot

Finally, press the Finish button and you are done. This will create an application container for us, called a gear, and setup all of the required SELinux policies and cgroup configuration.

Next open pom.xml and change the openshift profile as shown below. The profile added by JBoss tools has a bug that it uses outputDirectory as deployments. This works with JBoss EAP and JBoss AS7 cartridges but with Tomcat 6 and Tomcat 7 cartridges output directory should be webapps.

<profiles>
        <profile>
            <id>openshift</id>
            <build>
                <finalName>blog</finalName>
                <plugins>
                    <plugin>
                        <artifactId>maven-war-plugin</artifactId>
                        <version>2.2</version>
                        <configuration>
                            <outputDirectory>webapps</outputDirectory>
                            <warName>ROOT</warName>
                        </configuration>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>

Go to servers view , then right click on blog OpenShift server, and click Publish as shown below.

Publish Changes OpenShift screenshot

This will not work as we have not added Redis cartridge to the application.

Step 14 : Install Redis Cartridge

To install Redis cartridge to our blog application , run the command shown below. This command assumes you already have installed OpenShift rhc command line client tool.

This will add Redis support to our application.

Now just open up your favorite browser and go to http://blog-{domain-name}.rhcloud.com. You will see your application running. To create a new blog go to http://blog-{domain-name}.rhcloud.com/blogs.

Conclusion

In this blog we covered how you can develop Java based Redis applications using Spring Redis project and deploy them to OpenShift. You are now fully equipped to use Redis on OpenShift even though it is still not officially supported. It's fully functional and you can use you it for building your applications. So, if you are a Java (EE) or Spring developer looking for a deployment platform then give OpenShift a try.

What's Next?

Looks good. Do you have a zip or github repo of all the source code?

Here is the Github repository https://github.com/shekhargulati/blog-redis