Chris Love CNM | Tech Blog

Have a need for speed? Want to truly do some crazy stuff with web programming? Give the combination of nginx, lua, and redis a test drive.

So what the heck am I talking about. A couple of years ago I joined a band of motley men build high volume software platform to help out nonprofits on the web. More to come later about that. Doing prelim architecture, we figured out some system needs.

  • Crazy fast web server: nginx
  • Just as crazy fast nosql server: redis
  • Some glue to connect it together: lua

Quick background about these components

Nginx: Started off by Igor Sysoev in 2002, it is one of a few servers addressing C10k problem. I admire about Nginx because how it is based on an asynch event-driven architecture.

Nginx powers several high-visibility sites, such as Netflix, Hulu, Pinterest, CloudFlare, Airbnb, WordPress.com, GitHub, SoundCloud, Zynga, Eventbrite, Zappos, Media Temple, Heroku, RightScale, Engine Yard and NetDNA

Redis: Salvatore Sanfilippo, also known as antirez, created Redis as an in memory advanced key-value store. It is plain simple key based database that can contain strings, hashes, lists, sets and sorted sets. Redis is used by this sites such as: craigslist.org, github.com, guardian.co.uk, Disqus, Pinterest, stackoverflow.com. flickr.com, instagram.com, twitter.com. Lua is supported as a scripting language that can run operations in Redis, and this is how I discovered the wonderful scripting language call Lua.

Lua: The Pontifical Catholic University of Rio de Janeiro in Brazil designs, implements, and maintains this lightweight, super fast scripting language. Lua is actually one of the leading languages used in video games like World of Warcraft and Angry Birds. Xiaozhe Wang and Yichun “agentzh” Zhang maintain a nginx module that allows us to embed Lua scripts inside of Nginx. Since I always want Ferrari performance let’s throw in LuaJit into this stack of toys.

Well enough about what this stuff is, let’s install it.

The install

Most distros such as ubuntu have installs for the majority of these components. You may have to compile lua-nginx-module into nginx, and lua cjson may need to be installed via luarocks.

As Gentoo is my distro of choice here are the additions to the portage files.

make.conf:

NGINX_MODULES_HTTP="autoindex access browser charset empty_gif fastcgi gzip limit_req limit_conn map proxy referer rewrite  ssi userid lua ndk stub_status upstream_ip_hash geoip"

package.use

dev-db/redis jemalloc test
dev-libs/libpcre jit -bzip2 -readline -zlib unicode
www-servers/nginx fastcgi ssl xslt pcre-jit luajit aio

package.accept_keywords

=dev-lang/luajit-2.0.0_beta7 ~amd64
=dev-lua/lua-cjson-2.0.0 **

emerge

emerge redis dev-libs/libpcre lua dev-lua/lua-cjson dev-lang/lua dev-lang/luajit
emerge nginx

Redis client for Nginx

mkdir -p /u01/lua
cd /u01/lua
git clone https://github.com/agentzh/lua-resty-redis

Redis and Nginx should start without any initial configuration. We will need to tweak the nginx conf in order to run lua, but fire up Nginx and point you browser to localhost and voila. Start redis and you can test with:

./redis-cli
 redis> ping

should respond with ‘PONG’.

Other Install Notes

  1. Nginx install notes here outline source install
  2. Redis can be cloned from https://github.com/antirez/redis and installed from source
  3. Luarocks is a simple way to install cjson

Part 2 of this series will cover basic Nginx Lua programming.
Part 3 will cover using redis and Nginx.

No tags

Jul/13

1

Powering This Blog Nginx PHP FPM WordPress

I recently got a nasty notice that my ancient server was part of a Denial of Service attack. Plesk had been hacked. Needless to say it was time to upgrade my virtual server. Since I was upgrading the image, I moved on to a web server that I am most familiar with and have been using in high volume production environments for over two years. A couple of years back I started working with nonprofit, working on a high volume system. After a lot of research we decided to utilize nginx.

From nginx.com

You require breakthrough performance and scalability with seamless support.
We help 25% of the busiest web sites in the world shatter the status quo every day.
Simply put, we deliver 10x more performance at 1/10th the cost.
That’s why NGINX is used by thousands of organizations across Online Media, Retail, Gaming, Telecom, Government, and Cloud Computing.

We have experienced nginx providing amazing performance and incredibly low overhead in both production and during performance testing.

The install for this blog was fairly simple. Most Linux distributions include installs for nginx and php (with php-fpm). Fire up php-fpm running on a unix socket.

Here nginx config for this blog. This is included in my main nginx config with ‘include /etc/nginx/sites-enabled/*;’.

server {
  listen 80; 
  root /var/www/vhosts/chrislovecnm.com;
  index index.php index.html index.htm;
  server_name www.chrislovecnm.com chrislovecnm.com;
  server_name_in_redirect off;
  client_max_body_size 25M;
  access_log /var/log/nginx/chrislovecnm.com.access.log;
  error_log /var/log/nginx/chrislovecnm.com.error.log;

  location ~* \.(ico|pdf|flv|svg|tff|woff)$ {
    expires 1y; 
    access_log off;
  }

  location ~* \.(js|css|xml|txt|png|jpeg|jpg|gif)$ {
    expires 30d;
    access_log off;
  }

  location ~ /\. {
    deny all;
    access_log off;
    log_not_found off;
  }

  location / { 
    try_files $uri $uri/ /index.php;

    location ~* ^/wp-content/uploads/.*.php$ {
      deny all;
      access_log off;
      log_not_found off;
    }   

    location ~ \.php$ {
      try_files $uri =404;
      fastcgi_split_path_info ^(.+\.php)(/.+)$;
      include fastcgi_params;
      fastcgi_index index.php;
      fastcgi_pass   unix:/var/run/php5-fpm.sock;
      fastcgi_cache_valid   200 302  1h; 
      fastcgi_cache_valid   301      1d; 
      fastcgi_cache_valid   any      1m; 
      fastcgi_cache_min_uses  1;  
      fastcgi_cache_use_stale error timeout invalid_header http_500;
    }   
  }
}

No tags

So you are wanting to use git to clone a subversion repository because you know that git rocks and subversion is so 5 years ago! Like a great geek you run:
% git svn clone https://svn.apache.org/repos/asf/camel/trunk/ camel-trunk

But you then you get the following error:

fatal: git was built without support for git-svn (NO_PERL=YesPlease).

No need to nash you teeth you need to add two lines to /etc/portage/package.use file..

dev-vcs/git subversion perl
dev-vcs/subversion perl -dso

No tags

Older posts >>