Saturday, August 23, 2014

How to render image serverside using OpenShift Node.js?

Hey everyone,

Leaving a developer world in favor of non-coding job has certain advantages.
And one big disadvantage - no more coding. That is a big problem and I lasted about 6 months before I started created something after hours - just for my pleasure.

Things have changed significantly since I was studying. Back then it was a big problem (especially for a poor student) to rent a server and static IP address. And today, with the presence of Weebly and Openshift (a really great Platform As A Service) you can get up your small portal (like mine) in less than a couple of hourse and 100$.

Openshift provides a great array of technologies - I have chosen Node.js + MongoDB as it offers the quickest route to a working app.

One big problem I had was that I wanted to render images server side. This is not trivial as there is no javascript libraries that could do that. One can rely only on native software, but if you consider semi-automated PaaS and customer server-side solution, you will feel the pain.

So I'm sharing with you my setup, as I got it working - it renders images using node.js canvas module, that uses underlying Cairo library.

Installing Cairo on OpenShift gear is not so simple - it is necessary to build cairo manually:

export PATH=/sbin:$PATH
export LD_LIBRARY_PATH=$OPENSHIFT_DATA_DIR/usr/local/lib:/opt/rh/nodejs010/root/usr/lib64:$LD_LIBRARY_PATH
export PKG_CONFIG_PATH=$OPENSHIFT_DATA_DIR/usr/local/lib/pkgconfig

cd $OPENSHIFT_DATA_DIR
curl -L http://sourceforge.net/projects/libpng/files/libpng15/1.5.18/libpng-1.5.18.tar.xz/download -o libpng.tar.xz
tar -Jxf libpng.tar.xz && cd libpng-1.5.18/
./configure --prefix=$OPENSHIFT_DATA_DIR/usr/local
make 
make install

cd $OPENSHIFT_DATA_DIR
curl http://www.ijg.org/files/jpegsrc.v8d.tar.gz -o jpegsrc.tar.gz
tar -zxf jpegsrc.tar.gz && cd jpeg-8d/
./configure --disable-dependency-tracking --prefix=$OPENSHIFT_DATA_DIR/usr/local  
make
make install

cd $OPENSHIFT_DATA_DIR
curl http://www.cairographics.org/releases/pixman-0.28.2.tar.gz -o pixman.tar.gz  
tar -zxf pixman.tar.gz && cd pixman-0.28.2/  
./configure --prefix=$OPENSHIFT_DATA_DIR/usr/local   
make 
make install

cd $OPENSHIFT_DATA_DIR
curl http://public.p-knowledge.co.jp/Savannah-nongnu-mirror//freetype/freetype-2.4.11.tar.gz -o freetype.tar.gz
tar -zxf freetype.tar.gz && cd freetype-2.4.11/  
./configure --prefix=$OPENSHIFT_DATA_DIR/usr/local   
make 
make install 

cd $OPENSHIFT_DATA_DIR
curl http://cairographics.org/releases/cairo-1.12.14.tar.xz -o cairo.tar.xz  
tar -xJf cairo.tar.xz && cd cairo-1.12.14/  
./configure --disable-dependency-tracking --without-x --prefix=$OPENSHIFT_DATA_DIR/usr/local 
make 
make install

After that it is necessary to hack package.json file - it cannot contain canvas in it, as paths are wrong, and canvas will never discover cairo in non-standard location. Canvas has to be installed using build hook:

export PATH=/sbin:$PATH
export LD_LIBRARY_PATH=$OPENSHIFT_DATA_DIR/usr/local/lib:/opt/rh/nodejs010/root/usr/lib64:$LD_LIBRARY_PATH
export PKG_CONFIG_PATH=$OPENSHIFT_DATA_DIR/usr/local/lib/pkgconfig

cd $OPENSHIFT_REPO_DIR
scl enable nodejs010 v8314 'npm install canvas'    

And a really dirty hack. OpenShift tends to cache installed modules, which is a bit of a problem, because when scripts are restored, they know nothing about where the cairo is installed. This simple pre_build hook is reverse engineered from the OpenShift code - I remove cached modules, and therefore force canvas installation by my script.

rm -rf "${OPENSHIFT_NODEJS_DIR}/tmp/saved.node_modules"

I hope this will help someone trying to run canvas module in OpenShift - it took a couple of long hours to figure out what is going on.

7 comments:

  1. Cool! Any idea why Cairo would thinks image surface backend feature can't be enabled -- i.e., it can't tell pixman is installed?

    ReplyDelete
    Replies
    1. I *guess* it is because of paths.
      Cairo does not check whether pixman is installed in the registry, it does look for a certain libraries only in specific folders. Figuring out what those folders are and where to put/link pixman may be non-trivial.

      Delete
  2. Thanks, this was exactly what I needed!

    ReplyDelete
  3. Did you ever see this error? I am following your instructions on my gear and getting this error.

    Error: libpng15.so.15: cannot open shared object file: No such file or directory
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object. (/var/lib/openshift/5473dbe3ecb8d47187000167/app-root/runtime/repo/node_modules/canvas/lib/bindings.js:2:18)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)

    ReplyDelete
  4. Can't tell for sure but it looks like the libpng is not there. Could you try to locate it? Maybe the build script (first one in my blogpost) ended with an error?

    ReplyDelete
    Replies
    1. Yes found issue needed to add a pre_restart_nodejs and pre_restart_nodejs handlers where path is set..

      echo "setting up path for libraries"
      export LD_LIBRARY_PATH=$OPENSHIFT_DATA_DIR/usr/local/lib:/opt/rh/nodejs010/root/usr/lib64:$LD_LIBRARY_PATH
      export PKG_CONFIG_PATH=$OPENSHIFT_DATA_DIR/usr/local/lib/pkgconfig


      I am getting a different error now in openshift...

      stream.js:94
      throw er; // Unhandled stream error in pipe.
      ^
      Error: the surface type is not appropriate for the operation
      at /var/lib/openshift/5473dbe3ecb8d47187000167/app-root/runtime/repo/node_modules/canvas/lib/pngstream.js:42:19
      at process._tickDomainCallback (node.js:459:13)

      Delete
    2. Hi Justin,
      I hope to revalidate my blogpost over the weekend. I've looked at the code and I have following files:
      .openshift/action_hooks/build with the content described in the blogpost (exports and canvas installing) and .openshift/action_hooks/pre_build which removes the cache:
      rm -rf $OPENSHIFT_DATA_DIR/../app-root/repo/node_modules/*
      rm -rf "${OPENSHIFT_NODEJS_DIR}/tmp/saved.node_modules"

      I think that pre_restart_nodejs is not a right part, as paths may be lost/not passed properly *after* restart.

      HTH.

      Delete