Wrangling Ruby and ImageMagick in Cygwin
Cygwin can be a clever way for Windows environments to leverage the power of tools that might otherwise only be available in the Unix realm. If you attempt Ruby and ImageMagick stuff in it, though, be prepared for a fight…
- The Problem
- My Specific Problem with Ruby and ImageMagick
- My Solution
- The Steps
Rejected titles for this article include: (a) “It’s not whether you Cygwin or Cyglose: Just quit now!”; (b) “Build a Ruby Goldberg Machine in 146 Easy Steps”; and (c) “Cygwin and Ruby: Forking Hell!”
I won’t explain much of Cygwin other than to say it’s a way to run Unix software natively on Windows. More-precisely, it’s a platform that allows for Unix software to be compiled to target Windows. This is done with shared Cygwin DLLs which seek to provide POSIX-compliant system calls, path mapping, and so-on through a Unix compatibility layer.
While Cygwin is a clever piece of work and a means to an end, I’ve concluded that it is not a suitable way to build a complete development environment, especially where Ruby is involved. You will probably be better off if you switch entirely to (say) Linux, or stick with Windows, optionally with Cygwin playing a minor role in your toolchain.
One great thing about Unix software is the intent that the source code be portable between different Unix flavours. Libraries and conventions are used, as defined by the POSIX standards, such that there are accepted ways that common tasks are usually done. Most free and open Unix software is initially distributed as source code, able to be compiled specifically for the target system. In principle, any Operating System that is at least mostly POSIX-compliant should be able to compile and run this software without issues.
Unfortunately, Windows is not particularly POSIX-compliant, and while Cygwin makes a noble attempt to bridge the gap, there are some things which either haven’t been adequately realised in Cygwin, or will never be possible due to key differences in Windows’ core design.
One issue which comes up regularly is
fork() problems, related to multi-threading.
While Unix software relies heavily
on the fork-exec model, Windows uses a completely different approach which is not
compatible. Cygwin attempts to work around this by ensuring libraries (e.g. DLLs) will
be loaded into fixed, known address spaces between processes, and this requires “rebasing”
of the libraries to prevent collisions (and hence prevent them from relocating).
rebaseallcommand doesn’t always know how to automatically find all libraries/modules that may need treatment in order to be loaded properly;
- Sometimes there simply isn’t enough address space for all libraries; and
- Adding a new Cygwin package may require that you rebase all over again.
My Specific Problem with Ruby and ImageMagick
I’ve found that the
fork() problem has the potential to be significantly harder to deal with
when you’re using Ruby, as each gem may require
its own library to be loaded by the Ruby environment, and it is hard to get the
rebaseall command to consider these.
When I started using the RMagick gem, I hit a wall
and I couldn’t proceed. RMagick provides Ruby bindings for the ImageMagick
image processing libraries. I had intended to use it to process the images of my
blog, producing inline-sized versions and thumbnails from full-sized originals,
while also handling things like web optimisation. Instead, though, I was met with
errors along the lines of the following, at the instant that Ruby tries to load
the RMagick gem via
4 [main] ruby 3004 child_info_fork::abort: C:\cygwin\bin\cygpixman-1-0.dll: Loaded to different address: parent(0x5DF00000) != child(0x2B80000) 2 [main] ruby 284 child_info_fork::abort: C:\cygwin\bin\cygpixman-1-0.dll: Loaded to different address: parent(0x5DF00000) != child(0x5B0000) 1 [main] ruby 4548 child_info_fork::abort: C:\cygwin\bin\cygpixman-1-0.dll: Loaded to different address: parent(0x5DF00000) != child(0x370000) 2 [main] ruby 3052 child_info_fork::abort: C:\cygwin\bin\cygpixman-1-0.dll: Loaded to different address: parent(0x5DF00000) != child(0x2B50000) ...
Supposedly, this problem should all but disappear in a 64-bit version of Cygwin,
but I had problems with that too. Namely,
curl is faulty, and this is critical
to some of the other installation processes on which I rely. So, I had to stick
with the 32-bit version of Cygwin and figure out how to continue without RMagick.
rebase issue is more general than just Ruby and ImageMagick,
it was only since I started loading RMagick in my project that I started having
So, I had considered replacing RMagick with the (more lean) FreeImage
Ruby bindings (i.e. the
and even got it working in a way that I liked… But I then learned that it has no support
for vector images (e.g. SVG files), and I rely on this for many diagrams in my site.
So, what to do?
I decided to rework the RMagick code I had and instead call the
convert command-line utility
provided with a standard installation of ImageMagick. This is a bit of a kludge, but
I simply couldn’t find a way to get RMagick to work properly in my app under Cygwin.
I left the RMagick code in place for other platforms, but routed around it to instead
system("convert ...") when using Cygwin.
By way of an example, the following RMagick code:
# Load the image: pic = Magick::Image.read("my_file.png").first # Resize down (hence trailing ">" after dimensions) to fit a 400x300 space: pic.change_geometry("400x300>") do |cols, rows, img| img.resize!(cols, rows, Magick::CatromFilter) end # Reduce to 64 colours without dithering: pic = pic.quantize(64, Magick::YUVColorspace, Magick::NoDitherMethod) # Write out an 8-bit PNG image: pic.write("png8:output_file.png")
…became something like this:
# Build one long command-line string... # Load the file: cmd = "convert my_file.png " # Resize (scale down, only; not up-size) to fit within a 400x300 space: cmd << "-resize '400x300>' " # Reduce to 64 colours (web optimisation): cmd << "-quantize yuv +dither -colors 64 " # Write out the file as an 8-bit PNG: cmd << "png8:output_file.png" # Now call the command... convert_ok = system(cmd) raise "ERROR: convert command failed with status: $?" unless convert_ok
This documents what I did to install Cygwin from scratch, with some additions, followed by RVM, Ruby, and ImageMagick.
Part 1: Install Cygwin
I had to install the 32-bit version of Cygwin because it looks like there is a
bug with the 64-bit version of
on Cygwin. This leads to problems including:
- Cannot install RVM easily.
So, this is the procedure I followed, influenced by instructions for Installing RVM on Cygwin:
Make sure you are an Administrator when you run the installer, and if you’re using Windows Vista, 7, or above, be sure to right-click the installer and select the “Run as Administrator” option.
Advance through the wizard to the “Local Package Directory” and specify
Advance through to the mirror list and choose any mirror which sounds close to you. In principle, any mirror should do, though occasionally some of them are not completely up-to-date.
In the package list, add the following besides the defaults:
ca-certificates colorgcc (if available) curl gcc (note that this is "Src?" only) gcc-core (this is the compiler) git git-completion git-gui gitk git-svn ImageMagick libMagick-devel libncurses-devel libtool libxml2-devel libxslt libxslt-devel libyaml0_2 libyaml-devel make mercurial (if available) ncurses openssh openssl openssl-devel patch pkg-config readline rsync subversion unzip vim wget zlib zlib-devel
Proceed with the installation and it starts downloading what it needs.
At the end, launch the “Cygwin Terminal” as Administrator and verify it seems to work OK, optionally setting the scrollback count to 20,000 lines or more.
Part 2: Install
devtools and RVM
Run the following commands (noting that the actual paths are important, in this case):
mkdir -p repositories/developwithpassion cd repositories/developwithpassion git clone git://github.com/developwithpassion/devtools cd devtools ./osx_or_cygwin_kick_off
This will have created a file such as
your_windows_user.settings) which you can alter to:
- Specify your GitHub account details (if you have one).
- Comment out all of the
:windows[:paths]options, since they probably are not relevant for your machine (as they were indeed not relevant for mine).
Run this command again:
…which will now use your
...settingsfile to do its install.
You may see some errors along the way, but just ignore them until you see a prompt to
press any key to continue; hit ENTER each time it prompts you.
As it progresses, it will compile a version of Ruby (1.9.3?) and this will take a while; probably more than 15 minutes. Afterwards it will wait for you to hit ENTER again before trying to install Ruby 2.0.0.
NOTE: It may stall while installing Ruby 2.0.0, after this point:
Installing rubygems-2.0.6 for ruby-2.0.0-p247.........(and so on)...... Installation of rubygems completed successfully. Saving wrappers to '/home/anton/.rvm/wrappers/ruby-2.0.0-p247'........... ruby-2.0.0-p247 - #adjusting #shebangs for (gem irb erb ri rdoc testrb rake). ruby-2.0.0-p247 - #importing default gemsets, this may take time.................
After waiting at least half an hour, I hit
CTRL+C. It gave some errors, but I was able to proceed as follows…
Exit the Cygwin Terminal and launch it again. This will ensure the correct environment gets loaded.
At this point, you will probably be on Ruby 2.0.0. Install the latest Ruby 1.9.3 and switch to it:
rvm install 1.9.3 rvm use 1.9.3 --default
Part 3: Install
apt-cyg is loosely equivalent to
apt-get). It is a way to search for, and install packages
from the Cygwin command-line. It is not perfect but can be a convenient way
to install packages on demand or via a script. I recommend installing it, as
it may come in handy.
Run the following commands, which dump
apt-cygdirectly into your path and then make it executable:
svn --force export http://apt-cyg.googlecode.com/svn/trunk/ /bin/ chmod +x /bin/apt-cyg
apt-cygto install something:
apt-cyg install bc
Verify that it worked:
NOTE: If you retrospectively install any packages with
(or the Cygwin
for that matter), you may need to take the following steps, if you start
getting errors related to
Exit out of any Cygwin processes, including the Cygwin Terminal.
Launch a Windows Command Prompt, “As Administrator”.
c:\cygwin\bin\dash -c '/usr/bin/rebaseall'
Part 4: Enable OpenSSH server
This assumes you want to be able to SSH into your machine and use the Cygwin command-line (i.e. your default shell in Cygwin). Being able to do so is quite convenient, but not required in this process, and may pose security risks depending on your circumstances.
The steps below are based on these instructions:
…and when prompted, supply any password for the
cyg_serveruser that it will create.
Log in as each user that will need SSH access (or just stay as the current user), and per each run:
…hitting ENTER to avoid entering a passphrase for each key.
cygrunsrv -S sshd
You should now be able to SSH into your host, with your Windows username and password.
NOTE: Doing this will probably reveal a new “Privileged server” user on your Windows login screen. For more info, see: here.
Part 5: Fix
vim is my preferred editor. If you don’t use it, you can skip this section.
Grab a template that fixes cursor keys:
cp /usr/share/vim/vim73/vimrc_example.vim ~/.vimrc
Add the following to it for other enhancements:
" Add these if needed: " filetype plugin indent on " syntax on " Two-space soft tabs, always: set tabstop=2 set softtabstop=2 set shiftwidth=2 set expandtab " Line numbers: set number " Disable backup files: set nobackup
Part 6: Set yourself up for GitHub
Again, these steps are only required if you plan to use
git (and GitHub).
Create a place where project repos can be kept. For example:
cd ~ mkdir /cygdrive/c/projects ln -s /cygdrive/c ln -s c/projects
git config --global color.ui true git config --global user.name "Your Full Name" git config --global user.email "email@example.com"
Part 7: Create an example Ruby project
Create a directory for your project:
cd ~/projects mkdir mydemo cd mydemo
Create files that tell RVM which Ruby version to use, and that you want a specific set of gems to be tracked for this project:
echo "1.9.3" > .ruby-version echo "mydemo-gemset" > .ruby-gemset
Change out of, and back into the directory, to get it to reload the RVM config:
pushd ~ popd rvm list gemsets
…the final line of which should show something like:
=> ruby-1.9.3-p448@mydemo-gemset [ i386 ]
gem install bundler
Gemfilewhich is used by Bundler, for now just to select v1.1.1 of the
source 'https://rubygems.org' gem 'adsf', '1.1.1'
Test Bundler, getting it to read and respect
Test that it worked:
which adsf # /home/myuser/.rvm/gems/ruby-1.9.3-p448@mydemo-gemset/bin/adsf
If you get
child_info_fork errors, try:
Run this command, to find any modules (i.e. that may have been installed by Ruby gems):
find ~ -iname '*.so' > /tmp/rubygems.so.list
- Exit any Cygwin Terminals or processes.
Go in to an Administrator command prompt and run:
c:\cygwin\bin\dash -c '/usr/bin/rebaseall -T /tmp/rubygems.so.list'