update – this needs to have a recent version of RVM – it works with rvm 1.10 but not with 1.6 – do an rvm update first.

When using the mighty cobweb web crawler on my OSX 10.7 with Ruby 1.9.3 I was getting a seg fault in net http:

net/http.rb:799: [BUG] Segmentation fault

A bit of googling and isolation in IRB showed that this can be caused when hitting HTTPS urls using net http.

I saw a few different solutions to the problem, primarily talking about Ruby 1.9.2.Surprisingly it took me a while to end up at the RVM documentation about the problem: https://rvm.beginrescueend.com/packages/openssl/. This didn’t quite work out of the box for me, so I used the solution in the following gist.

rvm pkg install openssl
rvm remove 1.9.3
rvm install 1.9.3 --with-openssl-dir=$rvm_path/usr --with-gcc=clang

and now I can happily crawl https urls.