FastImage finds image dimensions fast using minimal resources

I just released a gem to find image dimensions and type information fast. I have previously done some work in this area, but this is a much more comprehensive solution, and fixes problems with certain jpegs.FastImage finds the size or type of an image given its uri by fetching as little as neededThe problemYour app needs to find the size or type of an image. This could be for adding width and height attributes to an image tag, for adjusting layouts or overlays to fit an image or any other of dozens of reasons.But the image is not locally stored – it’s on another asset server, or in the cloud – at Amazon S3 for example.You don’t want to download the entire image to your app server – it could be many tens of kilobytes, or even megabytes just to get this information. For most image types, the size of the image is simply stored at the start of the file. For JPEG files it’s a little bit more complex, but even so you do not need to fetch most of the image to find the size.FastImage does this minimal fetch for image types GIF, JPEG, PNG and BMP. And it doesn’t rely on installing external libraries such as RMagick (which relies on ImageMagick or GraphicsMagick) or ImageScience (which relies on FreeImage).You only need supply the uri, and FastImage will do the rest.Examples

require ‘fastimage’

FastImage.size(http://stephensykes.com/images/ss.com_x.gif”)
=> [266, 56]  # width, height
FastImage.type(http://stephensykes.com/images/pngimage”)
=> :png

InstallationGem

sudo gem install sdsykes-fastimage -s http://gems.github.com

RailsInstall the gem as above, and configure it in your environment.rb file as below:


Rails::Initializer.run do |config|

config.gem “sdsykes-fastimage”, :lib=>“fastimage”

end

Then you’re off – just use FastImage.size() and FastImage.type() in your code as in the examples.Documentationhttp://rdoc.info/projects/sdsykes/fastimage

Advertisements

Ferret on Ruby 1.9.1

I took the trouble to port ferret to ruby 1.9.1 yesterday evening.  I have it working on my mac. Here’s a gem for you to try – I have labelled it 0.11.6.19.  If you use it let me know how it runs, but it’s at your own risk, I haven’t extensively tested it. [UPDATE: this gem has been updated 5th April 2009 – please test.  There is also a fork at github] I’ve made mostly simple changes in the code:

  • Changed all struct RString -> ptr to use the RSTRING_PTR macro, except for cases where it was being used to add items to an array where rb_ary_store was used.
  • Changed all struct RString -> len to use the RSTRING_LEN macro
  • Changed all struct RArray -> ptr to use the RARRAY_PTR macro
  • Changed all struct RArray -> len to use the RARRAY_LEN macro
  • Removed manual adjustment of the len member of RArray. In fact ruby 1.9 stores small arrays of 3 items or less differently from larger ones, and this adds complexity. It is better to use the rb_ary_store method which will use the correct pointer and will keep the length in sync with the number of items in the array.
  • Changed all struct RHash -> tbl to ntbl
  • Removed references to rb_thread_critical
  • Removed 4th argument from calls to rb_cvar_set
  • Included ruby/re.h and not regex.h, and altered tokenizer code to correctly use the new regexp library
  • Included ruby/st.h and not st.h
  • Some other minor changes to error messages formats causing compiler warnings

By the way, acts_as_ferret also runs with some very minor surgery, Thomas von Deyen has a fork here.

Rails 2.3 breakage and fixage

Rails 2.3 will be with us soon, so I took the time to update our app to be compatible. It’s a reasonably large app (26,000 LOC), so there’s bound to be some issues.The first thing to notice is that the PStore store for sessions has completely gone away. This means that saying something like config.action_controller.session_store = :p_store in your environment file will no longer work.We don’t use the cookie store because our sessions can get bigger than the 4k limit in certain circumstances. So we use the memcache store on the production machines, and pstore on the dev and test machines. And we can’t do that any more, which is a shame as it worked well – particularly with Hongli Lai’s improvements.The remaining options are DRb, Memcached, or SQL. We didn’t want to add complexity to our environments, so none of those looked attractive. So we ended up rewriting some of our code so that the cookie store would be usable in most cases. We’ll keep memcache as the store on the production systems though.Talking of memcache, it seems we now need to include require ‘memcache’ in our production.rb file. It’s not automatically loaded before we want to configure it.The rest of the problems weren’t with the app itself, but with the incredible amount of failed and erroring tests due to changes in the Rails testing system.Firstly all the unit tests were not even running because they all inherited from Test::Unit::TestCase. Nowerdays they need to inherit from ActiveSupport::TestCase, and this is necessary in Rails 2.3.Also make sure your test_helper.rb opens the right class:

ENV[“RAILS_ENV”] = “test”
require File.expand_path(File.dirname(__FILE__) + “/../config/environment”)
require ‘test_help’

class ActiveSupport::TestCase

Next, if you were using assert_valid model_item you must change this to assert model_item.valid?, see here. No deprecation warning in 2.2 that it would be removed, but never mind, the fix is quite easy.We have tests for our routing. They live in the unit tests – it’s handy to test all the routes in one place. But in 2.3 the assert_routing method has disappeared. In fact it’s just not automatically available in unit tests any more, you can retrieve it by doing this in your test class:

include ActionController::Assertions::RoutingAssertions

But the routing assertion also needs clean_backtrace which seems to be part of the ActionController::TestCase. We opted to just define it in test_helper.rb (for a quick fix, just add this code):

  def clean_backtrace(&block)
    yield
  rescue ActiveSupport::TestCase::Assertion => error
    framework_path = Regexp.new(File.expand_path(
                                     “#{File.dirname(__FILE__)}/assertions”))
    error.backtrace.reject! {|line| File.expand_path(line) =~ framework_path }
    raise
  end

We also test cookies in functional tests, and the usage has changed in 2.3. So you’ll need to check through those.If you send multipart emails and have file fixtures (of the expected email contents) to test them, we noticed that instead of just saying Content-Type: text/plain in the header before the mime encoded parts, we now get Content-Type: text/plain; charset=iso-8859-1. Those need to be edited.Finally, if you are using assert_select_email in your tests for your mailer classes, you will find it is also no longer available. The fast solution is to put include ActionController::Assertions::SelectorAssertions in your mailer test class.We have worked around some of the issues presented to us with minimul changes to our code. It seems like Rails is encouraging us to organise our tests differently, particularly where functionality in ActionController::Assertions is no longer automatically available to unit tests. Working around this feels somewhat unclean, so we’ll take a look again whether tests should be moved or rewritten once the dust has settled on 2.3.

Using acts as ferret with phusion passenger / mod_rails

The passenger manual makes it clear that you need to close and reestablish your connections to things like memcached after it forks to avoid inadventently sharing file handles. The reason is well and clearly explained there.The api to do this is simple – just place this kind of code in your environment.rb file:

if defined?(PhusionPassenger)
  PhusionPassenger.on_event(:starting_worker_process) do |forked|
    if forked
      # We’re in smart spawning mode.
      reestablish connections
    else
      # We’re in conservative spawning mode. We don’t need to do anything.
    end
  end
end

All well and good, but how exactly do you reestablish those connections?In our case we have to deal with memcached and ferret (with ferret running in a DRb server via the acts_as_ferret plugin).Memcached is dead easy:

CACHE.reset

Ferret not so easy. It turns out that DRb has no in-built way to close its pool of connections. So a monkey patch is the only thing to do. I was inspired by some code you can find here. But since we want to blindly close all the connections, our case is simpler:

  class DRb::DRbConn
    def self.close_all
      @mutex.synchronize do
        @pool.each {|c| c.close}
        @pool = []
      end
    end
  end

  DRb::DRbConn.close_all

DRb will happily reconnect by itself when needed after its connection pool has been emptied.Putting it all together, it looks like this:

if defined?(PhusionPassenger)
  # monkey patch drb so we can close its connections
  class DRb::DRbConn
    def self.close_all
      @mutex.synchronize do
        @pool.each {|c| c.close}
        @pool = []
      end
    end
  end

  PhusionPassenger.on_event(:starting_worker_process) do |forked|
    if forked
      # We’re in smart spawning mode.
      CACHE.reset  # memcached
      DRb::DRbConn.close_all  # ferret
    else
      # We’re in conservative spawning mode. We don’t need to do anything.
    end
  end
end

Yum refuses to update

I happens that I wish to upgrade apache, and I know there is an update available.So, I follow the advice given:

# yum update httpdSetting up Update ProcessCould not find update match for httpdNo Packages marked for Update

Ok, that’s not working. Let’s try this:

# yum clean allCleaning up Everything# yum update httpdfedora                    100% |=========================| 2.1 kB    00:00     primary.sqlite.bz2        100% |=========================| 5.8 MB    00:02     updates                   100% |=========================| 2.6 kB    00:00     primary.sqlite.bz2        100% |=========================| 2.2 kB    00:00     Setting up Update ProcessCould not find update match for httpdNo Packages marked for Update

No dice.
I browsed the repo, and sure enough the update is there. Why won’t yum find it?
Eventually after a bunch of googling, I found this answer.

# yum upgradeSetting up Upgrade ProcessResolving Dependencies--> Running transaction check---> Package fedora-release.noarch 0:8-6.transition set to be updated--> Finished Dependency ResolutionDependencies Resolved============================================================================= Package                 Arch       Version          Repository        Size =============================================================================Updating: fedora-release          noarch     8-6.transition   updates            31 kTransaction Summary=============================================================================Install      0 Package(s)         Update       1 Package(s)         Remove       0 Package(s)         Total download size: 31 kIs this ok [y/N]: yDownloading Packages:(1/1): fedora-release-8-6 100% |=========================|  31 kB    00:00     Running rpm_check_debugRunning Transaction TestFinished Transaction TestTransaction Test SucceededRunning Transaction  Updating  : fedora-release               ######################### [1/2]   Cleanup   : fedora-release               ######################### [2/2] Updated: fedora-release.noarch 0:8-6.transitionComplete![root@c100586 yum.repos.d]# yum update httpdupdates-newkey            100% |=========================| 2.3 kB    00:00     primary.sqlite.bz2        100% |=========================| 4.6 MB    00:01     Setting up Update ProcessResolving Dependencies--> Running transaction check---> Package httpd.x86_64 0:2.2.9-1.fc8 set to be updated--> Processing Dependency: httpd-tools = 2.2.9-1.fc8 for package: httpd--> Processing Dependency: httpd = 2.2.6-3 for package: httpd-manual--> Processing Dependency: httpd = 2.2.6-3 for package: mod_ssl--> Running transaction check---> Package httpd-tools.x86_64 0:2.2.9-1.fc8 set to be updated---> Package mod_ssl.x86_64 1:2.2.9-1.fc8 set to be updated---> Package httpd-manual.x86_64 0:2.2.9-1.fc8 set to be updated--> Finished Dependency ResolutionDependencies Resolved============================================================================= Package                 Arch       Version          Repository        Size =============================================================================Updating: httpd                   x86_64     2.2.9-1.fc8      updates-newkey    983 kUpdating for dependencies: httpd-manual            x86_64     2.2.9-1.fc8      updates-newkey    832 k httpd-tools             x86_64     2.2.9-1.fc8      updates-newkey     68 k mod_ssl                 x86_64     1:2.2.9-1.fc8    updates-newkey     86 kTransaction Summary=============================================================================Install      0 Package(s)         Update       4 Package(s)         Remove       0 Package(s)         Total download size: 1.9 MIs this ok [y/N]:

That’s better.
Really, it was totally unclear from any documentation that I could find that I would need to run yum upgrade.

Breakage and fixage in Rails 2.2

Finally our app is completely Rails 2.2 ready.Some quick notes on some issues and things that needed to be fixed:1. Default error messagesThis call is no good any more

ActiveRecord::Errors.default_error_messages

Use this instead:

I18n.translate('activerecord.errors.messages')

2. Use ActiveSupport::Inflector rather than InflectorThe warning tells you all you need to know:

DEPRECATION WARNING: Inflector is deprecated! Use ActiveSupport::Inflector instead.

3. Integration tests are broken if you are not using the cookie storeSee here for details.  If you are seeing “NoMethodError: You have a nil object when you didn’t expect it!” inexplicably from your integration tests, then this could be the issue.I ended up placing this in environments/test.rb, even though it should not be needed:

config.action_controller.session = { :session_key => "_myapp_session",   :secret => "some secret phrase of at least 30 characters" }

4. Use of string keys in assert_redirected_to in tests no longer worksConsider this code:

assert_redirected_to "host"=>"foobar.com",   "action"=>"something", "controller"=>"hw"

It used to work, but in rails 2.2 it does not.  You need to use symbols for the keys, like this:

assert_redirected_to :host=>"foobar.com",   :action=>"something", :controller=>"hw"

I think the change was made in this commit.5. Render_partial is goneIf you are still using render_partial in places, you should replace it with render :partial=>”partial_name”6. HAML is not yet compatible with Rails 2.2There is a problem in HAML that causes output from calls to content_tag (and other tag helpers) to be lost.  See this thread for details.If you use HAML I do not recommend upgrading to rails 2.2 until this issue has been sorted out.  However, it’ll probably be fixed in a day or two, and the thread I linked to contains details of the patch I used if you need to fix it before that.7. Components are deprecatedWe had to rewrite some old code that was using components.  It actually wasn’t too much effort in the end, and the resulting refactoring was an improvement anyway.–As ever our tests were extremely valuable during this process.  Once HAML is sorted out we will be upgrading our production server to Rails 2.2, so hopefully that will be in a day or so.