Slim-Attributes v0.5.0 released

I just released a new version of slim-attributes.  There are some small speed gains and some other minor changes from 0.4.1, but there are no big changes.Read more about slim-attributes at the slim-attributes homepage, or read on below.IntroductionSlim-attributes is a small patch to the ActiveRecord Mysql adaptor that stops rails from immediately making ruby strings from the column names and data from your database queries. Because you probably don’t need them all!So ruby strings are lazily created on demand – it’s faster and uses less memory. And it drops directly in, requiring only the installation of a gem and adding 1 line to environment.rb.Measuring with just ActiveRecord code – fetching stuff from the database – we see anything up to a 50% (or more) speed increase, but it really depends on your system and environment, and what you are doing with the results from the database.  The more columns your tables have, the better the improvement will likely be.  Measure your own system and send me the results!InstallationTry:

gem install slim-attributes -- --with-mysql-config

or:

gem install slim-attributes

then add this to environment.rb:

require 'slim_attributes'

DescriptionNormally the mysql adaptor in Rails returns a hash of the data returned from the database, one hash per active record object returned by the query. The routine that generates these hashes is called all_hashes, and this is what we replace. The reason for overriding all_hashes is threefold:

  • making a hash of each and every row returned from the database is slow
  • rails makes frozen copies of each column name string (for the keys) which results in a great many strings which are not really needed
  • we observe that it’s not often that all the fields of rows fetched from the database are actually used

So this is an alternative implementation of all_hashes that returns a ‘fake hash’ which contains a hash of the column names (the same hash of names is used for every row), and also contains the row data in an area memcpy’d directly from the mysql API (which is much faster than creating ruby strings).The field contents are then instantiated into Ruby strings on demand – ruby strings are only made if you need them – when you ask for a particular attribute from the model object.Note that if you always look at all the columns when you fetch data from the database then this won’t necessarily be faster that the unpatched mysql adapter.  But it won’t be much slower either, and we do expect that most times not all the columns from a result set are accessed.Future developmentI speculate that further speed gains might be had through keeping the mysql result objects from mysql-ruby around, and not copying the data from them at all until it is needed. However, mysql-ruby limits the non freed result sets to just 20 before calling GC.start, so surgery inside mysql-ruby would be required to achieve this.

Advertisements

Splitting models into several smaller files in Rails

I was reading what Paul Barry had to say about splitting models into smaller files. It resonated with me a little – some of our models are approaching 1000 lines.But I felt the name ‘concerned_with’ did not fully / appropriately describe what is being done, and that there should be an easier way than having to specify every file to be required.So I ended up modifying the code to be a little easier to use.  If you place it in an initializer (i.e. in a file in your initializers directory), then you can specify in your model that you wish to require all the files from a subdirectory of the same name as the model.So if you model is called Customer, then the model file is customer.rb.  Now you can also have a subdirectory called customer that contains further files containing model code.In the original model class file you should add require_class_subdirectory to it, like this:

class Customer  require_class_subdirectory...end

This will cause all the files in the subdirectory to be required.In each file in the subdirectory you should open the model class like so:

class Customer  def something  # you can cut/paste code in from the main model file  end...end

The filenames you use don’t matter – in the above case it could be ‘something.rb’ for instance.So, to recap, your main class file customer.rb has ‘require_class_subdirectory’ added to it. You create a folder called ‘customer’ in your models directory, and place some .rb files in there. In each of those files you re-open the class (‘class Customer’) and place code there just as if you were writing into the main class file.This allows you to separate code according to function within a model, and to keep file sizes manageable.Here is the code to put in the initializer:

class << ActiveRecord::Base  def require_class_subdirectory    ActiveSupport::Dependencies.load_paths.select{|lp| lp =~ /app/models/}.each do |path|      Dir["#{path}/#{name.underscore}/*.rb"].each do |filename|        require_dependency "#{name.underscore}/#{File.basename(filename)}"      end    end  endend

Other approaches to this problem are possible. In particular it may be feasible to patch or hook the constant missing mechanism in rails to automatically load the files in the subdirectory, which would remove the need for the require_class_subdirectory line in your main model file.Finally, not even everyone thinks this is a problem that needs to be solved. My colleague who uses Aptana says it has a good outline mode that means it’s easier to work with one large file for a model than lots of smaller ones. In Textmate I find the smaller files easier.

How to disable the query cache in rails 2.1

If you need to disable the query cache in rails, it’s not particularly easy to do that.There is some discussion about it (dated March 2008) here.Although you can turn all caching off, and you can turn the query cache off explicitly in your code using uncached, there isn’t a way to turn just the query cache off globally at configuration time.So, then, here’s the monkey patch you need (tested on Rails 2.1.1). Although this is not particularly optimal (in that some query caching related code is still called), it will work.  You can put this at the bottom of your environment.rb somewhere, or even better put it in its own file in the initializers directory (e.g. query_cached_off.rb).

module ActiveRecord  module ConnectionAdapters    module QueryCache      private      def cache_sql(sql)        yield      end    end  endend