things learned from data migration

Spend a week for data migration.  Here are some things I learned:
1.

     class OldScenario < ActiveRecord::Base
            establish_connection configurations['old_db']
            set_table_name(connection.current_database + "." + "scenarios")
            has_many :old_page_groups, :order=>:created_at, :foreign_key => "scenario_id"
           
            def old_page_group
              self.old_page_groups.find_all_by_version("draft", :order => "created_at DESC")[0]
            end
     end
    debugger

 

  Above is code for create model "OldScenario" and link to old_db:scenarios. I made a lot of model like this.
But how to make sure all of the establishes are successful? I like to put a debugger here. It is very convenient to test.

2.
PageGroup.connection.execute("update page_groups set type='Preparation'")
is code to execute sql instead of rails. It is faster and works well here, but never use this in other conditions like in model.
here is a example:

self.page_parts.each do part|
        #do this do not use active_record because of observer
        #PagePart.connection.execute("UPDATE page_parts SET page_id=null WHERE id=#{part.id}")
        #part.update_attributes(:page_id=>nil)
        PagePart.update_all("page_id = NULL", {:id=>part.id}) #code from JP, update_all avoid the call back
end

 

it will cause wired bug because cache of rails.

use 

PagePart.update_all("page_id = NULL", {:id=>part.id})

 

3.
  If database is very big(like have 400000 more records), do not load all data together. It will be very slow and may crash the server. Below is code from seven, it solve this kind of problem well.

0.step(length, 1000) do |t|
    Page.find(:all, :conditions => "id>#{t} and id<=#{t}+1000").each do |page|
      page.destroy if page.page_groups.blank?
    end
end

A better way is use "find_in_batches" (from andy)