Tag Archives: Rake

ROR | Rake task for Data Import

Rake task for Data Import

Export data from external source in a custom format like so –

mysql> select employee_no, username into outfile ‘/tmp/projX_users.txt’ FIELDS TERMINATED BY ‘,’ OPTIONALLY ENCLOSED BY ‘”’ LINES TERMINATED BY ‘\n’ FROM users;

Save the external data text file as a CSV in excel.

Create an import function on User model:

CSV.foreach(filename, :headers => false) do |row|

# Process each row from csv and create user object

user.create!(row.to_hash)

end

In a new rake file for custom tasks- /lib/tasks/custom_tasks.rake

namespace :import  do

desc “Import from external data sources”

task :users => :environment do

User.import_users Rails.root.to_s + ‘/db/users.csv’

end

end

Run rake task:

rake import:users —trace RAILS_ENV=production

Don’t miss passing the environment as we have declared it as dependency in our rake task definition

References:

http://stackoverflow.com/questions/363084/mysql-how-would-i-export-tables-specifying-only-certain-fields

http://stackoverflow.com/questions/4410794/ruby-on-rails-import-data-from-a-csv-file

http://stdlib.rubyonrails.org/libdoc/csv/rdoc/classes/CSV.html