Minimal CSV import and export for Active Record models.
Provides streaming I/O with batched operations for constant memory usage on large datasets.
Add to your Gemfile:
gem 'csv_record'Extend your Active Record models with CSV Record functionality:
# In a specific model
class User < ApplicationRecord
extend CSVRecord::Methods
end
# Or in Application Record for all inheriting models
class ApplicationRecord < ActiveRecord::Base
extend CSVRecord::Methods
self.abstract_class = true
endThis adds three methods to your models: csv_import, csv_write, and table_empty?.
# Basic import - reads from db/csv/users.csv by default
User.csv_import
# => 1000
# Specify a file path
User.csv_import(filepath: "/path/to/users.csv")
# Custom filename (reads from configured csv_dir)
User.csv_import(filename: "imported_users.csv")
# With options
User.csv_import(
filepath: "/data/users.csv",
batch_size: 5000,
col_sep: "\t",
encoding: "ISO-8859-1",
)How it works:
- Streams CSV rows without loading the entire file into memory
- Uses bulk inserts (
insert_all!) for performance - Wraps import in a transaction (all-or-nothing)
- Returns count of records created
# Basic export - writes to db/csv/users.csv by default
User.csv_write
# => 1000
# Specify a file path
User.csv_write(filepath: "/path/to/export.csv")
# Export with a scope
User.csv_write(scope: User.where(active: true))
# Export specific columns
User.csv_write(column_names: %w[id name email created_at])
# With options
User.csv_write(
scope: User.where("created_at > ?", 1.year.ago),
column_names: %w[id email],
batch_size: 2000,
col_sep: "|",
force_quotes: true,
)How it works:
- Processes records in batches using raw SQL (
pluck) - No Active Record object instantiation
- Memory usage scales with batch size, not total records
- Returns count of records exported
class User < ApplicationRecord
def self.csv_export_excluded_column_names
%w[password_digest session_token]
end
end
User.csv_write
# Exports all columns except password_digest and session_tokenUser.table_empty? # => true, false, or nil (if table doesn't exist)
# Common pattern
User.csv_import if User.table_empty?# Global configuration (e.g., config/initializers/csv_record.rb)
CSVRecord.configure do |config|
config.csv_dir = Rails.root.join("db", "csv")
end
# Or direct assignment
CSVRecord.csv_dir = Rails.root.join("data", "csv")
# Per-model override
class User < ApplicationRecord
CSV_DIR = Rails.root.join("data", "users")
endDirectory precedence:
- Explicit
filepath:parameter - Per-model
CSV_DIRconstant - Global
CSVRecord.csv_dir - Rails default:
Rails.root.join("tmp", "csv") - Fallback:
Dir.pwd
Both csv_import and csv_write accept standard CSV gem options:
# Import options (passed to CSV.foreach)
User.csv_import(
headers: true, # default
col_sep: ",", # field separator
row_sep: "\n", # row separator
quote_char: '"', # quote character
encoding: "UTF-8", # file encoding
converters: [:numeric], # value converters
skip_blanks: true, # skip blank lines
)
# Export options (passed to CSV.open)
User.csv_write(
write_headers: true, # default
col_sep: ",", # field separator
force_quotes: false, # quote all fields
encoding: "UTF-8", # output encoding
)See the CSV documentation for available options.
After checking out the repo, run bin/setup to install dependencies. Then, run rake to run the tests and validate RBS.
Bug reports and pull requests are welcome on GitHub at https://github.com/wtn/csv_record.
The gem is available as open source under the terms of the MIT License.