Wednesday, October 27, 2010

Ruby comments in vim

Put this in your .vimrc

- will now comment a visual block, _ will uncomment the entire visual block. Enjoy.
map _ :s/^\/\/\\|^--\\|^> \\|^[#"%!;]//:nohlsearch
map - :s/^/#/:nohlsearch

Wednesday, October 20, 2010

<< vs +=

So I was assigned a story today at work that called for parsing a 80k+ row csv file, and adding some data to it, then throwing it into a model. Simple enough. My first run just to get it running looped over the rows and called model.create for each one. This was obviously pretty slow. It had to create a new object for every row in the csv file. So I re factored and decided to build one very large sql insert query string. The code looked something like this:

sql = "INSERT INTO table (column_1) VALUES "
rows.each do |row|
sql += "(row.value),"
end

To my surprise when I benchmarked it verses the previous method it was slower! After almost giving up on the insert statement I changed += to <<. I re-ran the code and expected minimal speed increase. To my surprise the code went from taking about 2 hours to about 2 minutes!

I think the speed increase comes from += creating a new string object every time, whereas
<< literally concatenates it. Here are some benchmarks.
>> slow = Benchmark.measure {
?> str = ""
>> 100000.times {
?> str += "a"
>> }
>> }
=> #

>>
?> fast = Benchmark.measure {
?> str = ""
>> 100000.times {
?> str << "a"
>> }
>> }
>> puts slow
86.310000 0.400000 86.710000 ( 86.780992)
=> nil
>> puts fast
0.030000 0.020000 0.050000 ( 0.049042)
=> nil
>>

86 seconds vs .04 seconds