Overriding Paperclip S3 Storage when Testing with RSpec and Cucumber

I have a fairly comprehensive test suite for MobManager that uses both Cucumber and RSpec tests. MobManager also uses Thoughtbot’s Paperclip for file and attachments, using Amazon S3 in production. While Paperclip and S3 are great in production, it’s less than ideal to need an internet connection to run tests that involve models with paperclip attachments. The MobManager test suite takes about 4 minutes to run on my MBP, and I assumed that writing the attachments to disk locally would speed up my tests and allow me to run my tests untethered from the internet.

After Googling around and trying various attempts to stub and mock the S3 calls unsuccessfully, I rolled up my sleeves and came up with this solution. Please let me know if you’ve got a better one.

In previous projects using attachment_fu, I used to do some if/then checking before defining the attachment and migrations. Doing so with Paperclip would have resulted in something like the following: (pseudocodeish)

class Photo < ActiveRecord::Base
  if Rails.root == 'production'
    has_attached_file :attachment, :storage => :s3 # other options here
  else
    has_attached_file :attachment # defaults to file system
  end
end

But this always seemed like an ugly solution. While I was looking at Paperclip’s own S3 cucumber tests for inspiration, I saw that their tests were redefining the has_attached_file for their models. Nice!

Override the call to Paperclip by overriding your model:

# Rails.root/spec/support/photo.rb
# BTW, /spec/support is automatically picked up by spec_helper.rb
class Photo < ActiveRecord::Base
  has_attached_file :attachment # this will now use the file system in testing
end

This allows me to override the Paperclip settings without having to pollute my model with the ugly if/then scenario. Awesome.

To get the same behavior in my Cucumber tests, I copied this line out of spec_helper.rb into my Cucumber env.rb (/features/support/env.rb).

Dir[Rails.root.join("spec/support/**/*.rb")].each {|f| require f}

Close but not quite

Some of my cucumber tests were failing after making this change. Long story short, there’s some kind of garbage collection/temp file race condition, and my model would save before writing the file to disk, and eventually causing my tests to fail.

I found the flush_writes method in the Paperclip storage adapter code and added this hack to forcefully flush the cash when saving the model.

class Photo < ActiveRecord::Base
  has_attached_file :attachment
  after_save :write_attachment

  private

  def write_attachment
    if attachment
      attachment.flush_writes
    end
  end
end

Hooray. All tests pass!

Paperclip doesn’t have an option to write the files to the database, so these new attachments will be written to your Rails.root/public/system directory. Don’t forget to update your .gitignore to exclude these files.

Unfortunately, this didn’t improve the test suite performance at all, but at least I can run the test suite when I don’t have an internet connection.

Testing with Timecop and Postgres - Fixing "Invalid input syntax for type timestamp"

I finally got around to installing postgres on my machine so that I could match heroku's environment.  I fire up my tests and I see the following error:

PGError: ERROR:  invalid input syntax for type timestamp: "-0001-12-31 06:00:00.000000"

Fortunately, the solution is an easy one.

In some of my tests I was using:

Timecop.freeze() do
  # stuff
end

Changing it the following fixed it.

Timecop.freeze(Time.now) do
  # same stuff
end

All tests pass again.

Rails SQLite Adapter handles Time in Date field incorrectly

I was running into a problem with one of my finder methods that tried to find objects with a certain date stamp. The schema shows that the date field is indeed of type date, and not datetime.

I opened up the rails console and began manually trying various queries to diagnose the problem (was it a problem with the query? was it a problem with the scope and the lambda? After some poking around, I was able to determine that the query worked when I had assigned a Date to the field, but not when I assigned a Time to the field.

I fired up sqlite to do more investigating, and sure enough, datetimes were stored in the record, and not just the date. I imagine this is a problem with the rails sqlite adapter, because I don't have this problem in postgres.

I'm going to try to do some looking at the adapter code, although it could be a limitation of sqlite date/date-time types.

Effective Affiliate Marketing

After a blistering start to June, I thought that I was well on my way to another 25% monthly revenue gain. However, it seems that about halfway through the month, the clicks slow down dramatically. I wonder if people spend their paychecks dining out early in the month? Anyways, the result was a June revenue number with 0 growth (actually a 5% decrease.) As we sit here on the first day of July, it once again appears that the clicks are starting to appear again. Nonetheless, I spent some time working on the sites last month, trying some ghetto-SEO to improve ranking for Atlanta and Seattle, also while launching Washington DC. So far, my ghetto-SEO has not seemed to have had any effect, but I think I will have to be a little bit more patient. The biggest news is that after having gone bowling with David Kadavy and a very, very, brief discussion affiliate marketing, I decided that it was worth another shot. I need to generate more revenue, and at this point, I'm definitely not generating more eyeballs. My fear is that at some point, the number of people who don't know about Groupon or Living Social will trend towards zero, and they will stop clicking on the Adsense ads. So, we need to develop different revenue streams.

Ineffective Affiliate Marketing

The first time around, I simply went and got some square advertisements for the affiliates that I thought might be related to the site. I just slapped those ads on the page and through dumb luck, got 1 click through sale on the first day. But after that, nothing, and I felt that having so many square advertisements on the page was going to seriously detract from the user's experience, as well as dilute Adsense revenue. Dilution would come in the form of users clicking on affiliate ads instead of Adsense ads, which return no monetary value unless the user continues on to purchase something. So instead of making a dollar on Adsense for the click, I would get $0. And if the conversion rate on affiliate click throughs is maybe 1%, that's about $100 lost Adsense revenue for a gain in $2 affiliate marketing revenue. A theoretical $98 loss.

Effective Affiliate Marketing

After reintroducing affiliate marketing back onto the site in a specific way, I have now made 3 additional affiliate marketing sales in less than 1 month. That's enough evidence to me to prove that it is not a fluke. So let's take a closer look. If the failed experiment the first time around was too generic (random ads strewn throughout the site), I wanted to try being as specific as possible. I looked up every restaurant listed on the site in Commission Junction to see if it had a Restaurant.com gift certificate. If it did, I added a restaurant specific link to the restaurant's page, as shown in the example below.
Media_httpwwwcodestor_mhdir
This I believed, would catch the website visitor at the time they were most likely to be interested in dining at the restaurant, which in turn would mean they would be interested in saving a little bit more money. Catching them at this moment, I believe is the best place for me to not only get a click-through, but a sale on the affiliate site. The problem with this approach is that it was very labor intensive on my part. I know that CJ has an API, but I have not delved into it deep enough to see if I could automate this whole process. It would be much nicer if I could automate looking a restaurant up on CJ and then adding the link to the site. At this point, injecting the link is also very specific, and I don't trust a VA to do it accurately. It would also take just as much time to verify their accuracy. So until I have a better method, I will just have to suck it up. So, while I spent at least several hours looking up all the restaurants across all the different cities, only a handful of restaurants actually offer the restaurant.com gift certificate. It also appears as if Restaurant.com is not nearly as popular in the other cities I have prix fixe sites in, but the company is also based here in Chicago. The next time I launch a city, I will probably have to spend a little bit more time doing research on this aspect as well.

Affiliate Marketing Can Work for You!

I'm now a 100% believer that affiliate marketing can work for you, if you implement in correctly. By injecting the links to your affiliate sites at the right time, you can indeed make some money off sales referrals! I'm very anxious for Groupon's Beta Affiliate program to start, and I hope that it can also help increase revenue for the site. Affiliate conversion rate also becomes very interesting, and I hope to be able to get into some A/B testing to see if I can increase the click-through and sale conversion rate by changing the wording or colors of the affiliate link.

Kadavy's Passive Revenue Mental Hack

While bowling the other day with David Kadavy, he mentioned a mental hack he uses when analyzing his passive income:
Multiply passive income by 10.
While passive income is fun, for beginners and those just starting out, sometimes it's hard to overlook the fact that you're talking about dollars per day. But, using Kadavy's Passive Revenue Mental Hack may help motivate you to work on increasing your passive revenue stream.