Paperclip and Amazon S3 Issue

Question

I have a rails app running on Heroku. I am using paperclip for some simple image uploads for user avatars and some other things, I have S3 set as my backend and everything seems to be working fine except when trying to push to S3 I get the following error:

The AWS Access Key Id you provided does not exist in our records. 

Thinking I mis-pasted my access key and secret key, I tried again, still no luck. Thinking maybe it was just a buggy key I deactivated it and generated a new one. Still no luck.

Now for both keys I have used the S3 browser app on OS X and have been able to connect to each and view my current buckets and add/delete buckets. Is there something I should be looking out for? I have my application's S3 and paperclip setup like so

development:
  bucket: (unique name)
  access_key_id: ENV['S3_KEY']
  secret_access_key: ENV['S3_SECRET']

test:
  bucket: (unique name)
  access_key_id: ENV['S3_KEY']
  secret_access_key: ENV['S3_SECRET']

production:
  bucket: (unique_name)
  access_key_id: ENV['S3_KEY']
  secret_access_key: ENV['S3_SECRET']

has_attached_file :cover,
    :styles => {
      :thumb => "50x50"
    },
    :storage => :s3,
    :s3_credentials => "#{RAILS_ROOT}/config/s3.yml",
    :path => ":class/:id/:style/:filename"

EDIT NOTE: The ENV['S3_KEY'] and ENV['S3_SECRET'] are environment variables in heroku which i have tried even using my keys directly and it still doesn't work

Note: I just added the (unique name) bits, those aren't actually there--I have also verified bucket names, but I don't even think this is getting that far. I also have my heroku environment vars setup correctly and have them setup on dev

Answer 1

You aren't setting a bucket. It's in your s3.yml file, but you aren't reading that value from your call to has_attached_file.

Paperclip S3 docs: http://rubydoc.info/gems/paperclip/Paperclip/Storage/S3#s3_protocol-instance_method

Also, pay attention to those people who are telling you not to use a s3.yml file with Heroku. It's a waste and just added abstraction that buys you nothing. You already have your ENV set up with the values you need, so use them.

I've done this before where I don't want to push an s3.yml file to Heroku, but I do want to use one for test and development. In an initializer you can do something like this:

# If an s3.yml file exists, use the key, secret key, and bucket values from there.
# Otherwise, pull them from the environment.
if File.exists?("#{Rails.root}/config/s3.yml")
  s3_config = YAML.load_file("#{Rails.root}/config/s3.yml")
  S3[:key] = s3_config[Rails.env]['key']
  S3[:secret] = s3_config[Rails.env]['secret']
  S3[:bucket] = s3_config[Rails.env]['bucket']
else
  S3[:key] = ENV['S3_KEY']
  S3[:secret] = ENV['S3_SECRET']
  S3[:bucket] = ENV['S3_BUCKET']
end

Then when you're setting up Paperclip in your model, you reference the value like this:

...
:s3_credentials => {
  :access_key_id => S3[:key],
  :secret_access_key => S3[:secret]
},
:bucket => S3[:bucket]

Obviously, this means that you do not want to have your s3.yml file in your git repository (which really, you shouldn't anyway).

Answer 2

I kept getting the same AWS::S3::InvalidAccessKeyId error, and had a very similar s3.yml file. As x1a4 recommended, I used ERB in my yaml file and it worked. Here's what it looks like now:

# myapp/config/s3.yml

development: &DEFAULTS
  bucket: myapp_dev
  access_key_id: <%= ENV['S3_KEY'] %>
  secret_access_key: <%= ENV['S3_SECRET'] %>

test:
  <<: *DEFAULTS
  bucket: myapp_test

production:
  <<: *DEFAULTS
  bucket: myapp

staging:
  <<: *DEFAULTS
  bucket: myapp_staging

I guess this might a tad too indirect for some folks, but it seemed like the cleanest implementation to me.

Answer 3

Your s3 yaml file is actually using the strings ENV['S3_KEY'] and ENV['S3_SECRET'] as the auth info for s3. They are not being evaluated as ruby code.

There are a couple of things at least that you can do outside of putting that actual info into the yaml file. You can look into enabling ERB in your yaml configs or just not use a yaml file for your credentials at all, because you're always pulling from the environment in every one of your rails_envs, so the yaml file is just an extra layer of indirection in your case that is useless.