Uploading File to Amazon S3 Using Plupload on Rails

PLUpload is a great freee javascript library that let’s you upload the file to the server, either it’s a real server or a storage cloud service like Amazon S3. It provides various upload interface through its runtimes (a frontend interface unit that handle file uploading). There are 6 runtimes available which are HTML4, HTML5, Flash, Silverlight, Gears and Browserplus. Each runtime has different characteristic. For example, Flash runtime lets you select the file from file explorer dialog, while HTML5 runtime also support drag-and-drop.

Plupload is used in one the projects I’m working with. At first, we use Plupload to upload the file to Heroku. However, there are needs from the client for large file upload support. This is quite a problem with Heroku as it requires the request to be finished within 30 second. This old upload is supported by Paperclip. When the user uses PLUpload to upload the file, it goes to Heroku, then Paperclip handles the post-processing task and uploads the file further to Amazon S3. If the users had to upload the large file, that file can not be passed to Heroku due to its time-out critiria. Thus, it will not be uploaded to S3 too. We need to change the behavior of this procedure.

The solution we can figure out is to use Plupload to upload the file directly to S3, then let’s Paperclip do post-processing as background job (This will be written in the next coming post). One thing to remind is that, currently, S3 does not support file uploading via HTML5. As a result, we need to select other runtimes. The best choice in my opinion is Flash even if it seemed outdate. The following code shows how the configuration is done.

index.html.erb
1
2
3
4
5
<div class="upload_container" id="media-upload">
  <span id="browse-prompt">Click here to upload new file</span> <span class="button altbutton" id="browser">browse</span> </div>
  <div id="progress" style="display:none"></div>
  <div id="uploading-item" style="display:none;"></div>
</div>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
<code language="javascript">
$(document).ready(function(){
  uploader = new plupload.Uploader({
    runtimes: 'flash,silverlight',
    container: 'media-upload',
    browse_button: 'browser',
    max_file_size: '200mb',
    flash_swf_url: '/javascripts/lib/plupload/plupload.flash.swf',
    silverlight_xap_url: '/javascripts/lib/plupload/plupload.silverlight.xap',
    url: gon.temporaryFileLocation,
    multipart: true,
    multipart_params:{
      'key': 'your/folder/${filename}', // for example 'media/tmp/${filename}'
      'Filename': '${filename}',
      'acl': gon.acl,
      'Content-Type': 'binary/octet-stream',
      'success_action_status': '201',
      'AWSAccessKeyId' : gon.accessKeyId,
      'policy': gon.policy,
      'signature': gon.signature
    },
    file_data_name: 'file',
    filters: [
      {title: "Image files", extensions: "jpg,jpeg,gif,png"},
      {title: "Video clips", extensions: "mov,mpg"},
      {title: "Executable files", extensions: "exe"},
      {title: "Compressed files", extensions: "rar,zip"},
      {title: "Document files", extensions: "pdf,ppt,pptx,doc,docx,xls,xlsx"}
    ]
  });

  uploader.init();

  uploader.bind('Error', function(up, args) {
    up.refresh();
  });

  uploader.bind('FileUploaded', function(up, file, response) {
    notice_uploaded_file(file.name);
    $("#uploading-item").fadeOut("fast");
    if (up.total.uploaded == up.files.length){
        finished_all_file = true;
    }
  });

  uploader.bind('FilesAdded', function(up, files) {
    uploader.start();
    $('#progress').progressbar();
    $('#progress').show();
    $('#progress').progressbar('value', 10);
    up.refresh();
  });

  uploader.bind('BeforeUpload', function(up, file){
    file_word = function(i){ if(i==1){return " file ";}return " files ";};
    file_upload_log = up.total.uploaded +  file_word(up.total.uploaded) +
                      "uploaded from "+ up.files.length + file_word(up.files.length) + "total";
    $("#uploading-item").html("Uploading "+
      file.name + "<br/>(<span id='progress_text'>0%</span>, "+
      file_upload_log + ")"
    );
    $("#uploading-item").fadeIn("slow");
  });

  uploader.bind('UploadProgress', function(up, file) {
    $('#progress').progressbar('value', file.percent);
    $("#progress_text").html("current progress " + file.percent + "%");
    up.refresh();
  });
});
</code>

<b>And Rails controller</b>
<code language="ruby">
def index
  access_key_id     = "your access key id"
  secret_access_key = "your secret access key"
  acl               = "your acl"
  max_file_size     = "100"
  expired_date      ||= 10.hours.from_now.utc.iso8601

  s3_bucket = "your bucket"
  gon.temporary_file_location = "http://s3.amazonaws.com/#{AppConfig[:s3_bucket]}/"
  gon.acl = acl
  gon.access_key_id = access_key_id
  gon.policy = Base64.encode64(
    "{'expiration': '#{expired_date}',
      'conditions': [
        {'bucket': '#{s3_bucket}'},
        {'acl': '#{acl}'},
        {'success_action_status': '201'},
        ['content-length-range', 0, #{max_file_size}],
        ['starts-with', '$key', ''],
        ['starts-with', '$Content-Type', ''], 
        ['starts-with', '$name', ''],
        ['starts-with', '$Filename', '']
      ]
    }").gsub(/\n|\r/, '')

  gon.signature = Base64.encode64(
    OpenSSL::HMAC.digest(
      OpenSSL::Digest::Digest.new('sha1'),
      secret_access_key, gon.policy
    )
  ).gsub("\n","")
end

Note: I use Gon to parse variable from Rails controller to Javascript.

Comments