First tests - Upload to R2

[ndrean]

R2 doesn"t seem to serve the files you upload...

[ndrean]

Turns out that - for server side - , you just need to use host in the :ex_aws config for ExAws to work (exactly like with B2, just change the endpoint) 🎉

config :ex_aws, :s3,
   access_key_id: xxxx,
   secret_key_id: xxxx,
   host: <ACCOUNT_ID>.r2.cloudflarestorage.com
   bucket: xxxx

and then you can Ctrl-C/V the standard code from ExAws.S3.Upload:

{:ok, %{body: %{location: location, key: key}, status_code: 200}} =
          path
          |> ExAws.S3.Upload.stream_file()
          |> ExAws.S3.upload(bucket, URI.encode(filename),
            acl: :public_read,
            content_type: mime,
            content_disposition: "inline"
          )
          |> ExAws.request()

[ndrean]

Fighting since this WE to let Cloudfare R2 serve images....

In the browser, I select some pics (FilePicker ou D&D via a Hook): note that you need two inputs (whatever the number of selected files).

<form
  class="mt-10 grid grid-cols-1 gap-x-6 gap-y-8 border-solid border-2"
  phx-change="validate"
  phx-submit="save"
  id="direct-upload-form"
>
  <div>
    <.live_file_input upload={@uploads.images} hidden />

    <label for="f-input">
      Choose a picture (JPEG, PNG, WEBP)
      <input
        type="file"
        data-el-input
        multiple
        accept=".jpeg, .jpg, .png, .webp"
        name="images"
        id="f-input"
        phx-hook="HandleImages"
      />
    </label>
  </div>
  <button type="submit">Upload</button>
</form>

<%= for entry <- @uploads.images.entries do %>
    <figure :if={String.contains?(entry.client_name, "m200.webp")}>
      <.live_img_preview entry={entry} id={entry.uuid}  />
      <figcaption><%= entry.client_name %></figcaption>
    </figure>
 <% end %>

I can "submit" and upload to an R2 bucket (and run a prediction):

but I can't serve them:

 <%= for %{label: label, thumb: thumb, full: full_loc} <- @uploaded_files do %>
    <figure>
      <a href={full_loc} target="_blank" rel="noreferrer">
        <img src={thumb} />
      </a>
      <figcaption><%= label %></figcaption>
    </figure>
  <% end %>

I bought a domain in the Cloudfare registar, now I am trying to link this to Fly.io and to cache the R2 bucket....

[ndrean]

🎉 So to let Cloudfare serve images from a bucket, you need to register a domain by them, and then "connect" the bucket, and:

• use your new domain to serve the pictures,
• set content-disposition to inline

So I can upload from the Elixir app into the bucket and then change the URL domain 👍

This means:

• you upload to https://[account-id]..r2.cloudflarestorage.com/[bucket-name]/[filename]
• you read from https://[your-cloudfare-domain]/[filename]

Settings: just change the ENV VARS form S3 to R2, AND declare a host. Then ExAws.S3 will automatically use it (clever!).

config :ex_aws, :s3,
  r2_account_id: System.get_env("R2_ACCOUNT_ID"),
  host: System.get_env("R2_ENDPOINT"),
  access_key_id: System.get_env("R2_CLIENT_ID"),
  secret_access_key: System.get_env("R2_CLIENT_SECRET"),
  bucket: System.get_env("R2_BUCKET")

where host: https://[account-id].r2.cloudflarestorage.com

The Elixir code to display HTML:

<%= for %{label: label, thumb: thumb, full: full_loc} <- @uploaded_files do %>
  <figure>
    <a href={full_loc} target="_blank" rel="noreferrer">
      <img src={thumb} />
    </a>
    <figcaption><%= label %></figcaption>
  </figure>
<% end %>

translates to:

<figure>
  <a href="https://up-image.org/dff3a13e58c2fcb2d1382f8f016e035405bad359-m1440.webp">
    <img src="https://up-image.org/dff3a13e58c2fcb2d1382f8f016e035405bad359-m200.webp">
  </a>
  <figcaption>a boat is docked on the shore of a lake</figcaption>
</figure>

You see that we display the list of uploaded files, for a given user. This is optimzed since we display - thus fetch - only a thumbnail version of the image, of max 200px, approx 10kB. The "full" image is avialable by clicking on it. This means we can easily return a list of 100 pictures with a payload of "only" 1MB. This can further be optimised using stream, and display them say 4 by 4, so a payload of say 100kB per scroll event.

About Content-Disposition:

! Important to save a couple of hours: when you upload, set the Content-Disposition to inline as it seems to default to attachment, meaning it downloads automatically, which is maybe not desired.

This means, the upload code in Elixir should be something like:

def upload_task(path, name) do
    Task.Supervisor.async_nolink(UpImg.TaskSup, fn ->
      path
      |> S3.Upload.stream_file()
      |> S3.upload(EnvReader.bucket(), name,
        acl: :public_read,
        content_type: "image/webp",
        content_disposition: "inline"
      )
      |> ExAws.request()
    end)
 rescue
    e ->
      {:error, inspect(e)}
 end

[ndrean]

CRUD to finish

It remains to manage these uploads per user:

• save to the database,
• delete from bucket & update DB,
• display the thumbs as stream, and check that the thumb is indeed served from the Cloudfare cache.

About Response images:

I decided to produce 3 versions of a picture:

• a 200px max to be used as the thumbnail of the list
• a 512px max to feed the Image-To-Text process and return a caption
• a 1440px-max for "full-screen".

You can imagine generating and uploading other sizes and use the srcset attribute. In this MDN, you see that each size has a base name termination that designs the screen size.

[nelsonic]

https://developers.cloudflare.com/r2/pricing/

https://www.cloudflare.com/en-gb/pg-cloudflare-r2-vs-aws-s3/#htmlIdPlansMatrix

Immediately saving on the S3 prices and Free Egress makes a huge difference. 💭

[ndrean]

I was editing meanwhile....Never ending? https://www.storj.io/