Large-scale remote-URL Amazon S3 image warehousing - SOLVED

I’ve been using ProGallery to present large galleries of locally-warehoused images. I.e. images uploaded directly to folders on the server which hosts my Rapidweaver site. My web server hosting provider supports PHP, etc. & all is hunky-dory at small scale & for short-term. (<3 years.)

I now want to migrate my RW galleries to a solution which uses Amazon S3 buckets to permanently store large quantities of images with high reliability, accessibility, & low long-term cost (potentially hundreds of directories w/ thousands of images per directory). My current web-hosting provider can’t meet this need; the per-GB costs are too high & there are no long-term guarantees for data reliability/durability.

I’ve spent an hour searching all variations of “S3”, “Rapidweaver”, “Amazon”, “Warehousing” “Photo”, etc. and can’t come up with anything. It looks like the 2 or 3 major image-gallery solutions which do support warehousing all do so only for locally-hosted directories, i.e. on the same server as the rapidweaver-exported .html / .php files.

How can I host giant galleries of images & arrange them in RW while meeting these requirements?

  1. Images must be stored in Amazon S3 buckets, not on the local web server.
  2. RW stack or plugin must be able to refer to & handle an entire directory of images at once. (Not require me to drag/drop images one by one into gallery grids, etc.)
  3. RW stack must be able to perform intelligent pre-loading of thumbnails, not dumb-load the entire full-size spread of images during initial page load. ProGallery does this, but alas ProGallery doesn’t support Amazon S3-hosted images.

Regards & thanks in advance

SOLVED! Actually it turns out that Stacks4Stacks ProGallery 2 (which I already owned) handles this just fine using custom .CSV files which can point to any remote URL including Amazon S3 buckets.

ProGallery was already highly performant & adapted to this task with its “lazy loading” feature. I.e. you could have a single directory with any number of images (even thousands of full-rez images) load on a single ProGallery web page (within your RapidWeaver site) and it will only pre-load the images the browser has scrolled to & perhaps another 1/2 screenful… further image loading doesn’t occur until the user scrolls down the web page.

@globramma would you mind elaborating on the work flow? Or perhaps sharing an example of your csv file? Interesting topic and just trying to understand the specifics a tad more. I don’t wish to put you out…maybe just a Bit more info on the specifics so I can look deeper? If not no worries. Thanks.

@danhmill there’s actually a good tutorial doc right on the ProGallery 2 info page at: https://stacks4stacks.com/progallery/ .

Basically the way it works is:

  1. You upload your sequentially-numbered set of images to, say, an Amazon S3 bucket.
  2. You also upload a similarly-named set of thumbnails to a subdirectory of that S3 bucket.
  3. You create a simple .CSV-format file which is hosted on your LOCAL webserver (i.e. on your website, not Amazon S3), each row of which refers to the image title, caption, remote URL, and remote thumbnail URL.

The ProGallery2 stack then loads the .CSV file from your local website & uses the remote image URLs to populate the gallery. Works like a charm for me.

Here’s a snippet of the sample .CSV file:

Title,Thumbnail image link,Normal image or video link,Retina image link,Image or video caption,Non lightbox link
Buttermere Valley,https://stacks4stacks.s3.amazonaws.com/test/progallery2/thumbs/1.jpg,https://stacks4stacks.s3.amazonaws.com/test/progallery2/normal/1.jpg,https://stacks4stacks.s3.amazonaws.com/test/progallery2/retina/1@2x.jpg,"Lake District, England. &copy; Will Woodgate, 2017",#null
View over the east of Buttermere Lake,https://stacks4stacks.s3.amazonaws.com/test/progallery2/thumbs/2.jpg,https://stacks4stacks.s3.amazonaws.com/test/progallery2/normal/2.jpg,https://stacks4stacks.s3.amazonaws.com/test/progallery2/retina/2@2x.jpg,"Lake District, England. &copy; Will Woodgate, 2017",#null
Looking down into Buttermere Valley, from the Summit of Haystacks",https://stacks4stacks.s3.amazonaws.com/test/progallery2/thumbs/3.jpg,https://stacks4stacks.s3.amazonaws.com/test/progallery2/normal/3.jpg,https://stacks4stacks.s3.amazonaws.com/test/progallery2/retina/3@2x.jpg,"Lake District, England. &copy; Will Woodgate, 2017",#null

@globramma thank you for the explanation, link, and snippet. Much appreciated and very cool solution.

If there is anything else I can add to ProGallery to make the process quicker or easier, please let me know :slight_smile:

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.