Let's say that you're uploading images to a server and for whatever crazy reason you're not immediately processing them with ruby, or io.js/node.js or whatever you're using to upload them.

Here's how you might routinetly process the images (or whatever type of file) in as a batch job with cron:

Create the Processing Script

Let's say that you just want to resize every file to 300px, you're script might look like this:

#!/bin/bash
set -e
set -u
IN_FILE=$1
OUT_FILE=/srv/processed_images/$(basename "${IN_FILE}")
convert ${IN_FILE} -resize 300 -quality 85 ${OUT_FILE}

Then maybe through that script into /usr/local/bin/process-one-upload.sh

Find only the unprocessed files

Keep a timestamp file and only process files that are newer than the timestamp using find. Use find's -exec option to run the proccessing script on each file.

#!/bin/bash
TS_FILE="./LAST_UPDATE"
PIC_DIR="./Pictures/"
if ! [ -e "${TS_FILE}" ]
then
  touch -t 197001010000.00 "${TS_FILE}"
fi
find "${PIC_DIR}" \
  -newer "${TS_FILE}" \
  -iname '*.jpg' \
  -exec /usr/local/bin/process-one-upload.sh {} \;
touch "${TS_FILE}"

Then maybe through that script into /usr/local/bin/process-new-uploads.sh

NOTE: {} is a kinda like a macro which will be replaced with the current file and the \; is not a typo, it's still part of -exec.

Create the cron job

If you wanted this processor to run every 5 minutes you could use a line like this:

# MIN HOUR DOM MON DOW CMD
*/5  *  *  *  *  /usr/local/bin/process-new-uploads.sh /srv/www/example.com/uploads/

And put it into /etc/crontab

FIN

There you have it: A cron job to iterate over a directory of uploads and run some operation on the ones that have been uploaded since the last time the script ran.


By AJ ONeal

If you loved this and want more like it, sign up!


Did I make your day?
Buy me a coffeeBuy me a coffee  

(you can learn about the bigger picture I'm working towards on my patreon page )



Published

2013-12-28



Buy me a coffeeBuy me a coffee


  73% of Goal Reached


Want more like this?