Hey! I'm available for hire (and I appreciate tips)  



Let's say that you're uploading images to a server and for whatever crazy reason you're not immediately processing them with ruby, or io.js/node.js or whatever you're using to upload them.

Here's how you might routinetly process the images (or whatever type of file) in as a batch job with cron:

Create the Processing Script

Let's say that you just want to resize every file to 300px, you're script might look like this:

#!/bin/bash
set -e
set -u
IN_FILE=$1
OUT_FILE=/srv/processed_images/$(basename "${IN_FILE}")
convert ${IN_FILE} -resize 300 -quality 85 ${OUT_FILE}

Then maybe through that script into /usr/local/bin/process-one-upload.sh

Find only the unprocessed files

Keep a timestamp file and only process files that are newer than the timestamp using find. Use find's -exec option to run the proccessing script on each file.

#!/bin/bash
TS_FILE="./LAST_UPDATE"
PIC_DIR="./Pictures/"
if ! [ -e "${TS_FILE}" ]
then
  touch -t 197001010000.00 "${TS_FILE}"
fi
find "${PIC_DIR}" \
  -newer "${TS_FILE}" \
  -iname '*.jpg' \
  -exec /usr/local/bin/process-one-upload.sh {} \;
touch "${TS_FILE}"

Then maybe through that script into /usr/local/bin/process-new-uploads.sh

NOTE: {} is a kinda like a macro which will be replaced with the current file and the \; is not a typo, it's still part of -exec.

Create the cron job

If you wanted this processor to run every 5 minutes you could use a line like this:

# MIN HOUR DOM MON DOW CMD
*/5  *  *  *  *  /usr/local/bin/process-new-uploads.sh /srv/www/example.com/uploads/

And put it into /etc/crontab

FIN

There you have it: A cron job to iterate over a directory of uploads and run some operation on the ones that have been uploaded since the last time the script ran.


By AJ ONeal

Was this useful to you? Share it!

Also, you can give me a tip or hire me.


Published

2013-12-28

Categories


Tags