The Lazy Gabbai: Shell Scripting ShulCloud

As a programmer, I'm all about embracing laziness. So when I found myself week-after-week clumsily clicking through the yahrzeit section of ShulCloud I knew I needed a better way. The shell script below is that better way.

The What

Taking a step back, ShulCloud is the platform my synagogue uses to help manage itself. One feature of ShulCloud is the ability to track yahrzeit's, that is the date an individual has passed away. We use this information to help loved ones honor their deceased relatives on the anniversary of their death.

As a gabbai in my shul, I use this information to let me know who should be honored in the upcoming week.

The How

ShulCloud doesn't offer a developer's API (not that I blame them, but a programmer can dream). So the strategy I'd have to use is good 'ol fashion web page scraping. Every time I want to download the list of yahrzeits I use curl to re-login into the site and invoke the export URL. The ability to pass a cookie jar file to curl (via -b $COOKIE_JAR and -c $COOKIE_JAR) means that cookies set during the login process are carried forward when I attempt to download the export.

I do a bit of date math to allow me to specify the time frame I want to view in terms of an offset. For example, I can run the following:

  # this week's yahrzeits
  $ shulcloudassist -a yz
  ...

  # last week's yahrzeits
  $ shulcloudassist -a yz -w -1
  ...

  # two week's from now yahrzeits
  $ shulcloudassist -a yz -w 2

A couple other notes of interest: I had to do use gdate on Mac to get access to GNU date. A case $(uname) in statement at the top of the script lets me set the date command to use and should make this script cross-platform. And finally, I used csvkit to process the exported data.

Here's what this week's yahrzeit list looks like. Note that I'm running the output through scrub_names.sh to replace real names with random ones. The output tells me not just who the mourner is and the anniversary date of their loved one, but also the date's offset relative to today. Below Berry Guss is remembering Randall Royce, and I can see that the exact yahrzeit was 2 days ago.

$ shulcloudassist -a yz |  sh ./scrub_names.sh 
Sat Apr 10 (-3d) Shannon & CorneliusAnn Portia, Rodger Hans: Ann Ezra
Sun Apr 11 (-2d) Berry Guss: Randall Royce
Mon Apr 12 (-1d) Teri Esmeralda, Lacy Willis: Margret Kathryne
Mon Apr 12 (-1d) Valorie Frank: Gregg Craig
Mon Apr 12 (-1d) Rodrigo Kattie: Madelyn Sol Gena
Tue Apr 13 (0d) Dixie Maud: Neal Toni
Wed Apr 14 (1d) grandmother of Alec Charlie: Norman William Dorian Ramon
Thu Apr 15 (2d) Christi Bryce: Jessie Sebastian

Even if you don't use ShulCloud, you may find this web scraping and date mashing example helpful. Happy Hacking!

The Script

#!/bin/bash

##
## Script to automate stuff within shulcloud
##

USER=XXXXXXXXXXXXXXXXXXXX
PASSWORD=XXXXXXXXXXXXXXXXXXXX
BASE=https://someurl.shulcloud.com
COOKIE_JAR=$HOME/.shulcloudassist.cookies

case $(uname) in
  Darwin)
    DATE=gdate
    ;;
  *)
    DATE=date
    ;;
esac

usage() {
  echo "Usage: $(basename $0) -a yz [-w <week-offset>]"
  exit 1
}


login() {
  sccsrf=$(curl -s -c $COOKIE_JAR $BASE/  | grep sccsrf | sed -e 's/^.*value="//' -e 's/".*//')
  curl -d action=login \
       -d redirect=$BASE \
       -d sccsrf=$sccsrf \
       -d "email=$USER" \
       -d "password=$PASSWORD" \
       -c $COOKIE_JAR -b $COOKIE_JAR \
       -s -i $BASE/login.php > /dev/null
}

while getopts ":a:w:h" o; do
  case "${o}" in
    a)
      action=$OPTARG
      ;;
    w)
      week=$OPTARG
      ;;
    * | h)
      usage
      ;;
  esac
done

case "$action" in
  yz)
    if [ -z "$week" ] ; then
      week=0
    fi
    shift=$(($week * 7))

    sow=$(($($DATE +%u) + 1 - $shift))
    eow=$((6 - $sow))
    start=$($DATE -d "$sow days ago" +%Y-%m-%d )
    end=$($DATE -d "$eow days" +%Y-%m-%d )
    start_fmt=$($DATE -d "$start" +%m/%d/%Y)
    end_fmt=$($DATE -d "$end" +%m/%d/%Y)
    now=$($DATE -d 00:00 +%s)
    
    login
    
    curl -s -G \
         -d order=next_date \
         -d order_asc=1 \
         -d search= \
         -d is_active= \
         -d search_mourners= \
         -d gender= \
         -d setPerPage=100 \
         -d start_date=$start \
         -d text_start_date=$start_fmt \
         -d last_start_date=$end \
         -d end_date=$end \
         -d text_end_date=$end \
         -d last_end_date=$end_fmt \
         -d has_plot= \
         -d has_plaque= \
         -d has_relatives=Y \
         -d action=export \
         -b $COOKIE_JAR -c $COOKIE_JAR \
         $BASE/admin/yahrzeits.php | \
      sed '1d' | 
      while read line ; do
        d_fname=$(echo $line | csvcut -c 3)
        d_lname=$(echo $line | csvcut -c 4)
        mourner=$(echo $line | csvcut -c 22 | tr -d '"')
        observed=$(echo $line | csvcut -c 10)
        offset=$(($(($($DATE -d "$observed" +%s) - $now)) / 86400))
        date=$($DATE -d "$observed" +"%a %b %d")
        echo "$date (${offset}d) $mourner: $d_fname $d_lname"
      done
    
    ;;

  *)
    usage
    ;;
esac

A Bonus

Here's the script I used to randomize names in an input stream. This script was intended as a throw-away, which is why it's not particularly elegant.

#!/bin/bash

##
## scrub names from input. Replace what appears to be a name
## with a random name from baby-names.csv.
##
## Grab baby-names.csv from
## https://github.com/hadley/data-baby-names/blob/master/baby-names.csv
##
sed 's/[A-Z][a-z][a-z][a-z][a-z]*/NAME/g' | tr '\n' '@' > /tmp/ping

while [ -n "$(grep NAME /tmp/ping)" ] ; do
  name=$(shuf -n 1 baby-names.csv | csvcut -c 2)
  sed "s/NAME/$name/" /tmp/ping > /tmp/pong
  mv /tmp/pong /tmp/ping
done

cat /tmp/ping | tr '@' '\n'