One of my favorite features of Gmail is that every message is backed by a unique URL. I frequently use this URL to easily refer back to a message. This works great, until it doesn't.
The Problem
Suppose this intriguing message from Mr. Helms is something I want to follow-up on. After all, seven million dollars is a lot of money. Normally, I'd grab the URL to this message and add it to a card on my 'On Deck' Trello Board. But in the Android version of the Gmail App, there's no obvious way to get a URL to a message.
(Incidentally, the above screenshot is from a Samsung DeX session. DeX is amazing and is on my list of topics to blog about.)
A Solution
The obvious work-around is to visit mail.google.com in the phone's browser. This will bring up the Desktop Gmail interface, which does have a URL to the message:
Depending on your device's screen size, you may end up at an alternative version of Google's web mail UI.
In this case it took some fiddling to land at the Desktop version of GMail. First I had to convince Google that I wanted to see the Basic HTML interface, and from there I could click over to the Standard interface:
So while it's possible to convince Chrome on my phone to navigate to the Desktop Gmail UI, in practice this can be maddeningly difficult to do. The multiple Google Accounts on my phone, multiple Gmail UIs and the 'smart' logic that guesses what I want to see, means that I often end up clicking in circles in search of the right page.
A Better Solution. Maybe.
An obvious replacement for all this clicking would be to leverage the Gmail API. I even have an existing command line tool, gmail_tool, that interacts with this API.
Currently, I can use gmail_tool to get me the details of the message in question:
$ gmail_tool -a list -q "label:SPAM Helms" 180df45360340ba9:.GREG HELMS, Director. Airport Storage and Cargo Unit Erie International Airport (Pennsylvania) PA 16505, USA eMAIL.
All that's need is to map the above message to the Gmail URL:
https://mail.google.com/mail/u/1/#spam/FMfcgzGpFzwKLQTspjZvpjCvNxLXMQjz
But alas, that's where things get tricky. The magic token FMfcgzGpFzwKLQTspjZvpjCvNxLXMQjz is neither a message ID nor a thread ID. Apparently, this is a 'view token' and while you can decode it in some respects, there's no obvious mapping from information in the API to this token.
A Better Solution. For Sure.
But all is not lost. This article suggests another way forward to uniquely identify a message within Gmail. The answer, which is obvious in hindsight, is to use the search operator rfc822msgid.
This is quite sensible. Each message comes with it's only unique Message-Id header. Searching by this value you should always bring up the one message in question.
So while I can't get the token FMfcgzGpFzwKLQTspjZvpjCvNxLXMQjz from the Gmail API, I can get the headers for a given message, from there search out the Message-Id value.
$ gmail_tool -a list -q "label:SPAM Helms"
180df45360340ba9:.GREG HELMS, Director. Airport Storage and Cargo Unit Erie International Airport (Pennsylvania) PA 16505, USA eMAIL.
# Pull the full JSON for all messages associated with thread id: 180df45360340ba9
$ gmail_tool -a get -i 180df45360340ba9 -v | head -4
{
"id": "180df45360340ba9",
"historyId": "357728706",
"messages": [
# Dump out all the headers associated with this thread
$ gmail_tool -a get -i 180df45360340ba9 -v | \
jq '.messages[] | .payload.headers[] | .name ' | gmail_tool -a get -i 180df45360340ba9 -v | jq '.messages[] | .payload.headers[] | .name '
"Delivered-To"
"Received"
"X-Received"
"ARC-Seal"
"ARC-Message-Signature"
"ARC-Authentication-Results"
"Return-Path"
"Received"
"Received-SPF"
"Authentication-Results"
"DKIM-Signature"
"X-Google-DKIM-Signature"
"X-Gm-Message-State"
"X-Google-Smtp-Source"
"X-Received"
"MIME-Version"
"Received"
"Reply-To"
"From"
"Date"
"Message-ID"
"Subject"
"To"
"Content-Type"
"Bcc"
Once I combined my knowledge that you can search by rfc822msgid and how to access the message headers using the Gmail API, I was able to put that together into a simple option for gmail_tool:
# set the shell variable $tid to the matching thread id $ tid=$(gmail_tool -a list -q "label:SPAM Helms" | cut -d: -f1) # look up the URL to for $tid $ gmail_tool -a url -i $tid https://mail.google.com/mail/u/0/?#search/rfc822msgid:CAMHjZTPcpcGY6nyxKNDTDhxjvw1GTrZf%3DVrH-BCtaLxq2d_vqg%40mail.gmail.com
Visiting this URL takes me a Gmail search result page with the one message I'm seeking:
Success!
Here's the latest version of gmail_tool with both url and header options added:
#!/bin/bash
##
## command line tools for working with Gmail.
##
CLIENT_ID=<from https://console.cloud.google.com/apis/>
CLIENT_SECRET=<from https://console.cloud.google.com/apis/>
API_SCOPE=https://www.googleapis.com/auth/gmail.modify
API_BASE=https://www.googleapis.com/gmail/v1
AUTH_TOKEN=`gapi_auth -i $CLIENT_ID -p $CLIENT_SECRET -s $API_SCOPE token`
usage() {
cmd="Usage: $(basename $0)"
echo "$cmd -a init"
echo "$cmd -a list -q query [-v]"
echo "$cmd -a get -i id [-v]"
echo "$cmd -a labels"
echo "$cmd -a update -i id -l labels-to-add -r labels-to-remove"
echo "$cmd -a headers -i id"
echo "$cmd -a url -i id"
echo "$cmd -a messages -q query [-v]"
exit
}
filter() {
if [ -z "$VERBOSE" ] ; then
jq "$@"
else
cat
fi
}
listify() {
sep=""
expr="[ "
for x in "$@" ; do
expr="$expr $sep \"$x\""
sep=","
done
expr="$expr ]"
echo $expr
}
while getopts ":a:r:q:i:l:vp" opt ; do
case $opt in
a) ACTION=$OPTARG ;;
v) VERBOSE=yes ;;
q) QUERY="$OPTARG" ;;
l) LABELS_ADD=$OPTARG ;;
r) LABELS_REMOVE=$OPTARG ;;
i) ID=$OPTARG ;;
p) PAGING=yes ;;
\?) usage ;;
esac
done
invoke() {
root=$1 ; shift
curl -s -H "Authorization: Bearer $AUTH_TOKEN" "$@" > /tmp/yt.buffer.$$
next_page=`jq -r '.nextPageToken' < /tmp/yt.buffer.$$`
if [ "$PAGING" = "yes" ] ; then
if [ "$next_page" = "null" -o -z "$next_page" ] ; then
cat /tmp/yt.buffer.$$
rm -f /tmp/yt.buffer.$$
else
jq ".$root" < /tmp/yt.buffer.$$ | sed 's/^.//'> /tmp/yt.master.$$
while [ "$next_page" != "null" ] ; do
curl -s -H "Authorization: Bearer $AUTH_TOKEN" "$@" -d pageToken=$next_page |
tee /tmp/yt.buffer.$$ |
( echo "," ; jq ".$root" | sed 's/^.//' ) >> /tmp/yt.master.$$
next_page=`jq -r '.nextPageToken' < /tmp/yt.buffer.$$`
done
rm -f /tmp/yt.buffer.$$
echo "{ \"$root\" : [ "
cat /tmp/yt.master.$$
echo ' ] }'
rm -f /tmp/yt.master.$$
fi
else
cat /tmp/yt.buffer.$$
rm /tmp/yt.buffer.$$
fi
}
case $ACTION in
init)
gapi_auth -i $CLIENT_ID -p $CLIENT_SECRET -s $API_SCOPE init
exit
;;
list)
if [ -z "$QUERY" ] ; then
echo "Uh, better provide a query"
echo
usage
fi
invoke threads -G $API_BASE/users/me/threads \
--data-urlencode q="$QUERY" \
-d maxResults=50 |
filter -r ' .threads[]? | .id + ":" + (.snippet | gsub("[ \u200c]+$"; ""))'
;;
get)
if [ -z "$ID" ] ; then
usage
fi
invoke messages -G $API_BASE/users/me/threads/$ID?format=full \
-d maxResults=50 |
filter -r ' .messages[] | .id + ":" + (.snippet | gsub("[ \u200c]+$"; ""))'
;;
labels)
invoke labels -G $API_BASE/users/me/labels |
filter -r ' .labels[] | .id + ":" + .name'
;;
update)
if [ -z "$ID" ] ; then
echo "Missing -i id"
exit
fi
if [ -z "$LABELS_ADD" -a -z "$LABELS_REMOVE" ] ; then
echo "Refusing to run if you don't provide at least one label to add or remove"
exit
fi
body="{ addLabelIds: $(listify $LABELS_ADD), removeLabelIds: $(listify $LABELS_REMOVE) }"
invoke messages -H "Content-Type: application/json" \
$API_BASE/users/me/threads/$ID/modify \
-X POST -d "$body" |
filter -r '.messages[] | .id + ":" + (.labelIds | join(","))'
;;
url)
if [ -z "$ID" ] ; then
echo "Missing thread ID"
exit
fi
base_url="https://mail.google.com/mail/u/0/?#search/rfc822msgid"
message_id=$(gmail_tool -a get -i $ID -v |
jq -r '.messages[0].payload.headers[] | select(.name | ascii_downcase == "message-id") | .value| @uri' |
sed -e 's/^%3C//' -e 's/%3E$//' )
echo "$base_url:$message_id"
;;
headers)
if [ -z "$ID" ]; then
echo "Missing thread ID"
fi
gmail_tool -a get -i $ID -v |
jq -r '.messages[] | .id + ":" + (.payload.headers[] | .name + ":" + .value)'
;;
*)
usage
;;
esac
