Python + Google Voice. Mass SMS and Iterative Calling at the Command Line

EDIT (Jan 2011): Google changed the way contacts are downloaded. Many of the scripts below will break. Use the updated version here.

EDIT (Oct 3 2009): Google changed a few things on their login page which broke my old scripts. I have updated the scripts in this post, including those that you can download. If you have been receiving "Could not log in with provided credentials" errors, these new scripts should fix that problem.

You can download a ready-to-use-no-Python-installation-required Windows executable version of the program here. Just download, unzip, find the GVMassContact.exe file and double click to use. This version's source is identical to those of scripts located below.

Click here to navigate to the downloadable Python scripts.

I have already written twice about Google Voice (here and here), but the scripts in this installment are an improvement over both of the scripts provided the other posts.

I wanted to create an interactive command line script that would allow me to either send an SMS message or call everyone in one of my Google Contacts Groups. Google Voice does not yet let you send mass text messages, or call people in an iterative fashion, so I came up with a script to let me do just that.

Instead of using Googles gdata Python API to access my Google Contacts, I decided to use Pythons "csv" module to parse an exported csv file downloaded from my GMail account. Doing this lets you run the script without needing to download any other libraries, assuming you are using a version of Python that has all the features used in this script (I use 2.6.2 currently).

The first thing I did was create a "gvoice.py" module with several helpful classes. These allow you to:

1. Log in
2. Gather all Google Contacts into separate groups
3. Selectively narrow down the contacts in a group
4. Gather the phone numbers that you have entered
5. Send SMS messages
6. Place Calls

I am by no means a seasoned Python developer, but what I have created works well for my purposes. I tried to make the classes as loosely coupled with the UI as possible in case I want to put a GUI around it sometime, but for now I am doing everything at the command line (which I often prefer).

The GoogleVoiceLogin class will allow you to log in, get the "opener" which lets you keep your log in cookie data for subsequent requests, and gives you access to the _rnr_se value (the "key") that is used when sending SMS messages or placing a call. Most of the other classes in the "gvoice" module will accept the opener and key as parameters to their constructors.

I created a sample script that uses the classes in my "gvoice" module to:

1. Prompt the user for his/her Google Account credentials
2. Allow the user to select a Group from his/her Google Contacts Groups.
3. Allow the user to narrow down the contact list of the Group (nothing permanent is done to the Contacts)
4. Choose whether to send an SMS message or call everyone in the list
5. If calling, it allows you to choose a number from your phone list, or enter a custom number (like the Google Voice site does when placing calls)
   

The script that I wrote assumes I always want to use the contact's mobile phone (the first one) to send SMS messages and place calls, but that is easy to change.

There are two files needed for this to work - here is the main driver program (gvMassContact.py):

from gvoice import *
import getpass
import sys
import re
import os

# Function used to create a separator
def separator():
return '-' * 25

def get_numeric_input(prompt):
try:
return int(raw_input(prompt))
except:
pass

# Function to clear the screen
def clear_screen():
if os.name == "posix":
# *nix systems
os.system('clear')
elif os.name in ("nt", "dos", "ce"):
# Windows
os.system('CLS')

# Main method to be run 
def main():
# Log in
print "Please enter your Google Account credentials"
email = raw_input("User name: ")
password = getpass.getpass("Password: ")

gv = GoogleVoiceLogin(email, password)
if not gv.logged_in:
print "Could not log in with provided credentials"
sys.exit(1)
else:
print "Login successful!"

# Use the ContactLoader to download Google Contacts 
contact_loader = ContactLoader(gv.opener)

# Use the ContactSelector to select the group and
# final list of contacts to contact
contact_selector = ContactSelector(contact_loader.contacts_by_group_list)

clear_screen()
group_list = contact_selector.get_group_list()
selected_group = None
while selected_group not in range(1, len(group_list)+1):
print "Your Google Groups"
print separator()
for group_item in group_list:
   print "{0}: {1}".format(group_item[0], group_item[1])
print separator()
selected_group = get_numeric_input("Enter the index of the group to select: ")

clear_screen()
# Now that a group is selected, narrow down the list of people in the group
contact_selector.set_selected_group(selected_group)
removing = True
while removing:
print "Contact List"
print separator()
for contact_item in contact_selector.get_contacts_list():
   print "{0}: {1}".format(contact_item[0], contact_item[1])
print separator()
try:
   input_list = raw_input("Enter a list of the indexes (coma, space or otherwise delimeted)\nof those contacts you do not wish to contact this session.\nPress enter when finished:  ")
   if input_list != '':
       contacts_to_remove_list  = [int(match.group(1)) for match in re.finditer(r"(\d+)", input_list)]
       contact_selector.remove_from_contact_list(contacts_to_remove_list)
   else:
       removing = False
except:
   pass

clear_screen()
# Print final list
clear_screen()
print "Final List:"
print separator()
for contact_item in contact_selector.get_contacts_list():
print "{0}".format(contact_item[1])
selected_option = None
while selected_option not in [1, 2]:
print separator()
print "Options: "
print "1: Send Text"
print "2: Call"
print separator()
selected_option = get_numeric_input("Select which action to take: ")

# Send texts to all people in contact list
if (selected_option == 1):
print separator()
text_sender = TextSender(gv.opener, gv.key)
text = raw_input("Enter text message. Press enter when finished: ")
text_sender.text = text
for contact in contact_selector.get_contacts_list():
   number = contact[1].mobile
   if number == '':
       print "{0} does not have a mobile number".format(contact[1])
   else:
       print "Sending message to {0} at {1}...".format(contact[1], contact[1].mobile),
       text_sender.send_text(contact[1].mobile)
       if text_sender.response:
           print "Success!"
       else:
           print "Failed!!"
  
# Call all people in contact list             
elif (selected_option == 2):
print separator()
number_dialer = NumberDialer(gv.opener, gv.key)

number_retriever = NumberRetriever(gv.opener)
phone_number_items = number_retriever.get_phone_numbers()

clear_screen()
# Get the forwarding number
forwarding_number_input = None
while forwarding_number_input not in range(1, len(phone_number_items) + 2):
   print "Select forwarding number"
   print separator()
   for phone_number_item in phone_number_items:
       print "{0}: {1}".format(phone_number_item[0], phone_number_item[1][0])
   print "{0}: {1}".format(len(phone_number_items) + 1, "Other")
   print separator()
   forwarding_number_input = get_numeric_input("Choose from your previously entered numbers, or select  \"Other\": ")

if forwarding_number_input in range(1, len(phone_number_items) + 1):
   forwarding_number = phone_number_items[forwarding_number_input - 1][1]
else:
   forwarding_number = ''
   while not re.match(r"\(?\b[0-9]{3}\)?[-. ]?[0-9]{3}[-. ]?[0-9]{4}\b\Z", forwarding_number):
       forwarding_number = raw_input("Enter the forwarding number to dial: ")
number_dialer.forwarding_number = forwarding_number

print separator()
# Loop through and make the calls
for contact in contact_selector.get_contacts_list():
   number = contact[1].mobile
   if number == '':
       print "{0} does not have a mobile number".format(contact[1])
   else:
       input = None
       while input not in ['', 'n','N', 'q', 'Q'] :
           input = raw_input("Press enter to call {0} at {1} ('n' to skip, 'q' to quit): ".format(contact[1], contact[1].mobile))
       if input == '':
           print "Calling {0}....".format(contact[1]),
           number_dialer.place_call(number)
           if number_dialer.response:
               print "Success!"
           else:
               print "Failed!!"
       elif input .upper() == 'N':
           pass
       elif input.upper() == 'Q':
           print "Call chain aborted."
           break
          
if __name__ == "__main__":
main()

And here is the other required script (must be named "gvoice.py"):

import csv
import sys
import re
import urllib
import urllib2

class GoogleVoiceLogin:
def __init__(self, email, password):
# Set up our opener
self.opener = urllib2.build_opener(urllib2.HTTPCookieProcessor())
urllib2.install_opener(self.opener)

# Define URLs
self.loing_page_url = 'https://www.google.com/accounts/ServiceLogin'
self.authenticate_url = 'https://www.google.com/accounts/ServiceLoginAuth'
self.gv_home_page_url = 'https://www.google.com/voice/#inbox'

# Load sign in page
login_page_contents = self.opener.open(self.loing_page_url).read()

# Find GALX value
galx_match_obj = re.search(r'name="GALX"\s*value="([^"]+)"', login_page_contents, re.IGNORECASE)

galx_value = galx_match_obj.group(1) if galx_match_obj.group(1) is not None else ''

# Set up login credentials
login_params = urllib.urlencode( {
   'Email' : email,
   'Passwd' : password,
   'continue' : 'https://www.google.com/voice/account/signin',
   'GALX': galx_value
})

# Login
self.opener.open(self.authenticate_url, login_params)

# Open GV home page
gv_home_page_contents = self.opener.open(self.gv_home_page_url).read()

# Fine _rnr_se value
key = re.search('name="_rnr_se".*?value="(.*?)"', gv_home_page_contents)

if not key:
   self.logged_in = False
else:
   self.logged_in = True
   self.key = key.group(1)

class ContactLoader():
def __init__(self, opener):
self.opener = opener
self.contacts_csv_url = "http://mail.google.com/mail/contacts/data/export"
self.contacts_csv_url += "?groupToExport=^Mine&exportType=ALL&out=OUTLOOK_CSV"

# Load ALL Google Contacts into csv dictionary
self.contacts = csv.DictReader(self.opener.open(self.contacts_csv_url))

# Create dictionary to store contacts and groups in an easier format
self.contact_group = {}
# Assigned each person to a group that we can get at later
for row in self.contacts:
   if row['First Name'] != '':
       for category in row['Categories'].split(';'):
           if category == '':
               category  = 'Ungrouped'
           if category not in self.contact_group:
               self.contact_group[category] = [Contact(row)]
           else:
               self.contact_group[category].append(Contact(row))

# Load contacts into a list of tuples...
# [(1, ('group_name', [contact_list])), (2, ('group_name', [contact_list]))]
self.contacts_by_group_list = [(id  + 1, group_contact_item)
                              for id, group_contact_item in enumerate(self.contact_group.items())]

class Contact():
def __init__(self,contact_detail):
self.first_name = contact_detail['First Name'].strip()
self.last_name = contact_detail['Last Name'].strip()
self.mobile = contact_detail['Mobile Phone'].strip()
self.email = contact_detail['E-mail Address'].strip()

def __str__(self):
return self.first_name + ' ' + self.last_name

# Class to assist in selected contacts by groups
class ContactSelector():
def __init__(self, contacts_by_group_list):
self.contacts_by_group_list = contacts_by_group_list
self.contact_list = None

def get_group_list(self):
return [(item[0], item[1][0]) for item in self.contacts_by_group_list]

def set_selected_group(self, group_id):
self.contact_list  = self.contacts_by_group_list[group_id - 1][1][1]

# Return the contact list so far
def get_contacts_list(self):
return [(id + 1, contact) for id, contact in enumerate(self.contact_list)]

# Accept a list of indexes to remove from the current contact list
# Assumes 1 based list being passed in
def remove_from_contact_list(self, contacts_to_remove_list):
if self.contact_list == None:
   return
for id in contacts_to_remove_list:
   if id in range(0, len(self.contact_list)+1):
       self.contact_list[id - 1] = None
self.contact_list = [contact for contact in self.contact_list if contact is not None]

class NumberRetriever():
def __init__(self, opener):
self.opener = opener
self.phone_numbers_url = 'https://www.google.com/voice/settings/tab/phones'
phone_numbers_page_content = self.opener.open(self.phone_numbers_url).read()

# Build list of all numbers and their aliases
self.phone_number_items = [(match.group(1), match.group(2))
                          for match
                          in re.finditer('"name":"([^"]+)","phoneNumber":"([^"]+)"',
                                                   phone_numbers_page_content)]

def get_phone_numbers(self):
return [(id + 1, (phone_number_item))
       for id, phone_number_item
       in enumerate(self.phone_number_items)]

class TextSender():
def __init__(self, opener, key):
self.opener = opener
self.key = key
self.sms_url = 'https://www.google.com/voice/sms/send/'
self.text = ''

def send_text(self, phone_number):
sms_params = urllib.urlencode({
   '_rnr_se': self.key,
   'phoneNumber': phone_number,
   'text': self.text
})
# Send the text, display status message
self.response  = self.opener.open(self.sms_url, sms_params).read()

class NumberDialer():
def __init__(self, opener, key):
self.opener = opener
self.key = key
self.call_url = 'https://www.google.com/voice/call/connect/'
self.forwarding_number = None

def place_call(self, number):
call_params = urllib.urlencode({
   'outgoingNumber' : number,
   'forwarding_number' : self.forwarding_number,
   'subscriberNumber' : 'undefined',
   'remember' : '0',
   '_rnr_se': self.key
   })

# Send the text, display status message
self.response  = self.opener.open(self.call_url, call_params).read()

For the "gvMassContact.py" script to work, you need to have the "gvoice.py" file located in the same folder.

If you would rather not copy and paste, you can download the files here:

• gvMassContact.py
• gvoice.py

If you don't care about having the files separated, and would rather have to deal with just one script that you can keep on your desktop, you can download this file:

• gvAllInOne.py

Here is a new, ready-to-use-no-Python-installation-required Windows executable version. Just download, unzip, find GVMassContact.exe in the GVMassContact folder, double click and go.

The only difference between the "allInOne.py" script and the others is that all of the classes contained in the "gvoice.py" file are located at the top of the file.

If you download the "gvAllInOne.py" script, you can keep it on your desktop and simply double click it to get the interactive command line session to come up.

Enjoy!

Python - Google Voice part 2

EDIT: Google recently made some behind the scenes changes to the login page, which broke this script. Please see the new and improved script here.

My last post was also about accessing your Google Voice account through Python, but there is more that you can do with it than just send SMS messages. You can make calls, cancel calls, view your call/sms/voicemail history and a few more others, as pointed out by this blogger here while talking about his Firefox plugin.

Since all of these options require you to be logged in, I thought that it would be easier to create a separate class that would:
1) Log you in and let you know whether or not your attempt was successful
2) Provide a method to get the "opener" (which is what keeps cookie data in order during multiple requests),
3) Provide a method to get your "_rnr_se" value, which is required when sending SMS messages, making calls, and canceling calls.

Instead of hardcoding in Google Account credentials like in my last post, the new class will prompt you for your Google Account user name and password. Since I want to use this script in public, I used the getpass module to hide the input as I type it. Here is the script:

# Get URL handling support
import urllib2, urllib
# Get regular expression support
import re, getpass

class GoogleVoiceLogin:
def __init__(self):
print "Please enter your Google Account credentials"
self.email = raw_input("User name: ")
self.password = getpass.getpass("Password: ")

# Set up an opener with HTTPCookieProcessor
self.opener = urllib2.build_opener(urllib2.HTTPCookieProcessor())
urllib2.install_opener(self.opener)

# Set up login credentials for Google Accounts
# The 'continue' param redirects us to the Google Voice 
# homepage, and gives us necessary cookie info
loginParams = urllib.urlencode( {
    'Email' : self.email,
    'Passwd' : self.password,
    'continue' : 'https://www.google.com/voice/account/signin'
} )

# Perform the login. Cookie info sent back will be saved, so we remain logged in
# for future requests when using the opener
self.opener.open( 'https://www.google.com/accounts/ServiceLoginAuth',  loginParams)

# Need to load the homepage to find user specific data
googleVoiceHomeData = self.opener.open('https://www.google.com/voice/#inbox').read()

# Go through the home page and grab the value for the hidden
# form field "_rnr_se", which must be included when sending texts and dealing with calls
match = re.search('name="_rnr_se".*?value="(.*?)"', googleVoiceHomeData)
if not match:
    print "Login Unsuccessful!"
    exit()
else:
    print "Login Successful!"
    self._rnr_se = match.group(1)

def getOpener(self):
return self.opener

def getRnrSe(self):
return self._rnr_se

The script, for the most part, should be self-explanatory. It pretty much follows the same pattern as the script in my previous post except it is more reusable. For example, here is the new script I use to send text messages from the command line:

# Send a text Message
from GoogleVoiceLogin import GoogleVoiceLogin
import urllib

# Create an instance of a GoogleVoiceLogin object
# This will prompt you for your Google Account credentials
gvLogin = GoogleVoiceLogin()

# Get the open (Cookie data still intact)
opener = gvLogin.getOpener()

# Get the _rnr_se value
_rnr_se = gvLogin.getRnrSe()

# Prompt for Text Message details
phoneNumber = raw_input("Destination number: ")
text = raw_input("Message to send: ")

# Insert blank line
print

# Set up parameters for sending text
sendTextParams = urllib.urlencode({
    '_rnr_se': _rnr_se,
    'phoneNumber': phoneNumber,
    'text': text
})

# Send the text, display status message  
response  = opener.open('https://www.google.com/voice/sms/send/', sendTextParams)
if "true" in response.read():
    print "Message successfully sent!"
else:
    print "Message failed!"
response.close()

Similar scripts could easily be created to call people, or perform other tasks on your account.

Google Voice - Python - SMS

EDIT: Google recently made some behind the scenes changes to the login page, which broke this script. Please see the new and improved script here.

After a long wait, I finally received my Google Voice invitation last weekend. It offers a lot of features, but one that I was really excited about was the free SMS service. I don't have a cell phone, but nearly all my family members, friends and co-workers do. The majority of them are more prone to check their phone than their email, so sending a text is the best way to contact them. For a while, I used Gmail's SMS Lab to get in contact with the leaders and the Scouts of a Boy Scout troop that I participate in, but Gmail shut that down recently.

After I got my account, I began thinking about how neat it would be to be able to set up a script to send reminder text messages for me (preferably scheduled) using my account. I searched all over the internet to find out if there was an API provided by Google to do this -like their excellent gdata API - but no luck. I didn't give up there, there had to be a way.

A while ago, I learned that you can use HTTPCookieProcessor() from the urllib2 module to create an opener that will keep track of Cookie data sent back and forth from a server. So, with the help of Firebug (the best friend of many web-developers) I looked at the headers and POST data being sent when logging in to Google Voice, and when sending a text message. Turns out, it isn't very hard to log into your account, grab some necessary information from the Google Voice Home page (the inbox) and send a text.

This script will do just that one thing - log in your account, load the homepage to get a hidden form field's value, and then send one text message to one number. It will still show up in your outbox, so you aren't losing anything by using this method. This ability opens up a lot of possibilities, especially when coupled with the Gdata Contacts API. I have a few little scripts that I will be creating to send out weekly reminders to other people - and might use Google App Engine to make it even more useful and automated.

# Get URL handling support
import urllib2, urllib
# Get regular expression support
import re

# Google Account login credentials
email = 'YOUR_EMAIL'
password = 'YOUR_PASSWORD'

# Text message details
sendToNumber = 'NUMBER_TO_SEND_A_TEXT_TO'
messageToSend = 'I used Python to send this!'

# Set up an opener with HTTPCookieProcessor
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor())
urllib2.install_opener(opener)

# Set up login credentials for Google Accounts
# The 'continue' param redirects us to the Google Voice 
# homepage, and gives us necessary cookie info
loginParams = urllib.urlencode( {
    'Email' : email,
    'Passwd' : password,
    'continue' : 'https://www.google.com/voice/account/signin',
} )

# Perform the login. Cookie info sent back will be saved, so we remain logged in
# for future requests when using the opener
opener.open( 'https://www.google.com/accounts/ServiceLoginAuth',  loginParams)

# Need to load the homepage to find user specific data
googleVoiceHomeData = opener.open('https://www.google.com/voice/#inbox').read()

# Go through the home page and grab the value for the hidden
# form field "_rnr_se", which must be included when sending texts
match = re.search('name="_rnr_se".*?value="(.*?)"', googleVoiceHomeData)
_rnr_se = match.group(1)

# Set up parameters for sending text
sendTextParams = urllib.urlencode({
    '_rnr_se': _rnr_se,
    'phoneNumber': sendToNumber,
    'text': messageToSend
})

# Send the text, store the return value  
f  = opener.open('https://www.google.com/voice/sms/send/', sendTextParams)
data = f.read()
print data
f.close()

Right now, if it succeeds nothing spectacular happens - it simply prints out the response: {"ok":true,"data":{"code":0}}. If it fails, {"ok":false,"data":{"code":20}}. Just look at the "ok" value to see if it worked.

You may notice that I don't have much (any) error checking in the script, nor do I usually when putting together scripts quickly just to test an idea out. When I do create something actually useful using this above script, I will put in the appropriate "checker" code. Just be aware that a text message can be 160 characters long, if you go over the remaining characters will be sent in a separate message.

Have fun, and don't be evil.