Saturday, August 8, 2009

Python + Google Voice. Mass SMS and Iterative Calling at the Command Line

EDIT (Jan 2011): Google changed the way contacts are downloaded. Many of the scripts below will break. Use the updated version here.

EDIT (Oct 3 2009): Google changed a few things on their login page which broke my old scripts. I have updated the scripts in this post, including those that you can download. If you have been receiving "Could not log in with provided credentials" errors, these new scripts should fix that problem.

You can download a ready-to-use-no-Python-installation-required Windows executable version of the program
here. Just download, unzip, find the GVMassContact.exe file and double click to use. This version's source is identical to those of scripts located below.

Click here to navigate to the downloadable Python scripts.


I have already written twice about Google Voice (here and here), but the scripts in this installment are an improvement over both of the scripts provided the other posts.

I wanted to create an interactive command line script that would allow me to either send an SMS message or call everyone in one of my Google Contacts Groups. Google Voice does not yet let you send mass text messages, or call people in an iterative fashion, so I came up with a script to let me do just that.

Instead of using Googles gdata Python API to access my Google Contacts, I decided to use Pythons "csv" module to parse an exported csv file downloaded from my GMail account. Doing this lets you run the script without needing to download any other libraries, assuming you are using a version of Python that has all the features used in this script (I use 2.6.2 currently).

The first thing I did was create a "gvoice.py" module with several helpful classes. These allow you to:
  1. Log in
  2. Gather all Google Contacts into separate groups
  3. Selectively narrow down the contacts in a group
  4. Gather the phone numbers that you have entered
  5. Send SMS messages
  6. Place Calls
I am by no means a seasoned Python developer, but what I have created works well for my purposes. I tried to make the classes as loosely coupled with the UI as possible in case I want to put a GUI around it sometime, but for now I am doing everything at the command line (which I often prefer).

The GoogleVoiceLogin class will allow you to log in, get the "opener" which lets you keep your log in cookie data for subsequent requests, and gives you access to the _rnr_se value (the "key") that is used when sending SMS messages or placing a call. Most of the other classes in the "gvoice" module will accept the opener and key as parameters to their constructors.

I created a sample script that uses the classes in my "gvoice" module to:
  1. Prompt the user for his/her Google Account credentials
  2. Allow the user to select a Group from his/her Google Contacts Groups.
  3. Allow the user to narrow down the contact list of the Group (nothing permanent is done to the Contacts)
  4. Choose whether to send an SMS message or call everyone in the list
  5. If calling, it allows you to choose a number from your phone list, or enter a custom number (like the Google Voice site does when placing calls)
The script that I wrote assumes I always want to use the contact's mobile phone (the first one) to send SMS messages and place calls, but that is easy to change.

There are two files needed for this to work - here is the main driver program (gvMassContact.py):


from gvoice import *
import getpass
import sys
import re
import os

# Function used to create a separator
def separator():
return '-' * 25

def get_numeric_input(prompt):
try:
return int(raw_input(prompt))
except:
pass

# Function to clear the screen
def clear_screen():
if os.name == "posix":
# *nix systems
os.system('clear')
elif os.name in ("nt", "dos", "ce"):
# Windows
os.system('CLS')

# Main method to be run
def main():
# Log in
print "Please enter your Google Account credentials"
email = raw_input("User name: ")
password = getpass.getpass("Password: ")

gv = GoogleVoiceLogin(email, password)
if not gv.logged_in:
print "Could not log in with provided credentials"
sys.exit(1)
else:
print "Login successful!"

# Use the ContactLoader to download Google Contacts
contact_loader = ContactLoader(gv.opener)

# Use the ContactSelector to select the group and
# final list of contacts to contact
contact_selector = ContactSelector(contact_loader.contacts_by_group_list)

clear_screen()
group_list = contact_selector.get_group_list()
selected_group = None
while selected_group not in range(1, len(group_list)+1):
print "Your Google Groups"
print separator()
for group_item in group_list:
print "{0}: {1}".format(group_item[0], group_item[1])
print separator()
selected_group = get_numeric_input("Enter the index of the group to select: ")

clear_screen()
# Now that a group is selected, narrow down the list of people in the group
contact_selector.set_selected_group(selected_group)
removing = True
while removing:
print "Contact List"
print separator()
for contact_item in contact_selector.get_contacts_list():
print "{0}: {1}".format(contact_item[0], contact_item[1])
print separator()
try:
input_list = raw_input("Enter a list of the indexes (coma, space or otherwise delimeted)\nof those contacts you do not wish to contact this session.\nPress enter when finished: ")
if input_list != '':
contacts_to_remove_list = [int(match.group(1)) for match in re.finditer(r"(\d+)", input_list)]
contact_selector.remove_from_contact_list(contacts_to_remove_list)
else:
removing = False
except:
pass

clear_screen()
# Print final list
clear_screen()
print "Final List:"
print separator()
for contact_item in contact_selector.get_contacts_list():
print "{0}".format(contact_item[1])
selected_option = None
while selected_option not in [1, 2]:
print separator()
print "Options: "
print "1: Send Text"
print "2: Call"
print separator()
selected_option = get_numeric_input("Select which action to take: ")

# Send texts to all people in contact list
if (selected_option == 1):
print separator()
text_sender = TextSender(gv.opener, gv.key)
text = raw_input("Enter text message. Press enter when finished: ")
text_sender.text = text
for contact in contact_selector.get_contacts_list():
number = contact[1].mobile
if number == '':
print "{0} does not have a mobile number".format(contact[1])
else:
print "Sending message to {0} at {1}...".format(contact[1], contact[1].mobile),
text_sender.send_text(contact[1].mobile)
if text_sender.response:
print "Success!"
else:
print "Failed!!"

# Call all people in contact list
elif (selected_option == 2):
print separator()
number_dialer = NumberDialer(gv.opener, gv.key)

number_retriever = NumberRetriever(gv.opener)
phone_number_items = number_retriever.get_phone_numbers()

clear_screen()
# Get the forwarding number
forwarding_number_input = None
while forwarding_number_input not in range(1, len(phone_number_items) + 2):
print "Select forwarding number"
print separator()
for phone_number_item in phone_number_items:
print "{0}: {1}".format(phone_number_item[0], phone_number_item[1][0])
print "{0}: {1}".format(len(phone_number_items) + 1, "Other")
print separator()
forwarding_number_input = get_numeric_input("Choose from your previously entered numbers, or select \"Other\": ")

if forwarding_number_input in range(1, len(phone_number_items) + 1):
forwarding_number = phone_number_items[forwarding_number_input - 1][1]
else:
forwarding_number = ''
while not re.match(r"\(?\b[0-9]{3}\)?[-. ]?[0-9]{3}[-. ]?[0-9]{4}\b\Z", forwarding_number):
forwarding_number = raw_input("Enter the forwarding number to dial: ")
number_dialer.forwarding_number = forwarding_number

print separator()
# Loop through and make the calls
for contact in contact_selector.get_contacts_list():
number = contact[1].mobile
if number == '':
print "{0} does not have a mobile number".format(contact[1])
else:
input = None
while input not in ['', 'n','N', 'q', 'Q'] :
input = raw_input("Press enter to call {0} at {1} ('n' to skip, 'q' to quit): ".format(contact[1], contact[1].mobile))
if input == '':
print "Calling {0}....".format(contact[1]),
number_dialer.place_call(number)
if number_dialer.response:
print "Success!"
else:
print "Failed!!"
elif input .upper() == 'N':
pass
elif input.upper() == 'Q':
print "Call chain aborted."
break

if __name__ == "__main__":
main()



And here is the other required script (must be named "gvoice.py"):


import csv
import sys
import re
import urllib
import urllib2

class GoogleVoiceLogin:
def __init__(self, email, password):
# Set up our opener
self.opener = urllib2.build_opener(urllib2.HTTPCookieProcessor())
urllib2.install_opener(self.opener)

# Define URLs
self.loing_page_url = 'https://www.google.com/accounts/ServiceLogin'
self.authenticate_url = 'https://www.google.com/accounts/ServiceLoginAuth'
self.gv_home_page_url = 'https://www.google.com/voice/#inbox'

# Load sign in page
login_page_contents = self.opener.open(self.loing_page_url).read()

# Find GALX value
galx_match_obj = re.search(r'name="GALX"\s*value="([^"]+)"', login_page_contents, re.IGNORECASE)

galx_value = galx_match_obj.group(1) if galx_match_obj.group(1) is not None else ''

# Set up login credentials
login_params = urllib.urlencode( {
'Email' : email,
'Passwd' : password,
'continue' : 'https://www.google.com/voice/account/signin',
'GALX': galx_value
})

# Login
self.opener.open(self.authenticate_url, login_params)

# Open GV home page
gv_home_page_contents = self.opener.open(self.gv_home_page_url).read()

# Fine _rnr_se value
key = re.search('name="_rnr_se".*?value="(.*?)"', gv_home_page_contents)

if not key:
self.logged_in = False
else:
self.logged_in = True
self.key = key.group(1)

class ContactLoader():
def __init__(self, opener):
self.opener = opener
self.contacts_csv_url = "http://mail.google.com/mail/contacts/data/export"
self.contacts_csv_url += "?groupToExport=^Mine&exportType=ALL&out=OUTLOOK_CSV"

# Load ALL Google Contacts into csv dictionary
self.contacts = csv.DictReader(self.opener.open(self.contacts_csv_url))

# Create dictionary to store contacts and groups in an easier format
self.contact_group = {}
# Assigned each person to a group that we can get at later
for row in self.contacts:
if row['First Name'] != '':
for category in row['Categories'].split(';'):
if category == '':
category = 'Ungrouped'
if category not in self.contact_group:
self.contact_group[category] = [Contact(row)]
else:
self.contact_group[category].append(Contact(row))

# Load contacts into a list of tuples...
# [(1, ('group_name', [contact_list])), (2, ('group_name', [contact_list]))]
self.contacts_by_group_list = [(id + 1, group_contact_item)
for id, group_contact_item in enumerate(self.contact_group.items())]

class Contact():
def __init__(self,contact_detail):
self.first_name = contact_detail['First Name'].strip()
self.last_name = contact_detail['Last Name'].strip()
self.mobile = contact_detail['Mobile Phone'].strip()
self.email = contact_detail['E-mail Address'].strip()

def __str__(self):
return self.first_name + ' ' + self.last_name

# Class to assist in selected contacts by groups
class ContactSelector():
def __init__(self, contacts_by_group_list):
self.contacts_by_group_list = contacts_by_group_list
self.contact_list = None

def get_group_list(self):
return [(item[0], item[1][0]) for item in self.contacts_by_group_list]

def set_selected_group(self, group_id):
self.contact_list = self.contacts_by_group_list[group_id - 1][1][1]

# Return the contact list so far
def get_contacts_list(self):
return [(id + 1, contact) for id, contact in enumerate(self.contact_list)]

# Accept a list of indexes to remove from the current contact list
# Assumes 1 based list being passed in
def remove_from_contact_list(self, contacts_to_remove_list):
if self.contact_list == None:
return
for id in contacts_to_remove_list:
if id in range(0, len(self.contact_list)+1):
self.contact_list[id - 1] = None
self.contact_list = [contact for contact in self.contact_list if contact is not None]

class NumberRetriever():
def __init__(self, opener):
self.opener = opener
self.phone_numbers_url = 'https://www.google.com/voice/settings/tab/phones'
phone_numbers_page_content = self.opener.open(self.phone_numbers_url).read()

# Build list of all numbers and their aliases
self.phone_number_items = [(match.group(1), match.group(2))
for match
in re.finditer('"name":"([^"]+)","phoneNumber":"([^"]+)"',
phone_numbers_page_content)]

def get_phone_numbers(self):
return [(id + 1, (phone_number_item))
for id, phone_number_item
in enumerate(self.phone_number_items)]

class TextSender():
def __init__(self, opener, key):
self.opener = opener
self.key = key
self.sms_url = 'https://www.google.com/voice/sms/send/'
self.text = ''

def send_text(self, phone_number):
sms_params = urllib.urlencode({
'_rnr_se': self.key,
'phoneNumber': phone_number,
'text': self.text
})
# Send the text, display status message
self.response = self.opener.open(self.sms_url, sms_params).read()

class NumberDialer():
def __init__(self, opener, key):
self.opener = opener
self.key = key
self.call_url = 'https://www.google.com/voice/call/connect/'
self.forwarding_number = None

def place_call(self, number):
call_params = urllib.urlencode({
'outgoingNumber' : number,
'forwarding_number' : self.forwarding_number,
'subscriberNumber' : 'undefined',
'remember' : '0',
'_rnr_se': self.key
})

# Send the text, display status message
self.response = self.opener.open(self.call_url, call_params).read()


For the "gvMassContact.py" script to work, you need to have the "gvoice.py" file located in the same folder.

If you would rather not copy and paste, you can download the files here:

If you don't care about having the files separated, and would rather have to deal with just one script that you can keep on your desktop, you can download this file:
Here is a new, ready-to-use-no-Python-installation-required Windows executable version. Just download, unzip, find GVMassContact.exe in the GVMassContact folder, double click and go.

The only difference between the "allInOne.py" script and the others is that all of the classes contained in the "gvoice.py" file are located at the top of the file.

If you download the "gvAllInOne.py" script, you can keep it on your desktop and simply double click it to get the interactive command line session to come up.

Enjoy!

10 comments:

  1. Have you been able to get this to work on app engine? I know it takes away a bit of your command line ability :-)

    ReplyDelete
  2. Sam - not yet, but it is still something that I want to do. Glad to hear others are interested by the possibility.

    ReplyDelete
  3. Would this work on a mac? I tried using the "gvAllInOne.py" script and the editor came up with the script as well as the Python Shell, but there was no prompt for the user name as in Windows.

    Thanks for this awesome code by the way!

    ReplyDelete
  4. It should work on a Mac just fine - I have a Mac as well and just had to install Python 2.6.2 on it before it would run (I use some 2.6.2 features that 2.5.1 doesn't have). Once you have 2.6.2 installed, just type "python gvAllInOne.py" at the terminal (after navigating to the folder where it is located) and it should go! I tried for a bit to make it run when I clicked on it using the Python Launcher, but it came up in a completely unstyled terminal window (looked pretty ugly), so I resorted to running it through the terminal. I may create a standalone Mac App (like the Windows .exe) using py2app. If I do, I will supply a download.

    ReplyDelete
  5. Umm how do you use this??? I'm kinda clueless

    ReplyDelete
  6. Ameer -

    You have a couple of options (depending on which OS you are using), but the gvAllInOne.py script is probably your best bet. Make sure you have Python 2.6.2 installed on your computer, and then just double click on the file (you can also invoke the script at the command line by typing "python gvAllInOne.py"). It will prompt you for your Google Account email and password, and then will give you options to select a group from your Google Contacts. After that you can select how you want to contact them.

    ReplyDelete
  7. Thank you, this is fantastic!

    If you'd like to put up a paypal button I'll definitely show my appreciation.

    ReplyDelete
  8. So, I hate to be a buzz kill but I tried using this tonight and it seems to have issues logging in... Is this code deprecated?

    ReplyDelete
    Replies
    1. Yes, it is deprecated. At the top of the post you will see an alert telling you so. The updated scripts linked to work.

      Delete

Please comment!