Email to SMS Service with Python

Text Messaging and Email remain the top forms of communication, even with the recent wave of Social Media platforms such as Facebook, Twitter, Instagram, etc. Email is still the king of all forms of electronic communication. Maybe it has something to do with it's decentralized exchange, or maybe something to do with being able to setup your own server and control who you communicate with through it. Either way, SMS and Email are both useful. Recently I had the need to allow a conversation to start with a SMS be delivered to an Email inbox and then replied to from the email. Here is how I was able to get this accomplished with the Python programming language and it's amazing libraries.

 

The workflow of communication.

Imagine you need to receive text messages for your business and you don't want to reply from your phone all the time. Or you need to manage 20 different texting phone numbers, but you want to be able to reply from a single location. I have developed a solution for these exact situations. Here is a little workflow diagram.

How is this accomplished?

This is actually relatively simple in concept. There are only 4 pieces to the entire system:

  1. Receive SMS
  2. Deliver message as Email
  3. Read Email replies
  4. Deliver Email reply as SMS

I chose to use Twilio to purchase phone numbers and to send/receive sms messages. The connection to Twilio is super simple and takes only a few lines of python code. On to tackling the first task.

Receive SMS

Twilio has made this ridiculously easy! After purchasing the number you want, you simply configure the number to send and SMS messages to a URL which it will post data to. Whenever an SMS is received by a number you have in Twilio, it will be POSTed to the URL you configured for that number. This is what it looks like where you configure this Webhook:

The data received by this Webhook should look something like the following:

{"From": "15555555555", "To": "18002335555", "Body": "This is the actual SMS Message sent"}

Of course, this data will be in the $_POST array if you are using PHP. There is some additional data included in the POST body, but I opted to ignore it, as it wasn't important to me during this setup. We now have the text message, it's time to send it off in an email. Which is the first tricky part.

Deliver SMS Message to Email

However you want to determine which email address the message should be delivered to is up to you. Maybe you could have a unique Webhook for each email address, or you could keep a list of Phone number Ranges, and select an email address based on that. Twilio also gives you the location or city the message came from, maybe you want to send it to your Sales Rep. for that area. At any rate, it is time to create the email to send. For this example we will send all emails to a single email address salesperson1@domain.com

Here is the PHP using PHP Mailer and a gmail account to send that email to your sales person.


date_default_timezone_set('Etc/UTC');
require '../PHPMailerAutoload.php';

//Create a new PHPMailer instance
$mail = new PHPMailer;

$mail--->isSMTP();
$mail->SMTPDebug = 0;
$mail->Debugoutput = 'html';

$mail->Host = 'smtp.gmail.com';
$mail->Port = 587;
$mail->SMTPSecure = 'tls';
$mail->SMTPAuth = true;

$mail->Username = "username@gmail.com";
$mail->Password = "yourpassword";

// GMail requires this be your account information
$mail->setFrom('username@gmail.com', 'Firstname Lastname');

// Here is the Tricky Part
// We are going to need an email server that can have a Catchall for this method to work
// Or at least configure a single email address with an Alias for each phone number you have
$mail->addReplyTo($_POST['To'] . '@example.com', $_POST['From']);

// Alternatively you can solely use the Subject to handle routing
//$mail->addReplyTo('smsemailreplies@domain.com', 'SMS Reply Handler');

// Set this address to send to your sales person
$mail->addAddress('salesperson1@domain.com', 'John Doe');

//Set the subject line
$mail->Subject = 'SMS Message #' . $_POST['From'] .'#';

// Alternate Method
//$mail->Subject = 'SMS Message From: ' . $_POST['From'] . ' ##' . $_POST['To'] . '##';

// Set the Text body (We do not want an HTML Email)
$mail->Body = $_POST['Body'];

//send the message, check for errors
if (!$mail->send()) {
    echo "Mailer Error: " . $mail->ErrorInfo;
}

That's it for sending the email to the Sales Person. Now we can move on to handling his reply to the message.

Read email replies & Send SMS

This is by far the most fuzzy or complicated part of the whole process. For this step I have used Python, it has some amazing libraries which just aren't available in other languages or are not as mature. This process will need to run every minute or so, to have a good flow of messages without huge delays. You can create this as a simple CLI script run by Cron every minute or two.

On your server create a Python file: process-replies.py

#!/usr/bin/env python

import re
import email

import imapclient
from imapclient import IMAPClient
from backports import ssl

from twilio.rest import TwilioRestClient

# Twilio Account Information
ACCOUNT_SID = "ACXXXXXXXXXXXXXXXXX"
AUTH_TOKEN = "YYYYYYYYYYYYYYYYYY"

# Email Account Information
HOST = 'imap.mailserver.com'
USERNAME = '18002335555@domain.com'
PASSWORD = 'password'
PHONE = '+18002335555'

# Define some Regex matches for our emails
regex_quote = re.compile("[Oo]n (.*) [Ww]rote:", re.DOTALL)
regex_subject = re.compile("#([0-9]+)#", re.DOTALL)

# Alternate using a single inbox for multiple receiving numbers
#regex_subject = re.compile("([0-9]+) ##([0-9]+)##", re.DOTALL)

# We now need to connect to the IMAP Server
context = imapclient.create_default_context()

# don't check if certificate hostname doesn't match target hostname
context.check_hostname = False

# don't check if the certificate is trusted by a certificate authority
context.verify_mode = ssl.CERT_NONE

server = IMAPClient(HOST, use_uid=True, ssl=True, ssl_context=context)
server.login(USERNAME, PASSWORD)

# Select the Inbox for this email address
select_info = server.select_folder('INBOX')

messages = []
if select_info['EXISTS'.encode('ascii')] > 0:

	# we have something to process
	# connect to twilio as we will need it a little later
	twilio_api = TwilioRestClient(ACCOUNT_SID, AUTH_TOKEN)


	messages = server.search(['NOT', 'DELETED'])
	res = server.fetch(messages, ['RFC822'])

	for message_id in messages:
		msg = email.message_from_bytes(res[message_id]['RFC822'.encode('ascii')])

		# Load in the initial data needed
		msg_subject = msg.get('Subject')
		msg_body = ''

		matches = regex_subject.findall(msg_subject)
		if len(matches) == 1:
			if len(matches[0]) == 2:
				msg_twilio_number = PHONE
				msg_extern_number = matches[0][0]
			else:
				# This email is malformed, Reply with an error & delete it
				# Then move to the next email
                server.delete_messages(message_id)
				continue
		else:
			# This email is malformed, Reply with an error & delete it
			# Then move to the next email
            server.delete_messages(message_id)
			continue

        # Get the email body
		if not msg.is_multipart():
			#print(msg.get_payload())
			msg_body = msg.get_payload(decode=True)
		else:
			parts = msg.get_payload()
			msg_body = ''
			for part in parts:
				if part.get_content_type() == 'text/plain':
					# We finally found the text message needed
					msg_body = part.get_payload(decode=True)
					break

        # clean up the message text
        msg_body = msg_body.decode('utf-8')
		msg_body = msg_body.replace("=0A", "\n").replace("=0D", "\r").replace("\r\n", " ").replace("\n", " ")
		msg_body = regex_quote.split(msg_body)[0].strip()

		# DEBUG:
		# print(msg_body)

		# TODO: Send the message off to Twilio
		# We are sending a response to the customer here
        message = twilio_api.messages.create(to=msg_extern_number, from_=msg_twilio_number,
                                             body=msg_body)

		# we are all done with this message, Delete it from the server
		server.delete_messages(message_id)

# Delete the messages that are flagged for deletion.
server.expunge()

# Logout of IMAP connection to the email server
server.logout()

Getting it all going

Now that just about does it for receiving and send SMS Messages to and from emails. The last piece of the system is to have this python script run as a service or in a cron job every minute or two. I have mine running as a microservice which check the email inbox every 60 seconds. I also have more alterations which allow for my single microservice to handle emails from many different phone numbers and email addresses.