Contents


Create a chatbot, Part 1

Create a news chatbot to deliver content through Facebook Messenger

Comments

Content series:

This content is part # of 3 in the series: Create a chatbot, Part 1

Stay tuned for additional content in this series.

This content is part of the series:Create a chatbot, Part 1

Stay tuned for additional content in this series.

This tutorial series shows how you can create a news chatbot through two messaging applications: Facebook and Slack. You can then use IBM Watson services to enhance the chatbot. In this tutorial, I show you how to develop and deploy this chatbot on the Facebook Messenger platform. The news chatbot in this series uses developerWorks content as an example, but you can modify the content source to meet your own needs.

How the example chatbot works

The chatbot brings news and tutorials to you, and for the chatbot to do its job, you must identify the topics that you are interested in.

  1. In Facebook Messenger, open a chat with developerWorks News Bot.
  2. Specify the topics that interest you. Screen shot of chat on Facebook Messenger
    Screen shot of chat on Facebook Messenger
  3. You are shown a list of recent tutorials based on your interests. Continuing the chat
    Continuing the chat
  4. You can drill down to specific topics. For each tutorial, you can have the chatbot give you a summary or even read the summary aloud for you. From there, you can see the full tutorial on the developerWorks website.

    The chatbot also supports searching for articles by using Google-like queries.

Note: Optionally, you can also interact with the dW News chatbot by using text or voice messages. You can use Facebook Messenger to enter a voice message, and the chatbot transcribes the message to text before responding. You can also get the chatbot to read a tutorial summary for you. This voice feature is available in the source code for this chatbot in GitHub, but it is not deployed in the demo application. You can find this chatbot application's full source code on GitHub: https://github.com/juntao/dwnewsbot

Create a Java servlet-based chatbot application

Because this dW news chatbot is on Facebook Messenger, the application you develop is a "webhook" that listens for messages from Facebook users, and then sends the chatbot's response back to Facebook Messenger. In the world of Java™ applications, you write a servlet that is mapped to the webhook URL.

To use the framework, you can extend the abstract class BaseServlet and implement the converse method. The converse method String parameter human is the text message the chat user sent. In a simple case, the converse method can just return a String response to the user. For example, the following simple implementation echoes the user message back to the user.

public Object converse (String human, ConcurrentHashMap<String, Object> context) {
    return "You just said: " + human;
}

The converse method can also return a JSONObject that specifies a structured message per Facebook Messenger specifications. For instance, the following code snippet returns an image with a button that says "Click me" below it. If the user selects the button, the next message the converse method receives will have CLICKME as the input for the human parameter.

public Object converse (String human, ConcurrentHashMap<String, Object> context) {
    try {
        JSONObject payload = new JSONObject();
        payload.put("template_type", "button");
        payload.put("image_url", "http://host.com/img.png");
        
        JSONArray buttons = new JSONArray ();
        buttons.put((new JSONObject())
            .put("type", "postback")
            .put("title", "Click me")
            .put("payload", "CLICKME"));
        payload.put("buttons", buttons);

        return payload;
    } catch (Exception e) {
        e.printStackTrace();
        return "";
    }
}

What if your chatbot needs to send back multiple messages to respond to a user message? Well, it can return a Java List collection that contains a mixture of String and JSONObject objects.

If your chatbot is a simple command and response bot, your converse method could just contain a list of if/else statements to map out how to respond to each input. But most chatbots are more complex than that. Complex chatbots need to maintain a conversation state so that it can respond based on the context of the conversation. You can save any conversation state object in the Java HashMap parameter called context. In the following code, the chatbot first asks the user to enter a search query, and in the next exchange, it remembers that the user's input is a search query and performs a search by using Hibernate Search and Lucene. The search results are again saved into the context variable so that the chatbot knows how to page through it when the user selects Next.

public Object converse (String human, ConcurrentHashMap<String, Object> context) {

    String cstr = classifyText(human);
    if ("SEARCH".equalsIgnoreCase(cstr)) {
        context.put("search", true);
        return "Enter your search query here. For example, you can enter \"Java Stream\"";
    }

    if (context.get("search") != null) {
        List <NewsItem> items = dm.searchNewsItems(human);
        context.put("items", items);
        context.remove("search");

        List replies = new ArrayList();
        replies.add ("Search results for: " + human);
        replies.add(dm.replyItems(items, true));
        return replies;
    }
    ...
}

As a bonus of the contextHashMap, the framework automatically populates it with session-scoped data. For example, the converse method can always access the Facebook user ID by using the sender_id field in the context variable. The chatbot application can retrieve and save the user's profile data.

public Object converse (String human, ConcurrentHashMap<String, Object> context) {
    User user = dm.getUser((String) context.get("sender_id"));
    if (user == null) {
        user = new User ();
        user.setFbId((String) context.get("sender_id"));

        HashMap profile = getUserProfile(user.getFbId());
        if (profile != null && !profile.isEmpty()) {
            user.setFirst_name((String) profile.get("first_name"));
            user.setLast_name((String) profile.get("last_name"));
             user.setProfile_pic((String) profile.get("profile_pic"));
            user.setLocale((String) profile.get("locale"));
            user.setGender((String) profile.get("gender"));
            try {
                user.setTimezone((Integer) profile.get("timezone"));
            } catch (Exception e) {
                user.setTimezone(0);
            }
        }

        dm.saveUser(user);
        new_user = true;
    }
    ...
}

Finally, for a news delivery chatbot, it is important to reach out to users when there is news. In other words, the chatbot should not always wait to respond to a user's command. In a Java application server, this is easily done by setting up a Quartz scheduler task. I have configured the task worker class SendNewsWorker to run at 4 p.m. UTC three days a week in the StartupServlet.

private static final String CRON_EXPRESSION_2 =
      "0 0 16 ? * MON,WED,FRI";

// ... ...

JobDetail jobDetail2 = new JobDetail(
      "MyJob2", "MyJobGroup", SendNewsWorker.class);
JobDataMap dataMap2 = new JobDataMap ();
dataMap2.put("emf", emf);
dataMap2.put("scontext", getServletContext());
jobDetail2.setJobDataMap (dataMap2);

CronTrigger cronTrigger2 = new CronTrigger(
      "MyTrigger2", "MyTriggerGroup");
CronExpression cexp2 = new CronExpression(CRON_EXPRESSION_2);
cronTrigger2.setCronExpression(cexp2);

scheduler.scheduleJob(jobDetail2, cronTrigger2);

The following code snippet is from the SendNewsWorker class. It sends news to the user as a structured carousel JSONObject, using the user's Facebook ID retrieved from the database.

// Get all users from database
List <User> users = dm.getActiveUsers();
for (User user : users) {
    List <String> faves = user.getFavesList();
    List <NewsItem> items = new ArrayList <NewsItem> ();
    for (String fave : faves) {
        List <NewsItem> nis = dm.getNewsItems(fave);
        if (nis == null || nis.isEmpty()) {
            continue;
        }
        for (NewsItem ni : nis) {
            items.add(ni);
        }
    }

    if (items.isEmpty()) {
        continue; // there is no update for today
    }

    // send the replies to the user proactively
    try {
        BaseServlet.sendReply("Hey, " + user.getFirst_name() + ", here are some tutorials you might have missed.", user.getFbId());
        BaseServlet.sendReply(dm.replyItems(items, true), user.getFbId());
        BaseServlet.sendReply("To stop future news delivery messages, please text STOP", user.getFbId());
    } catch (Exception e) {
        e.printStackTrace();
    }

    user.setUpdateDate(new Date ());
    dm.saveUser(user);
}

The replyItems method that is shown in the code displays the tutorials carousel in a series of List objects (that is, titles, subtitles, image_urls, button_titles, and button_payloads). The method then calls the BaseServletcreateCarousel utility method to create a JSON object for the carousel according to Facebook Messenger specifications.

Connect the chatbot to Facebook Messenger

In this section, I walk you through the process of creating a Facebook Messenger chatbot and connecting your application to it.

  1. The public "face" of a Facebook Messenger bot is a Facebook page. To create a new Facebook page, go to https://www.facebook.com/pages/create/ and follow the steps. After the page is created, you can customize its appearance by adding an icon and call-to-action buttons. Screen showing the Facebook dW Bot page
    Screen showing the Facebook dW Bot page
  2. To create a Facebook app for the chatbot, go to https://developers.facebook.com/ and log in. From the My Apps menu in the upper right, select Add a New App. You can select Basic Setup at the bottom of the pop-up window because you do not need to integrate with anything yet. The Add a new app window
    The Add a new app window
  3. From the Facebook app dashboard, use the Add Product link in the left panel to add "Messenger" integration.
  4. On the Messenger product tab, create a token to link your app with the page. Your app's webhook, which you will set up soon, will receive messages sent to the specified page. Creating a token window
    Creating a token window
  5. Enter the token into the NewsServlet class in your Java chatbot application. Notice that the NewsServlet class has other empty key or token fields—those are optional web services the chatbot could use to improve its intelligence. These additional services are covered in the other tutorials in the series.
  6. Run the following command from your computer with the page access token for the page subscription to take effect.
    	curl -X POST "https://graph.facebook.com/v2.6/me/subscribed_apps?access_token=<PAGE_ACCESS_TOKEN>"
  7. On the Messenger product tab, under the Webhooks section, subscribe the app to the page you just created. Make sure that you select the messages and messaging_postbacks options to receive both messages and button clicks from Facebook Messenger. The Webhooks window
    The Webhooks window

You need to set up a webhook URL for your application to receive the subscribed messaging events. To do that, you must deploy your application as a public web application. IBM Bluemix is a great choice for deployment.

Deploy the chatbot on Bluemix

Now you're ready to deploy the chatbot web application to IBM Bluemix. First, you need to set up two containers on Bluemix: an Apache Tomcat application server to run the web application for the webhook and a MySQL relational database for the application to store application data (for example, the user profile data for Facebook users and the contents of developerWorks tutorials).

IBM Bluemix is based on the Cloud Foundry platform. You will need to install Cloud Foundry command-line tools and their Bluemix extensions to manage your application containers in Bluemix.

The MySQL container is provided by ClearDB, a third-party provider on the Bluemix platform. The free plan from ClearDB limits the database size to 5 MB, which is not sufficient for your application because the downloaded dW tutorial content quickly exceeds 5 MB. You must upgrade to a paid ClearDB plan if you plan to run the dW news chatbot yourself.

  1. In IBM Bluemix, create an Apache Tomcat container.
  2. Create a MySQL database and select a ClearDB plan. Bluemix screen to select a MySQL database
    Bluemix screen to select a MySQL database
  3. Link the MySQL database with the Tomcat application. Window to link the services
    Window to link the services
  4. From the Bluemix dashboard, you can open the Tomcat application and review the database settings, including the connection URL and auto-generated user name and password. Reviewing the database settings
    Reviewing the database settings
  5. Import an SQL script to set up the database schema. You can run the following command from any computer with the MySQL client library installed. The schema.sql file is located in the bin directory of the project source code.
    	mysql -h HOST -u USERNAME -pPASSWORD DBNAME < schema.sql
  6. In the context.xml file of your web application, enter the database connection URL, user name, and password. This file is located in the src/main/resources/META-INF directory of the project source code.
    	<?xml version='1.0' encoding='utf-8'?>
    	<Context>
    	  <Resource name="jdbc/dwnews" auth="Container"
    	    driverClass="com.mysql.jdbc.Driver"
    	    jdbcUrl="jdbc:mysql://host:3306/DBNAME?autoReconnect=true"
    	    user="USERNAME"
    	    password="PASSWORD"
    	    preferredTestQuery="SELECT 1"
    	    testConnectionOnCheckin="true"
    	    idleConnectionTestPeriod="300"
    	    factory="org.apache.naming.factory.BeanFactory"
    	    type="com.mchange.v2.c3p0.ComboPooledDataSource"
    	    maxPoolSize="4"
    	    minPoolSize="2"
    	    acquireIncrement="1"/>
    	</Context>
  7. To build the Java application you just developed, you need Apache Maven. After you install Maven, use the following command to build the WAR file.
    	mvn clean package

    You can now deploy the WAR file by using the Bluemix cf push command. See the Bluemix documentation for details on how to set up the manifest.yml to deploy the WAR file to your Bluemix Tomcat container.

Your webhook is available at https://your-bluemix-appname.mybluemix.net/facebook/ibmdw.

Test and publish your chatbot

Enter the Bluemix application URL as the webhook URL in the Facebook app console's Webhooks tab. For the Verify Token field, you can enter any string that you like, and your chatbot framework automatically handles it.

Enter the webhook URL
Enter the webhook URL

Now you can test the chatbot. Go to the bot's Facebook page and send a message to the page. The Facebook Messenger window opens and you will see it respond.

All Facebook Messenger chatbots must be approved by Facebook before they can be made available to the public. During the testing period, you can add testers to the application by manually adding their Facebook ID in the apps dashboard. After you are finished testing, you can click Request Permission to submit your application for approval. You will be asked to provide a privacy policy (https://www.iubenda.com/en is a website that can generate one for you for free) and a video demo of the application in action.

Note: Because the application sends news tutorials to the user three days a week, you must select the pages_messaging_subscriptions permission for review. Without this permission, the chatbot can only send messages to the user after the user initiates requests (that is, to send reply messages to the user within a 24-hour window after the user explicitly messaged the chatbot).

It will take a few days for Facebook to approve the chatbot, and you are notified when they approve it. At that point, go back to the Facebook app dashboard and flip the switch to make the app public.

Summary

In this tutorial, I described the process to build and publish a Facebook Messenger chatbot on the Bluemix platform. This approach shows you the nuts and bots (pun intended) of how a chatbot works, and gives you much more freedom in your hosting options (as Java programming is almost universally supported among cloud and on-premise IT providers).

In the next tutorial, I will describe the process for developing an application for the dW news bot on the Slack messaging platform.


Downloadable resources


Comments

Sign in or register to add and subscribe to comments.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Cognitive computing
ArticleID=1038447
ArticleTitle=Create a chatbot, Part 1: Create a news chatbot to deliver content through Facebook Messenger
publish-date=02062017