Community

Getting Started with Piwik on IBM Bluemix™ using MySQL, SendGrid and Twilio

Share this post:

PIWIK

BluemixRox

Here’s the latest post in my “BluemixRox” series on deploying popular open-source projects onto Bluemix. In past posts, I’ve shared how to deploy Drupal, WordPress and Ghost.js. I’ve now turned my focus to Piwik, a PHP-based open source web analytics platform that gives valuable insights into your website’s visitors, your marketing campaigns and much more, so you can optimize your strategy and online experience of your visitors. While exploring Ghost.js, I stumbled across instructions on integrating Piwik with Ghost.js and became intrigued with the tech in general. I discovered that Piwik is used by over a million websites for things like web, ecommerce, server log, and intranet analytics. After a few mods, I’m happy to say – Piwik runs great on Bluemix! With the ease of standing up Bluemix services I was also able to get Email and SMS reporting for my Piwik reports. With the opportunity to better secure the app, I happily leveraged the built-in SSL context provided to my Bluemix Piwik app and forced the app to always use it while tweaking the code to remove the MySQL and SendGrid credentials from being persisted in clear text within a config file. Sweet! As extra sugar, the Piwik project also offers a free mobile app to access your Piwik dashboard hosted on Bluemix “on the go”. If you’re still reading and would like to setup your own Piwik instance on IBM Bluemix, here’s how:

Plan of Attack

  • Phase 1: Download and configure Piwik project code for deployment. This configuration will also automatically configure email support if a SendGrid service has been provisioned and declared in the manifest.yml.
  • Phase 2: Add support for Twilio SMS Support of Piwik Reports {Optional}

Showtime! Track Responsibly 🙂 [Steps Verified for Piwik 2.3.0, 2.4.0, 2.4.1 & 2.5.0]

Phase 1 – Download and Configure Piwik project code for deployment

  1. Download a current stable version of Piwik from http://builds.piwik.org/piwik-latest.zip
  2. Untar or unzip the download and change to the base piwik directory, usually “piwik-latest/piwik
  3. Create a manifest.yml file in the base piwik directory to describe and simplify deployment using the following structure:[code]—
    applications:
    – name: &lt;<b>replace_me_with_your_app_name</b>&gt;
    memory: 256M
    instances: 1
    host: &lt;<b>replace_me_with_your_app_name</b>&gt;
    domain: mybluemix.net
    path: .
    buildpack: https://github.com/joshisa/cf-php-build-pack.git
    services:
    – piwikdb[/code]
  4. Assuming you have downloaded the cf command-line tool from here or via the IBM Bluemix ACE UI, provision a new MySQL or ClearDB Service within your organization with the name defined within the manifest.yml. For example, [code]$cf login -a https://api.ng.bluemix.net
    $cf create-service mysql 100 piwikdb[/code]

  5. tip_iconOptional Step – If SendGrid email reporting support is desired, you will need to also provision or reuse a SendGrid service and add the service name to the services section of your manifest.yml.
    [code]$ cf cs sendgrid free piwikmail
    $ echo " – piwikmail" >> manifest.yml[/code]
  6. Piwik provides a mechanism to execute custom code before it runs. Piwik will load bootstrap.php if it exists in the top level Piwik folder. This file is not overwritten by subsequent software updates. Click here for more info about bootstrap.php. To inform Piwik about the rich set of information available within the Environment Variables provided by Bluemix, we will populate a series of global values within bootstrap.php to provide Piwik code with granular access of Bluemix environment configuration details. Place bootstrap.php within the top level piwik folder alongside index.php.
    [code language=”php”]
    <?php
    # Copyright IBM 2014
    # Filename: bootstrap.php
    # Purpose: Populate global environment vars with Bluemix runtime values

    $_ENV["SQLDB"] = NULL;
    $_ENV["SQLHOST"] = NULL;
    $_ENV["SQLPORT"] = NULL;
    $_ENV["SQLUSER"] = NULL;
    $_ENV["SQLPASSWORD"] = NULL;
    $_ENV["MAILUSER"] = NULL;
    $_ENV["MAILPASSWORD"] = NULL;
    $_ENV["MAILHOST"] = NULL;
    $_ENV["SMSACCOUNT"] = NULL;
    $_ENV["SMSTOKEN"] = NULL;
    $_ENV["SMSURL"] = NULL;

    $application = getenv("VCAP_APPLICATION");
    $application_json = json_decode($application,true);
    if (isset($application_json["application_uris"])) {
    $_ENV["APPURIS"] = $application_json["application_uris"];
    }

    $services = getenv("VCAP_SERVICES");
    $services_json = json_decode($services,true);
    if (isset($services_json)) {
    if (isset($services_json["mysql-5.5"][0]["credentials"])) {
    $mysql_config = $services_json["mysql-5.5"][0]["credentials"];
    $_ENV["SQLDB"] = $mysql_config["name"];
    $_ENV["SQLHOST"] = $mysql_config["host"];
    $_ENV["SQLPORT"] = $mysql_config["port"];
    $_ENV["SQLUSER"] = $mysql_config["user"];
    $_ENV["SQLPASSWORD"] = $mysql_config["password"];
    }

    if (isset($services_json["cleardb"][0]["credentials"])) {
    $mysql_config = $services_json["cleardb"][0]["credentials"];
    $_ENV["SQLDB"] = $mysql_config["name"];
    $_ENV["SQLHOST"] = $mysql_config["hostname"];
    $_ENV["SQLPORT"] = $mysql_config["port"];
    $_ENV["SQLUSER"] = $mysql_config["username"];
    $_ENV["SQLPASSWORD"] = $mysql_config["password"];
    }

    if (isset($services_json["sendgrid"][0]["credentials"])) {
    $sendgrid_config = $services_json["sendgrid"][0]["credentials"];
    $_ENV["MAILUSER"] = $sendgrid_config["username"];
    $_ENV["MAILPASSWORD"] = $sendgrid_config["password"];
    $_ENV["MAILHOST"] = $sendgrid_config["hostname"];
    }

    if (isset($services_json["user-provided"][0]["credentials"])) {
    $twilio_config = $services_json["user-provided"][0]["credentials"];
    $_ENV["SMSACCOUNT"] = $twilio_config["accountSID"];
    $_ENV["SMSTOKEN"] = $twilio_config["authToken"];
    $_ENV["SMSURL"] = $twilio_config["url"];
    }
    }
    ?>
    [/code]

  7. tip_iconOptional Step – While not obvious, the host field within the database installation wizard page will require both host and port data using the form <host>:<port>. The install wizard will require you to manually enter the MySQL Service details found in VCAP_SERVICES. The mod steps below are a convenience tweak to pre-populate the install wizard with Bluemix MySQL service details and is very useful for scenarios like:
    • Repeat unique deployments to Bluemix
    • You plan on sharing your project for others to fork
    • You plan on tweaking piwik code
    • Experimenting with other Bluemix MySQL services

    To modify the built-in Piwik configuration install wizard slightly to make it smarter about the DB configuration info available to the application via the VCAP_SERVICES environment variable, browse to the sub-folder /piwik/plugins/Installation/FormDatabaseSetup.php and replace line 80 …
    [code language=”php” firstline=”69″ highlight=”80″]

    # source file: /piwik/plugins/Installation/FormDatabaseSetup.php
    $this -> addElement(‘select’, ‘adapter’)
    ->setLabel(Piwik::translate(‘Installation_DatabaseSetupAdapter’))
    ->loadOptions($adapters)
    ->addRule(‘required’, Piwik::translate(‘General_Required’, Piwik::translate(‘Installation_DatabaseSetupAdapter’)));

    $this->addElement(‘submit’, ‘submit’, array(‘value’ => Piwik::translate(‘General_Next’) . ‘ »’, ‘class’ => ‘submit’));

    // default values
    $this->addDataSource(new HTML_QuickForm2_DataSource_Array(array(
    ‘host’ => ‘127.0.0.1’,
    ‘tables_prefix’ => ‘piwik_’,
    )));
    }

    /**
    * Creates database object based on form data.
    *
    * @throws Exception|Zend_Db_Adapter_Exception
    * @return array The database connection info. Can be passed into Piwik::createDatabaseObject.
    */
    public function createDatabaseObject()

    [/code]

    with lines 3-6 of this code snippet …
    [code lang=”php” highlight=”3,4,5,6″]
    // IBM Bluemix friendly values
    $this->addDataSource(new HTML_QuickForm2_DataSource_Array(array(
    ‘host’ => $_ENV["SQLHOST"].’:’.$_ENV["SQLPORT"],
    ‘username’ => $_ENV["SQLUSER"],
    ‘password’ => $_ENV["SQLPASSWORD"],
    ‘dbname’ => $_ENV["SQLDB"],
    ‘tables_prefix’ => ‘piwik_’,
    )));
    [/code]

  8. Next, to improve security of our deployment, we’ll modify the Piwik ini parsing function to read directly from our bootstrap.php environment variables. The advantage with this approach is that our MySQL DB credentials will no longer need to be persisted in clear text within the config.ini.php file generated by the Piwik application. Unfortunately, to accomplish this – we’ll need to revert the scan mode from INI_SCANNER_RAW to its default (INI_SCANNER_NORMAL). Browse to the sub-folder /piwik/libs/upgradephp/upgrade.php. We’ll be modifying one helper function _parse_ini_file and adding one new helper function updateINIWithVCAPSERVICES. Replace the following highlighted line …
    [code language=”php” firstline=”114″ highlight=”134″]

    /**
    * parse_ini_file() replacement.
    * source file: /piwik/libs/upgradephp/upgrade.php
    * Behaves like parse_ini_file($filename, $process_sections);
    *
    * @author Andrew Sohn <asohn (at) aircanopy (dot) net>
    * @author anthon (dot) pang (at) gmail (dot) com
    *
    * @param string $filename
    * @param bool $process_sections (defaults to false)
    * @return array
    */
    if(function_exists(‘parse_ini_file’)) {
    // provide a wrapper
    function _parse_ini_file($filename, $process_sections = false) {
    if(!file_exists($filename)) {
    return false;
    }
    // Note: INI_SCANNER_RAW is important here!
    return parse_ini_file($filename, $process_sections, INI_SCANNER_RAW);
    }

    } else {

    [/code]
    with …
    [code language=”php” firstline=”126″ highlight=”134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154″]

    if(function_exists(‘parse_ini_file’)) {
    // provide a wrapper
    function _parse_ini_file($filename, $process_sections = false) {
    if(!file_exists($filename)) {
    return false;
    }
    // Note: INI_SCANNER_RAW is important here!
    // Choices:
    // 1. Use INI_SCANNER_RAW.
    // Pros: Allows use of config values in format of ${@Piwik(c)}.
    // Cons: Must keep username/pw in clear text within config.ini.php
    // 2. Use INI_SCANNER_NORMAL.
    // Pros: Username/pw of DB and Mail are now retained only in environment variables and not clear text
    // Cons: config.ini.php file will cause error if values are in the format of ${@Piwik(c)}
    // For more info on impact: https://dev.piwik.org/trac/ticket/5349

    // Disable Original (Choice 1)
    // return parse_ini_file($filename, $process_sections, INI_SCANNER_RAW);

    // Enable Choice 2
    # Modified wrapper function. Capture result from built-in parse_ini_file function or false
    $ini_temp = parse_ini_file($filename, $process_sections, INI_SCANNER_NORMAL);
    # Test if result is an array. If so, let’s modify the array to reflect VCAP_SERVICES values
    if (is_array($ini_temp)) {
    updateINIWithVCAPSERVICES($ini_temp, "");
    }
    # Return result. Could be modified or original.
    return $ini_temp;
    }
    } else {

    [/code]
    and insert/embed the following code snippet within upgrade.php as a new helper function at the bottom of the file
    [code language=”php”]
    /**
    * source file: /piwik/libs/upgradephp/upgrade.php
    * Function Helper to set the DB Configuration details based on the VCAP_SERVICES
    * environment variable. This is a useful modification to support
    * dynamic services within the IBM Bluemix platform.
    * Using this function has the added benefit of removing MySQL & SendGrid configuration
    * details from the installation generated config.ini.php file – which improves security.
    *
    * @author Sanjay Joshi <joshisa (at) us (dot) ibm (dot) com
    * @param array $arrayObject
    * @param string $section
    * @return none. Modifies param array by reference
    */

    function updateINIWithVCAPSERVICES(&$arrayObject, $section) {
    if (is_array($arrayObject)) {
    foreach($arrayObject as $key=> &$data) {
    if(is_array($data)) {
    updateINIWithVCAPSERVICES($data, $key);
    } elseif ($section === ‘database’ && ($key === ‘host’)) {
    $arrayObject[$key] = $_ENV["SQLHOST"];
    } elseif ($section === ‘database’ && ($key === ‘port’)) {
    $arrayObject[$key] = $_ENV["SQLPORT"];
    } elseif ($section === ‘database’ && ($key === ‘username’)) {
    $arrayObject[$key] = $_ENV["SQLUSER"];
    } elseif ($section === ‘database’ && ($key === ‘password’)) {
    $arrayObject[$key] = $_ENV["SQLPASSWORD"];
    } elseif ($section === ‘database’ && ($key === ‘dbname’)) {
    $arrayObject[$key] = $_ENV["SQLDB"];
    } elseif ($section === ‘mail’ && ($key === ‘username’)) {
    $arrayObject[$key] = $_ENV["MAILUSER"];
    } elseif ($section === ‘mail’ && ($key === ‘password’)) {
    $arrayObject[$key] = $_ENV["MAILPASSWORD"];
    }
    }
    }
    }
    [/code]

  9. Next, browse to the sub-folder /piwik/plugins/Installation/Controller.php. We’ll be modifying two functions: createConfigFile and addTrustedHosts. Modify the function addTrustedHosts by replacing …
    [code language=”php” firstline=”572″ highlight=”575″]

    private function addTrustedHosts($siteUrl)
    {
    $trustedHosts = array();

    // extract host from the request header

    [/code]
    with …
    [code language=”php” firstline=”572″ highlight=”575, 576, 577, 578, 579″]

    private function addTrustedHosts($siteUrl)
    {
    $trustedHosts = Config::getInstance()->General[‘trusted_hosts’];

    if (!is_array($trustedHosts)) {
    $trustedHosts = array();
    }
    // extract host from the request header

    [/code]
    and within createConfigFile, insert the following highlighted lines …
    [code language=”php” firstline=”503″ highlight=”511, 512, 513, 514, 515, 516, 517, 518, 519, 520, 521, 522, 523, 524, 525, 526, 527, 528, 529, 530, 531, 532, 533, 534, 535, 536, 537, 538, 539, 540, 541, 542, 543, 544″]

    $config->General[‘installation_in_progress’] = 1;

    $config->database = $dbInfos;
    if (!DbHelper::isDatabaseConnectionUTF8()) {
    $config->database[‘charset’] = ‘utf8’;
    }

    # Improved Security with IBM Bluemix
    # With SSL ALWAYS available for all Bluemix apps, let’s require all requests
    # to be made over SSL (https) so that data is NOT sent in the clear.
    # Non-ssl requests will trigger a
    # Error: Form security failed.
    # Please reload the form and check that your cookies are enabled
    # Reference: http://piwik.org/faq/how-to/faq_91/
    # Reference: https://developer.ibm.com/answers/questions/8312/how-do-i-enable-tlsssl-for-my-bluemix-application/
    $config->General[‘assume_secure_protocol’] = 1;
    $config->General[‘force_ssl’] = 1;

    # Setup proxy_client_headers to accurately detect GeoIPs of visiting clients
    $config->General[‘proxy_client_headers’] = array("HTTP_X_CLIENT_IP","HTTP_X_FORWARDED_FOR","HTTP_X_CLUSTER_CLIENT_IP","HTTP_CLIENT_IP");

    $config->General[‘proxy_host_headers’] = "HTTP_X_FORWARDED_HOST";

    # Let us have this Piwik deploy track itself to get some early data and success 🙂
    $config->Debug[‘enable_measure_piwik_usage_in_idsite’] = 1;

    # Let’s setup the config files trusted hosts entries to handle
    # 1…N amount of user-defined IBM Bluemix app routes
    if (isset($_ENV["APPURIS"])) {
    foreach ($_ENV["APPURIS"] as $application_uri) {
    $this->addTrustedHosts("https://" . $application_uri);
    }
    }

    # Emailing the easy way with IBM Bluemix + the SendGrid Service
    if (isset($_ENV["MAILHOST"])) {
    $config->mail[‘transport’]="smtp";
    $config->mail[‘port’]=587;
    $config->mail[‘type’]="Plain";
    $config->mail[‘host’]=$_ENV["MAILHOST"];
    }

    $config->forceSave();
    }

    [/code]
    We are changing the addTrustedHosts function to handle a potential array of Bluemix hosts (routes). Without this change, the function will not persist all possible bluemix hosts within your config.ini.php as *trusted*. This would result in a warning message when browsing to your Piwik deployment. We are changing the createConfigFile function to add a few additional key+value pairs into the config.ini.php file during installation setup to improve security (e.g. force https only), setup non-credential based SendGrid settings and adding of trusted hosts.

  10. Finally, to complete Phase 1, browse to the sub-folder /piwik/plugins/Installation/FormFirstWebsiteSetup.php. We’ll be modifying one function: init. Modify by replacing …
    [code language=”php” firstline=”62″ highlight=”66″]
    $this->addElement(‘submit’, ‘submit’, array(‘value’ => Piwik::translate(‘General_Next’) . ‘ »’, ‘class’ => ‘submit’));

    // default values
    $this->addDataSource(new HTML_QuickForm2_DataSource_Array(array(
    ‘url’ => $urlExample,
    )));
    [/code]
    with …
    [code language=”php” firstline=”62″ highlight=”66,67″]
    $this->addElement(‘submit’, ‘submit’, array(‘value’ => Piwik::translate(‘General_Next’) . ‘ »’, ‘class’ => ‘submit’));

    // default values
    $this->addDataSource(new HTML_QuickForm2_DataSource_Array(array(
    ‘siteName’ => $_ENV["APPURIS"][0],
    ‘url’ => "https://" . $_ENV["APPURIS"][0],
    )));
    [/code]
    We are changing the default first website setup during Piwik installation to point to itself. Earlier in Step 9, during the createConfigFile modification (~ line #523) … we defined a parameter to enable self monitoring. With this mod, our installer will now ensure that our Piwik installation will be defined with an id=1 and show data immediately after deployment – NICE.

    PiwikSelfMonitoringSetup

  11. You should see something like this after executing the push command and be greeted by the Piwik installer when browsing to your deployment for the first time.
    [code]$ cf push

    1 of 1 instances running

    App started

    Showing health and status for app yourpiwikapp in org your_org_name / space dev as your_bluemix_login…
    OK

    requested state: started
    instances: 1/1
    usage: 256M x 1 instances
    urls: yourpiwikapp.mybluemix.net

    state since cpu memory disk
    #0 running 2014-06-27 03:22:29 PM 0.0% 87.7M of 256M 74.8M of 1G[/code]

    Installer

  12. After completing the online Piwik installer wizard, it is recommended that you persist your generated configuration settings by downloading the config.ini.php file. The reason is simple. Due to the ephemeral filesystem nature of PaaS platforms – subsequent application uploads or restarts will lose the generated configuration data contained within the /config/config.ini.php file and the installation wizard will unnecessarily be reinitiated when accessing the restarted application. To persist the generated configuration file, execute the following command or browse to the Files and Logs left-side nav bar of your application within the Bluemix ACE UI and render a view of the /app/htdocs/config/config.ini.php file. Then click the download icon in the upper right hand side and save to the config folder of your local piwik application.
    [code]$ cf files &lt;name_of_your_piwik_app> /app/htdocs/config/config.ini.php > ./config/config.ini.php[/code]
    tip_iconIf you use the CLI to download, you will need to remove the first three (3) lines of the file which contains extraneous cf command-line stdout text using your favorite editor or command-line utility. For example,
    [code]$ sed -i -e ‘1,3d’ ./config/config.ini.php[/code]
    Finally, you will want to re-push the application one last time so that the config.ini.php file is present within the uploaded application file set … resulting in a Piwik Bluemix application with a persisted configuration
    [code]$ cf push[/code]

  13. Epic! That completes Phase 1.
    REMINDER: When accessing your new piwik site, be sure to use an SSL context https:// or you will encounter a form error.

Phase 2 – Adding SMS Reporting Support via Twilio

  1. Provision a Twilio Bluemix service and update your manifest.yml service’s section with the provisioned name. You will need to register and create a Twilio.com account. You will then select a phone number (incoming) to associate with your account. During Twilio Service provisioning, it is vital that you enter your account’s SID and Auth Token values into the Bluemix service provision creation dialog. See here for more details. If you forget to do this, you will have to delete the service instance and create a new one. The following image illustrates where your account’s SID and Token values can be found:
    twilio_sid_token
    Twilio is a cloud communications company based in San Francisco, California. Twilio allows software developers to programmatically make and receive phone calls and send and receive text messages using its web service APIs.
    tip_iconThis service cannot be created via the command-line interface (CLI). You must create this service using the ACE (Bluemix) User Interface. There, you will be required to register for a Twilio account and required to provide account details prior to service provisioning.
  2. Download a current stable version of the Twilio PHP Helper Library from https://github.com/twilio/twilio-php/archive/master.zip
  3. Untar or unzip the download and copy the entire twilio-php-master folder to the Piwik application sub-folder /piwik/libs/.
  4. Browse to the sub-folder /piwik/plugins/MobileMessaging/SMSProvider and create a new file named Twilio.php that contains the following content:
    [code language=”php”]
    <?php
    namespace PiwikPluginsMobileMessagingSMSProvider;

    /**
    * IBM Bluemix(tm) Twilio Piwik SMSProvider Plugin
    *
    * filename: Twilio.php
    * @link https://www.bluemix.net
    * @license Apache 2.0
    * @author Sanjay Joshi <joshisa (at) us (dot) ibm (dot) com>
    * @copyright IBM 2014
    */

    use PiwikPluginsMobileMessagingAPIException;
    use PiwikPluginsMobileMessagingSMSProvider;
    use Services_Twilio;

    require_once PIWIK_INCLUDE_PATH . "/libs/twilio-php-master/Services/Twilio.php";
    require_once PIWIK_INCLUDE_PATH . "/plugins/MobileMessaging/APIException.php";

    /**
    * Used to provide Twilio SMS Capability within Piwik
    *
    */
    class Twilio extends SMSProvider
    {

    // Read your Twilio AuthToken from www.twilio.com/user/account
    public function verifyCredential($apiKey)
    {
    $this->getCreditLeft($apiKey);
    return true;
    }

    /**
    * Send the SMS Text associated with this SMSProvider Library
    *
    * @throws Exception If the Twilio Library encounters an error
    * @param string $apiKey – Normalized Valid Twilio Incoming phone number in the form of "+{Country Code}{TwilioPhoneNumber}" (e.g. +19195551212, …)
    * @param string $smsText – Text payload
    * @param string $phoneNumber – Valid phone number for receipt of text payload
    * @param string $from – Valid Twilio Incoming phone number in the form of "+{Country Code}{TwilioPhoneNumber}" (e.g. +19195551212, …)
    * @return PiwikPluginsMobileMessagingSMSProvider
    */
    public function sendSMS($apiKey, $smsText, $phoneNumber, $from)
    {
    try {
    if (!isset($_ENV["SMSACCOUNT"]) || !isset($_ENV["SMSTOKEN"])) {
    throw new APIException("IBM Bluemix Twilio Environment Variable not set. Inspect bootstrap.php for environment variable parsing logic.");
    }

    $client = new Services_Twilio($_ENV["SMSACCOUNT"],$_ENV["SMSTOKEN"]);

    // Twilio forbids use of a $from number other than the Twilio Registered Number. It is not possible to customize the Sender ID for SMS messages sent from Twilio.
    // To learn more, visit this link: https://www.twilio.com/help/faq/sms/can-i-specify-the-phone-number-a-recipient-sees-when-getting-an-sms-from-my-twilio-app
    // Changing $from parameter to be validated Twilio Incoming Number provided by the user via the apiKey field.
    $from = $apiKey;

    $message = $client->account->messages->sendMessage(
    $from,
    $phoneNumber,
    $smsText);
    } catch (Services_Twilio_RestException $e) {
    throw new APIException(‘Twilio API returned the following error message: ‘ . $e->getMessage());
    }
    }

    /**
    * Function used to roughly complete account validation (e.g. Enough credits, valid incoming number provided, etc …)
    * Function name follows Clockwork SMS Provider class
    *
    * @throws Exception If the Twilio Library encounters an error
    * @param string $proposedFrom – Valid Twilio Incoming phone number in the form of "+{Country Code}{TwilioPhoneNumber}" (e.g. +19195551212, …)
    * @return boolean
    */
    public function getCreditLeft($apiKey)
    {
    try {
    $incomingTwilio = $this->normalizeTwilioNumber($apiKey);
    if ($this->validateTwilioNumber($incomingTwilio)) {
    //"Bluemix Twilio Service Account verified using Incoming #".$number->phone_number;
    return "Incoming Number (".$incomingTwilio.") was successfully validated. Happy Messaging!";
    } else {
    //"Bluemix Twilio Service Account verified using Incoming #".$number->phone_number;
    return "Unable to verify (".$incomingTwilio.") as defined by the bound Bluemix Twilio Service Account";
    }
    } catch (Exception $e) {
    throw new APIException(‘Error during Twilio account validation ‘ . $e->getMessage());
    return false;
    }
    }

    /**
    * Validate user-provided Twilio Phone Number for source of SMS
    *
    * @throws Exception If the Twilio Library encounters an error
    * @param string $proposedFrom – Valid Twilio Incoming phone number in the form of "+{Country Code}{TwilioPhoneNumber}" (e.g. +19195551212, …)
    * @return boolean
    */

    private function validateTwilioNumber($proposedFrom)
    {
    try {
    if (!isset($_ENV["SMSACCOUNT"]) || !isset($_ENV["SMSTOKEN"])) {
    throw new APIException("IBM Bluemix Twilio Environment Variable not set. Inspect bootstrap.php for environment variable parsing logic.");
    }
    $client = new Services_Twilio($_ENV["SMSACCOUNT"],$_ENV["SMSTOKEN"]);
    foreach ($client->account->incoming_phone_numbers as $number) {
    $cleanRegisteredIncomingNumber = $this->normalizeTwilioNumber($number->phone_number);
    if ($proposedFrom === $cleanRegisteredIncomingNumber) {
    //Only return true if the phone number has been matched
    return true;
    }
    }
    throw new APIException("Failed to validate user provided number as a valid Incoming Number within the bound IBM Bluemix Twilio Account. Did you forget to include the Country Code associated with this number in your Twilio account? Rather than the Twilio Friendly Name, please provide the full phone number with country code.");
    } catch (Services_Twilio_RestException $e) {
    throw new APIException(‘Twilio API returned the following error message: ‘ . $e->getMessage());
    }
    }

    /**
    * Validate user-provided Twilio Phone Number for source of SMS
    *
    * @throws Exception If the pre_match_all encounters an error
    * @param string $proposedFrom – User Provided Twilio Incoming phone number in unknown format
    * @return string in Twilio PHP Library expected form of "+{Country Code}{TwilioPhoneNumber}" (e.g. +19195551212, …)
    */

    private function normalizeTwilioNumber($proposedFrom)
    {
    try {
    if (preg_match(‘/^+d+$/’, $proposedFrom)) {
    return $proposedFrom;
    } else {
    // Credit to: http://stackoverflow.com/questions/4708248/formatting-phone-numbers-in-php
    $normalized = preg_replace(‘~[^d]*(d{1,4})[^d]*(d{3})[^d]*(d{3})[^d]*(d{4}).*~’, ‘+$1$2$3$4’, $proposedFrom);
    return $normalized;
    }
    } catch (Exception $e) {
    throw new APIException(‘Error during normalization of Twilio Number: ‘ . $e->getMessage());
    return $proposedFrom;
    }
    }
    }
    [/code]

  5. Modify the /piwik/plugins/MobileMessaging/SMSProvider.php file by inserting the following highlighted code:
    [code language=”php” firstline=”23″ highlight=”27,28,29,30,31,32,33,34,35,36,37″]
    const MAX_UCS2_CHARS_IN_ONE_UNIQUE_SMS = 70;
    const MAX_UCS2_CHARS_IN_ONE_CONCATENATED_SMS = 67;

    static public $availableSMSProviders = array(
    ‘Twilio’ => ‘You can use <a target="_blank" href="?module=Proxy&action=redirect&url=https://www.twilio.com/sms"><img src="https://www.twilio.com/packages/foundation/img/twilio-messaging.png"/></a> to send SMS Reports from Piwik.<br/>
    <ul>
    <li> First, <a target="_blank" href="?module=Proxy&action=redirect&url=https://www.twilio.com/sms"> Register for a Twilio Account and attach a <a href="?module=Proxy&action=redirect&url=https://www.bluemix.net">IBM Bluemix</a> Twilio service to your Piwik application (Signup is free!) To learn more about IBM Bluemix + Twilio, look <a href="?module=Proxy&action=redirect&url=https://www.twilio.com/blog/2014/02/twilio-and-ibm-codename-bluemix-nt.html">here</a>.
    </li><li> Enter your Twilio Account&apos;s Incoming Phone Number on this page. </li>
    </ul>
    <br/><em>About Twilio: </em><ul>
    <li>Lets you use standard web languages to build SMS and voice applications. We’re connected to carrier networks globally and expose them to you via a clean, powerful web API. So bring your favorite programming language, a web server, and build the next generation of communications with us.
    </li><li>Pay as you go model. Cost per SMS message is around ~0.08USD (0.06EUR).
    </li><li>Built in the cloud. API is always available, continuously upgraded and auto-scales to meet your needs. When you move your communications to the cloud, there are no tricky VPNs to configure or SMPP binds to manage. Just send us your message via HTTP, and we’ll deliver it anywhere in the world.
    </li>
    </ul>’,
    ‘Clockwork’ => ‘You can use <a target="_blank" href="?module=Proxy&action=redirect&url=http://www.clockworksms.com/platforms/piwik/"><img src="plugins/MobileMessaging/images/Clockwork.png"/></a> to send SMS Reports from Piwik.<br/>
    <ul>
    <li> First, <a target="_blank" href="?module=Proxy&action=redirect&url=http://www.clockworksms.com/platforms/piwik/">get an API Key from Clockwork</a> (Signup is free!)
    </li><li> Enter your Clockwork API Key on this page. </li>
    </ul>
    <br/><em>About Clockwork: </em><ul>
    [/code]

  6. Update the deployed Bluemix application with your Phase 2 updated files which now include the Twilio PHP Helper Library and a new SMS Provider.
    [code]$ cf push[/code]
  7. Login to Piwik and click on the Settings menu link found in the upper right hand corner of the screen.
  8. Click on Mobile Messaging in the left-hand nav bar. If everything went well, you should be greeted with the Twilio SMS provider info as shown below:
    PiwikTwilioSMS
    Take special note of the Piwik SMS Provider API Key field.
  9. Look-up your Twilio Incoming Number found on your Twilio Account screen as shown below:

    and enter that Twilio number into the Piwik SMS API Key field
  10. After clicking Save, Piwik will confirm/validate the API Key number provided and present you with an opportunity to register target phone numbers for receipt of Piwik SMS reports as shown below:
    PiwikTwilioSMSConfirmed
  11. After adding a target phone number, the target will be SMS texted a validation code which is subsequently required within the Piwik form to complete the number registration process. Remember, if you are using a Twilio Trial account – your target phone number must already be listed as a verified number within the account. Refer to the SMS section located here for more information.
  12. Congratulations! Using the Reports menu link at the top, you can now create and schedule Piwik SMS (and Email) reports leveraging Bluemix Twilio and SendGrid services.

FAQ

Q: I keep encountering the error below when I attempt to login to my Piwik Bluemix app? What gives?

Piwik SSL Login Error

Piwik SSL Login Error


A: This is a SSL Context error. Admittedly, the error message within Piwik could be better. Make sure that you are browsing to your site using https:// rather than http://.

Q: I notice a file integrity error when invoking the system check as seen below. Why?
PiwikFileIntegrityError
A: This is a normal side-effect of completing the modifications outlined in this article. Also, .gitignore files are ignored by the cloud foundry CLI during application uploads. It is safe to ignore this warning.

Q: I’m using the ClearDB MySQL service with Piwik. After following the instructions, I’m intermittently seeing errors within my dashboard widgets about trouble with the config file and red text prompting me to re-install Piwik. What’s going on?
A: This is due to the limited amount of connections (4) available with the “free” tier plan of ClearDB on Bluemix. When larger plans (and connection limits) are offered by ClearDB on Bluemix, you can opt to change the MySQL ClearDB plan used by your provisioned service to lessen the chances of seeing these errors on your dashboard. They are intermittent and can show up randomly within different widgets. As a temporary alternative, you can also close many of the widgets and keep a minimal amount open. Refreshing the page can also help. For more information, see https://github.com/piwik/piwik/issues/2574.

Q: I am not seeing any data after my first login. I’ve completed the deployment and installation of Piwik. I’ve clicked on different links within the Piwik dashboard. I feel betrayed. Where’s the data?
A: Apologies. The article’s intent is not to cause anxiousness. Did you leave the default entries for the first website – during setup? Your piwik deployment must be associated with the site id of 1. Or perhaps you have the “Do not track” option enabled on your browser. Piwik respects user privacy and honors that setting by default. However, if you want to feel that early success of data coming in (who doesn’t?) – you can feel free to disable that setting and ignore the “Do not track” option (temporarily one would presume :-)). Later, if you’d like to remove this self-tracking, simply delete the website entry in Piwik and remove the key:value pair [enable_measure_piwik_usage_in_idsite = 1] from your /piwik/config/config.ini.php file. Then, re-deploy your Piwik application.
PiwikDoNotTrack

Q: During Phase 2, I’m encountering the error …
The requested resource /2010-04-01/Accounts/{AccountSID}/IncomingPhoneNumbers.json was not found. Help!

twilio_missing_sid_token_error
A: This results from provisioning a Twilio Bluemix service that is either/or missing, incorrect or incompatible AccountSID or Auth Token values associated with the provided Incoming Number (APIKey). Either provide a different incoming number (APIKey) value or delete and reprovision the Twilio service with the correct SID/Token values.

Reference Articles on Piwik

Piwik Tracking for this blog post over the 2014 year timeframe
Published by pushing Up, Up, Down, Down, Left, Right, Left, Right, B, A, Select, Start


More stories
December 10, 2018

An Introduction to PostgreSQL’s hstore

One of the highlights of PostgreSQL is its versatility, especially when you need a flexible data model. In this post, we'll explore hstore, PostgreSQL's simple key-value store, and take a look at how it works using a catalog of books.

Continue reading

December 7, 2018

Highly Available Applications with IBM Cloud Foundry

To properly deploy an application in a cloud environment and ensure maximum responsiveness, your app needs to be deployed in a certain (and easy) way that maximizes the chance of an instance always being ready to respond to a user request. This article will explain how to deploy your Cloud Foundry applications in the IBM Cloud such that you reach your target application availability.

Continue reading

December 5, 2018

Cloud Foundry Container-to-Container Networking

If you're like many developers who are deploying applications to Cloud Foundry, you probably don't think about networking too often. After all, as a PaaS, Cloud Foundry takes care of all the routing and connectivity for you. There is one feature, however, you might consider before writing your next app: container-to-container networking.

Continue reading