05 August 2014
PDF (350 KB)
Share:
Peter Parente

Peter Parente

STSM, SWG Emerging Technology

@parente on Twitter

Sign up for IBM Bluemix™
This cloud platform is stocked with free services, runtimes, and infrastructure to help you quickly build and deploy your next mobile or web application.

IPython Notebook is a web-based environment for interactive computing and reproducible research. You can write code in a notebook in your web browser, run it with the click of a button, and see its output right in your browser.

The tight feedback loop between coding and exploration — and the ability to include Markdown text, web media, and interactive widgets right in a notebook — makes IPython a powerful environment for playing with data and new ideas:

IPython Notebook screenshot

Bluemix makes it easy to deploy an instance of IPython Notebook for off-the-cuff use. This article covers the steps you need to get a notebook server up and running along with a few popular Python scientific computing packages: numpy, Pandas, and matplotlib.

In addition, you'll set a password for the notebook server as a basic means of security. Finally, I'll cover some ideas of what you can (and cannot) do with your notebook environment.

Note: The sample server is configured with a blank password and can be accessed by all users. Don't use it for real work!

Bluemix makes it easy to deploy an instance of IPython Notebook for off-the-cuff use.

READ: Installing the cf command line interface

WATCH: IPython videos and screencasts

What you'll need

 

Step 1. Get the code

 

Clone the source code of the parente | ipython-on-bluemix project on IBM DevOps Services:

git clone https://hub.jazz.net/git/parente/ipython-on-bluemix

If you plan to play with the recipe later (for example, add additional scientific computing libraries to the notebook environment), you might consider getting a DevOps Services account and forking the repository instead. Then clone your fork with the command git clone https://hub.jazz.net/git/your DevOps Services username/ipython-on-bluemix.

Step 2. Review the code

 

You don't need to understand the code to run it. If you just want to get the notebook running, feel free to skip this step.

If you peek at the code, you'll notice there's not much to it. The manifest.yml file contains instructions to Bluemix about how to deploy your IPython Notebook instance. It states the application name, amount of memory to allocate, host name to set, and so on:

---
applications:
- name: ipython
  memory: 1G
  instances: 1
  host: ipython-${random-word}
  domain: mybluemix.net
  path: .
  buildpack: https://github.com/joshuamckenty/heroku-buildpack-python

The requirements.txt file contains the list of libraries the buildpack will install. You can add or remove libraries here:

ipython[notebook]
numpy
pandas
matplotlib

Finally, Procfile instructs Bluemix on how to run your application once it's built:

web: ipython profile create; echo "c.NotebookApp.password = '$( python -c 'from 
IPython.lib import passwd; import os; print passwd(os.environ.get("PASSWORD", 
""))' )'" >> ~/.ipython/profile_default/ipython_notebook_config.py; 
ipython notebook --port=$VCAP_APP_PORT --ip=$VCAP_APP_HOST --no-mathjax

This is the most complicated piece of the solution. (It's also the most messy, because it must be a single line. It's shown here with line breaks for readability only.) Breaking it down, the code:

  1. Creates an IPython user profile.
  2. Generates a secure hash of the PASSWORD environment variable and inserts it into the profile.
  3. Starts the notebook server using the port and host information provided by Bluemix.

Step 3. Push the app bundle to Bluemix

 

Use the cf command-line tool to target Bluemix, log in, and push your notebook bundle.

In a local command shell, run the following commands, responding to any prompts as you go:

cd ipython-on-bluemix
cf api https://api.ng.bluemix.net
cf login
cf push --no-start

Step 4. Set a password

 

With Bluemix now aware of your app, you can set a password for it with the following command on a *nix or OS X operating system. The subcommand prompts you to enter a password and then echo it for the cf command to use:

cf set-env ipython PASSWORD $( echo -n "Password: " && read -s PASSWORD && echo $PASSWORD )

If your shell doesn't support read (for example, on Windows) or you're unconcerned about the password appearing in your console input history, you can state it directly instead. Here, my password would be mysecretpassword:

cf set-env ipython PASSWORD mysecretpassword

Note: The cf command might suggest that you use cf push to ensure that your environment variable changes take effect. You do not need to push at this time because you have not yet started the app.

Step 5. Deploy the app

 

With your application environment now pushed and configured, you can deploy the app with the following command:

cf start ipython

This command might run for some time as Bluemix builds and installs the dependencies listed in the requirements.txt file. You can monitor its progress by running the following command:

cf logs --recent ipython

Sometimes, in fact, the buildpack takes so long that Bluemix notes it as a staging error even though it completes successfully. If you receive such an error in the console, run the cf logs command a few times to check if progress is being made in staging the app. If you see progress, let processing continue until you see the following line in the log:

Successfully installed ipython numpy pandas matplotlib tornado pyzmq jinja2 
python-dateutil pytz pyparsing nose backports.ssl-match-hostname markupsafe six
       Cleaning up...

Then run the following commands to get the app running properly:

cf stop ipython
cf start ipython

You can confirm that the notebook server is running properly by querying its status:

cf app ipython

Inspect the output to ensure that the application instance is in the running state:

Showing health and status for app ipython in org pparent@us.ibm.com / space dev as pparent@us.ibm.com...
OK
requested state: started
instances: 1/1
usage: 1G x 1 instances
urls: ipython-unspecious-subconsul.mybluemix.net
     state     since                    cpu    memory         disk
#0   running   2014-05-28 09:10:09 PM   0.4%   149.1M of 1G   282.5M of 1G

Step 6. Use the IPython Notebook

 

Now your IPython Notebook instance is up and running. Visit it in your browser at the URL mentioned in the output of the last command you ran in Step 5. For instance, based on the example output at the end of Step 5, you would visit https://ipython-unspecious-subconsul.mybluemix.net.

When prompted by the notebook server, enter the password that you set in Step 4. If you skipped that step, the password is blank, so just press Enter:

Screenshot of the IPython Notebook login page

On the dashboard page, click New notebook to open a blank notebook in a new browser tab:

Screenshot of a new IPython Notebook

In your new notebook, enter %matplotlib inline into the first cell and click the Run button in the toolbar.

Do the same for the following three blocks of code, putting each one in its own cell and running that cell:

import matplotlib.pyplot as plt
import numpy as np

x = np.arange(0, 4*np.pi, 0.05)
y = [np.sin(i) for i in x]

plt.plot(x, y)

In the end, you should see a simple sine wave plot:

Screenshot of sine wave plot in a notebook

Go further

 

The simple plot at the end of Step 6 barely scratches the surface of what you can do with your IPython Notebook. Here are some ideas for further exploration:

  • Visit the nbviewer site or search for public notebooks to learn more about what you can do in your new environment. You can download any notebook you find and then upload it to the dashboard of your notebook server to play with the notebook.
  • Install some additional packages, such as scikit-learn (machine learning), nltk (natural language processing), and PyMC (Bayesian statistical modeling). You can add them to your requirements.txt and redeploy. Alternatively, you can install them by shelling to bash right in the notebook interface with a code cell such as !pip install scikit-learn— but see "Limitations and alternatives" for more information about this approach.
  • Connect Bluemix services to your notebook instance to interactively explore their APIs via a notebook. See the Bluemix documentation about how to add services to your application and access their respective APIs.
  • Check out the Help menu in your notebook for keyboard shortcuts to improve your proficiency.

Limitations and alternatives

 

In this article, you deployed an instance of IPython Notebook to Bluemix. Although the process is relatively simple, the result does have its limitations.

At the moment, renaming a notebook fails to work, because the HTTP method used by the IPython Notebook server is not handled by the Bluemix router.

Bluemix is optimized for hosting stateless, I/O-heavy web servers and expects an application to behave as such. For example, Bluemix caps the application memory space at the limit specified in the manifest.yml file. If you perform a computation in a notebook that consumes more RAM than you allotted, Bluemix might stop and restart the server. Bluemix might do the same if your average CPU usage rises above a provider-configured threshold — say if you perform a CPU-intensive computation. Either can occur if you try installing additional packages by using pip as noted in the "Go further" section.

Finally, the disk space allocated to your notebook server is ephemeral and will not persist long-term.

If you plan to use IPython Notebook in earnest, you can overcome these issues by deploying a long-lived instance to IBM SoftLayer. Even better, you can do so using a Docker container image, which removes the need to build and install all the necessary dependencies. Stay tuned for a future article on exactly this very topic.


RELATED TOPICS:IPythonScientific computing

Add a comment

Note: HTML elements are not supported within comments.


1000 characters left

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Cloud computing
ArticleID=979745
ArticleTitle=Run IPython Notebook on IBM Bluemix
publish-date=08052014