Data Science

Lifelong (machine) learning: how automation can help your models get smarter over time

Share this post:

Machine learning should happen constantly

Imagine you’re interviewing a new job applicant who graduated top of their class and has a stellar résumé. They know everything there is to know about the job, and has the skills that your business needs. There’s just one catch: from the moment they join your team, they’ve vowed never to learn anything new again. You probably wouldn’t make that hire, because you know that lifeMachine Learning Brainlong learning is vital if someone is going to add long-term value to your team. Yet when we turn to the field of machine learning, we see companies making a similar mistake all the time. Data scientists work hard to develop, train and test new machine learning models and neural networks. However, once the models get deployed, they don’t learn anything new. After a few weeks or months, become static and stale, and their usefulness as a predictive tool deteriorates.

Why models stop learning

Data scientists are well aware of this problem, and would love to find a way to enable their models to participate in the equivalent of lifelong learning. However, moving a model into production is typically a tough task, and deployment requires help from busy IT specialists. When a single deployment can take weeks, it’s no wonder that most data scientists prefer to hand over their latest model and move onto the next project, rather than persist with the drudgery of continually retraining and redeploying their existing models.

Deployment isn’t just painful for data scientists—it can be a headache for IT teams too. Data scientists might have used any one of a wide variety of languages, frameworks and tools to build their models, and there is no guarantee that those choices will make the model easy to integrate into production systems. In a worst-case scenario, the model may need to be substantially refactored or even rebuilt from scratch before it can be deployed. As a result, if data scientists ask for their models to be redeployed too frequently, they may be met with significant resistance from the IT department.

Streamlining deployment to keep models in training

The good news is that model deployment isn’t inherently labor-intensive. Just as in other forms of software development, the principles of DevOps apply here. With the right platform, it is possible to create seamless continuous deployment pipelines that automate many aspects of the process, transforming deployment from weeks of manual effort to a matter of a few mouse-clicks.
For example, with IBM® Watson® Machine Learning integrated in IBM Data Science Experience, data scientists can develop models using a wide range of languages (including Python, R and Scala) and frameworks (such as SparkML, Scikit-Learn, xgboost and SPSS). The solution will abstract the models into a standardized API that can be integrated easily with production systems. This gives data scientists the flexibility they need to choose best-of-breed tools and techniques during development, without increasing the complexity of deployment for the IT team.

Watson Machine Learning aims to combine other elements of IBM Watson Data Platform to provide a continuous feedback loop. When your model is ready to move into production, you can specify how frequently you would like to retrain it, and automate the redeployment process. You can also monitor and validate the results of the retrained model to ensure that the new version is an improvement—and with integrated version control, you can easily roll back to the previous release if necessary.

Giving data scientists more power

Lead Offering Manager, IBM DSX & WML

Angshuman Roy

Global Product Marketing Manager, IBM DSX & WML

More Data Science stories
April 29, 2019

Transforming Customer Experiences with AI Services (Part 1)

This is an experience from a recent customer engagement on transcribing customer conversations using IBM Watson AI services.

Continue reading

April 26, 2019

Updated Tutorial: Database-Driven Chatbot

The tutorial on how to build a database-driven chatbot has been updated. It's now simpler to deploy and and offers more options—Slack, Facebook Messenger, Wordpress, and more.

Continue reading

April 23, 2019

Announcing the Deprecation of the Watson Machine Learning JSON Token Authentication Service

We’d like to inform you about the deprecation of the Watson Machine Learning JSON Token Authentication service. This method of authentication will be retired on May 30, 2019.

Continue reading