Data Science

Lifelong (machine) learning: how automation can help your models get smarter over time

Share this post:

Machine learning should happen constantly

Imagine you’re interviewing a new job applicant who graduated top of their class and has a stellar résumé. They know everything there is to know about the job, and has the skills that your business needs. There’s just one catch: from the moment they join your team, they’ve vowed never to learn anything new again. You probably wouldn’t make that hire, because you know that lifeMachine Learning Brainlong learning is vital if someone is going to add long-term value to your team. Yet when we turn to the field of machine learning, we see companies making a similar mistake all the time. Data scientists work hard to develop, train and test new machine learning models and neural networks. However, once the models get deployed, they don’t learn anything new. After a few weeks or months, become static and stale, and their usefulness as a predictive tool deteriorates.

Why models stop learning

Data scientists are well aware of this problem, and would love to find a way to enable their models to participate in the equivalent of lifelong learning. However, moving a model into production is typically a tough task, and deployment requires help from busy IT specialists. When a single deployment can take weeks, it’s no wonder that most data scientists prefer to hand over their latest model and move onto the next project, rather than persist with the drudgery of continually retraining and redeploying their existing models.

Deployment isn’t just painful for data scientists—it can be a headache for IT teams too. Data scientists might have used any one of a wide variety of languages, frameworks and tools to build their models, and there is no guarantee that those choices will make the model easy to integrate into production systems. In a worst-case scenario, the model may need to be substantially refactored or even rebuilt from scratch before it can be deployed. As a result, if data scientists ask for their models to be redeployed too frequently, they may be met with significant resistance from the IT department.

Streamlining deployment to keep models in training

The good news is that model deployment isn’t inherently labor-intensive. Just as in other forms of software development, the principles of DevOps apply here. With the right platform, it is possible to create seamless continuous deployment pipelines that automate many aspects of the process, transforming deployment from weeks of manual effort to a matter of a few mouse-clicks.
For example, with IBM® Watson® Machine Learning integrated in IBM Data Science Experience, data scientists can develop models using a wide range of languages (including Python, R and Scala) and frameworks (such as SparkML, Scikit-Learn, xgboost and SPSS). The solution will abstract the models into a standardized API that can be integrated easily with production systems. This gives data scientists the flexibility they need to choose best-of-breed tools and techniques during development, without increasing the complexity of deployment for the IT team.

Watson Machine Learning aims to combine other elements of IBM Watson Data Platform to provide a continuous feedback loop. When your model is ready to move into production, you can specify how frequently you would like to retrain it, and automate the redeployment process. You can also monitor and validate the results of the retrained model to ensure that the new version is an improvement—and with integrated version control, you can easily roll back to the previous release if necessary.

Giving data scientists more power

Lead Offering Manager, IBM DSX & WML

Angshuman Roy

Global Product Marketing Manager, IBM DSX & WML

Add Comment
No Comments

Leave a Reply

Your email address will not be published.Required fields are marked *

More Watson stories

Anomaly detection in mobile sensor data using Machine Learning

This blog post is an excerpt from our solution tutorial - "Gather, visualize, and analyze IoT data". The tutorial walks you through setting up an IoT device, gathering mobile sensor data in the Watson IoT Platform, exploring data and creating visualizations and then using advanced machine learning services to analyze data and detect anomalies in the historical data.

Continue reading

Taming your neural networks: how controlled experimentation can help you build better machine learning models

Businesses today are eager to harness machine learning and deep learning for competitive advantage—yet few businesspeople realize that building a machine learning model or neural network is a marathon, not a sprint.

Continue reading

Getting closer to the source: the power of self-service data preparation

IBM Data Refinery, a feature of Watson Data Platform, helps reduce reliance on IT and give knowledge workers faster access to high-quality data

Continue reading