September 9, 2019 By Kevin Oliver 4 min read

Artificial intelligence and machine learning (ML) have become very popular recently due to their ability to both optimize processes and provide the deep insights that push enterprises and industries forward. In fact, 68 percent of respondents in a recent 451 Research Report, Accelerating AI with Data Management; Accelerating Data Management with AI noted that they are either using machine learning or plan to within the next three years.

What’s more impressive is that 92 percent of those currently using machine learning – including companies that have it in proof of concept – “have positive opinions about the performance of their machine learning projects.” Yet, implementation challenges remain. Data management technologies that are themselves infused with AI can overcome these challenges and help enterprises get the most out of their plans for AI and ML.

Implementation Challenges

Three key barriers stood out in 451 Research’s report: a lack of skilled resources, accessing and preparing data and limited budget.

The lack of skilled resources was the top concern noted by respondents with 40 percent listing it as a barrier and 21 percent saying it was the most significant barrier.

That’s no surprise, given that there are only one to two million data scientists, compared to five to ten million business intelligence power users and two hundred to two hundred fifty knowledge workers. Simply put: there aren’t enough data scientists to go around for every company, or even to serve the needs of knowledge workers within individual companies.

Problems with accessing and preparing data, noted by 33 percent of respondents, can stop AI and ML in its tracks. A large body of data is required to deliver the depth of insight expected with these rising technologies. Lacking that data may reduce the accuracy of insight as well, due to missing crucial variables. The limited budgets mentioned by 32 percent of respondents can similarly act as a blocker to AI and ML projects from the start. Such projects will receive intense scrutiny and be able to show their worth from both a cost and value perspective up front. Their champions will need to show how they can increase efficiencies as well.

The role of data management

Data management is capable of helping with all of these challenges, particularly where data ingestion and preparation is concerned—a stage of the AI process 39 percent said they consider to be “the most demanding in relation to their underlying infrastructure.” Data management solutions should be infused with AI and part of a robust data management environment.

More than 66 percent of respondents agreed that “AI and ML are important components of data platforms and analytics initiatives.” That number increases to 88 percent among companies where “nearly all strategic decisions are data-driven.” In particular, ML can help increase efficiencies by optimizing the path a query takes to data. ML also can enable confidence-based querying where answers are returned in order of predicted accuracy, which is determined by looking at historical accuracy data.

The right data management environment is crucial. By choosing an environment with AI built in instead of stand-alone AI solutions, you can control management in a single system. Environments with integration and data virtualization capabilities also enable data to be queried where it resides, reducing the need for costly and time-consuming data movement.

How to overcome the AI challenges

A data management environment with the interconnected, AI-infused nature described above helps mitigate the challenges identified previously. The lack of skilled resources is offset in two ways. Data scientists and database administrators (DBAs) are given back valuable time through the inclusion of development languages and frameworks and automation of routine tasks like query optimization. Both result from building AI directly into data management solutions. Greater connectivity between data repositories in the environment can also help enable knowledge workers to pull some of their own data using intuitive data exploration tools, reducing the number of requests data scientists and DBAs must take on.

The challenges related to data access and preparation are directly affected by choosing the right data management environment. The connectivity created by data virtualization enables access across multiple repositories with different data types without data movement concerns. This should not only make data access and preparation easier than it would be in ecosystems where a reduced subset of siloed data is migrated to be used as part of AI and ML projects. It will also help increase accuracy by involving more data.

Finally, while no data management solution can solve up front budget limitations, it can demonstrate efficiencies and ROI that make investment more palatable. The 92 percent satisfaction rate is a good indicator that businesses are seeing the return they expect. The greater efficiencies which allow data scientists and DBAs to focus on value-additive tasks should also be encouraging. And, though hard to predict, the deeper, more accurate insights AI-infused data management can produce have the potential to open up new revenue-generating opportunities.

The Db2 family of data management solutions is focused on bringing AI-infused offerings capable of offsetting the common challenges described above. Learn more about data management and AI or Db2 11.5, the AI database, by reading:

More from Artificial intelligence

IBM named a Leader in the 2023 Gartner® Magic Quadrant™ for Data Integration Tools

4 min read - IBM’s data integration tools are a core piece of IBM’s Data Fabric, providing customers with a secure data foundation to accelerate and scale AI implementations. Forward-thinking businesses see the value that multi-cloud adoption offers. The only question is: How do you ensure effective ways of breaking down data silos and bringing data together for self-service access? This is especially integral in today’s AI-driven market, where businesses are continuously feeding and training their ML models on large data foundations. To confidently…

Now generally available, watsonx.governance helps businesses build trust in their generative AI

4 min read - Before AI can help your business reach new levels of productivity, you need to be able to trust what it’s doing. While generative AI has the potential to unlock tremendous productivity and economic value, it comes with new complexities and increased risks not previously seen with predictive machine learning (ML). This ranges from the origin of underlying training data to the potential of AI to perpetuate bias to a lack of explainable outputs. Businesses must establish guardrails to manage these…

A look into IBM’s AI ethics governance framework

3 min read - “Organizations are responsible for ensuring that AI projects they develop, deploy or use do not have negative ethical consequences,” as per Gartner. Yet while 79% of executives say AI ethics is important to their enterprise-wide AI approach, less than 25% have operationalized ethics governance principles. In a new case study featuring IBM, Gartner talks about how to establish a governance framework to streamline the process of detecting and managing technology ethics concerns in AI projects. Addressing the need for an…

How generative AI delivers value to insurance companies and their customers

4 min read - Insurers struggle to manage profitability while trying to grow their businesses and retain clients. They must comply with an increasing regulatory burden, and they compete with a broad range of financial services companies that offer investment products that have potential for better returns than traditional life insurance and annuity products. Although interest rates have increased at an unprecedented rate over the past year as central banks attempt to curb inflation, a significant part of insurers’ reserves are locked into low-yield…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters