September 9, 2019 By Kevin Oliver 4 min read

Artificial intelligence and machine learning (ML) have become very popular recently due to their ability to both optimize processes and provide the deep insights that push enterprises and industries forward. In fact, 68 percent of respondents in a recent 451 Research Report, Accelerating AI with Data Management; Accelerating Data Management with AI noted that they are either using machine learning or plan to within the next three years.

What’s more impressive is that 92 percent of those currently using machine learning – including companies that have it in proof of concept – “have positive opinions about the performance of their machine learning projects.” Yet, implementation challenges remain. Data management technologies that are themselves infused with AI can overcome these challenges and help enterprises get the most out of their plans for AI and ML.

Implementation Challenges

Three key barriers stood out in 451 Research’s report: a lack of skilled resources, accessing and preparing data and limited budget.

The lack of skilled resources was the top concern noted by respondents with 40 percent listing it as a barrier and 21 percent saying it was the most significant barrier.

That’s no surprise, given that there are only one to two million data scientists, compared to five to ten million business intelligence power users and two hundred to two hundred fifty knowledge workers. Simply put: there aren’t enough data scientists to go around for every company, or even to serve the needs of knowledge workers within individual companies.

Problems with accessing and preparing data, noted by 33 percent of respondents, can stop AI and ML in its tracks. A large body of data is required to deliver the depth of insight expected with these rising technologies. Lacking that data may reduce the accuracy of insight as well, due to missing crucial variables. The limited budgets mentioned by 32 percent of respondents can similarly act as a blocker to AI and ML projects from the start. Such projects will receive intense scrutiny and be able to show their worth from both a cost and value perspective up front. Their champions will need to show how they can increase efficiencies as well.

The role of data management

Data management is capable of helping with all of these challenges, particularly where data ingestion and preparation is concerned—a stage of the AI process 39 percent said they consider to be “the most demanding in relation to their underlying infrastructure.” Data management solutions should be infused with AI and part of a robust data management environment.

More than 66 percent of respondents agreed that “AI and ML are important components of data platforms and analytics initiatives.” That number increases to 88 percent among companies where “nearly all strategic decisions are data-driven.” In particular, ML can help increase efficiencies by optimizing the path a query takes to data. ML also can enable confidence-based querying where answers are returned in order of predicted accuracy, which is determined by looking at historical accuracy data.

The right data management environment is crucial. By choosing an environment with AI built in instead of stand-alone AI solutions, you can control management in a single system. Environments with integration and data virtualization capabilities also enable data to be queried where it resides, reducing the need for costly and time-consuming data movement.

How to overcome the AI challenges

A data management environment with the interconnected, AI-infused nature described above helps mitigate the challenges identified previously. The lack of skilled resources is offset in two ways. Data scientists and database administrators (DBAs) are given back valuable time through the inclusion of development languages and frameworks and automation of routine tasks like query optimization. Both result from building AI directly into data management solutions. Greater connectivity between data repositories in the environment can also help enable knowledge workers to pull some of their own data using intuitive data exploration tools, reducing the number of requests data scientists and DBAs must take on.

The challenges related to data access and preparation are directly affected by choosing the right data management environment. The connectivity created by data virtualization enables access across multiple repositories with different data types without data movement concerns. This should not only make data access and preparation easier than it would be in ecosystems where a reduced subset of siloed data is migrated to be used as part of AI and ML projects. It will also help increase accuracy by involving more data.

Finally, while no data management solution can solve up front budget limitations, it can demonstrate efficiencies and ROI that make investment more palatable. The 92 percent satisfaction rate is a good indicator that businesses are seeing the return they expect. The greater efficiencies which allow data scientists and DBAs to focus on value-additive tasks should also be encouraging. And, though hard to predict, the deeper, more accurate insights AI-infused data management can produce have the potential to open up new revenue-generating opportunities.

The Db2 family of data management solutions is focused on bringing AI-infused offerings capable of offsetting the common challenges described above. Learn more about data management and AI or Db2 11.5, the AI database, by reading:

Was this article helpful?

More from Artificial intelligence

In preview now: IBM watsonx BI Assistant is your AI-powered business analyst and advisor

3 min read - The business intelligence (BI) software market is projected to surge to USD 27.9 billion by 2027, yet only 30% of employees use these tools for decision-making. This gap between investment and usage highlights a significant missed opportunity. The primary hurdle in adopting BI tools is their complexity. Traditional BI tools, while powerful, are often too complex and slow for effective decision-making. Business decision-makers need insights tailored to their specific business contexts, not complex dashboards that are difficult to navigate. Organizations…

Introducing the watsonx platform on Microsoft Azure

4 min read - Artificial intelligence (AI) is revolutionizing industries by enabling advanced analytics, automation, and personalized experiences. According to The business value of AI, from the IBM Institute of Business Value, AI adoption has more than doubled since 2017. Enterprises are taking an intentional design approach to hybrid cloud and AI to drive technology decisions and enable adoption of Generative AI. According to the McKinsey report,  The economic potential of generative AI: The next productivity frontier, generative AI is projected to add $2.6…

Democratizing Large Language Model development with InstructLab support in

5 min read - There is no doubt that generative AI is changing the game for many industries around the world due to its ability to automate and enhance creative and analytical processes. According to McKinsey, generative AI has a potential to add $4 trillion to the global economy. With the advent of generative AI and, more specifically, Large Language Models (LLMs), driving tremendous opportunities and efficiencies, we’re finding that the path to success for organizations to effectively use and scale their generative AI…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters