Being question led in big data projects is the first critical success factor which I put forward in my original blog post on this topic. In this article I describe three aspects of this factor in more detail.
Be question led. A single business sponsor for a big data project is likely to articulate a business problem clearly. Such a problem will have the attention of the organisation because it is a source of pain or it offers new opportunity. The problem can typically be expressed as a question and the business will place value on a solution which can give answers. However such an answer is likely to prompt a new question. By using big data technology which, for example, is not constrained to a pre-defined schema and structuring the project to accommodate exploration, analysts are able to gain insights to ask questions which weren't conceived at the outset. The results may lead to greater efficiency or opportunity not originally envisaged.
Learn from the process. A big data project is not predictable in the way that many transactional system implementations are today. The first question posed in an attempt to solve the business problem may not solicit a useful response. There may be problems not being able to access the right type of data, poor data quality, insufficient volume or the wrong analytic, for example. Exploration by analysts may take longer than planned, or turn out to be too complex for their expertise and intuition. Or even the approach taken to tackle the problem is unsuitable. What is important is the recognition that failure to reach a desired outcome will lead to discovery which informs an alternative path. For example, other problems may be identified which need addressing, such as with the data, or incremental insight prompts a different analysis approach.
Deploy sufficient infrastructure. Sizing infrastructure – servers, storage and network – which is optimised to answer the first question is unlikely to lead to a successful outcome. The next question might demand more computational power or working storage, or indeed need more data. It is necessary to have a platform which has the agility not to inhibit exploration, and it must be able to scale to support the execution of different styles of analytics. Looking at this another way, a platform is needed which can cope with the next questions prompted by the answer to the current question which produces more than expected or even the unexpected. Affordability of the platform will be a concern and acquisition cost is one factor. However, according to IBM's paper on, “The future of analytics infrastructure,” published in May, flexibility and the cost of change is another.
In my next post I shall consider the second of my three critical success factors to realising business value from big data and analytics: Be able to act on insight.