Generations are notorious for discounting the value of what can be learned from those that precede or follow them. But as technology that grew up serving often youthful consumer demands � social, mobile, gaming � continues to reshape the enterprise, how effectively we transfer and combine skills across generations will determine the pace of business innovation and economic growth moving forward.
It was in this context on December 13 that San Jose State University (SJSU) and IBM hosted an event bringing together students, faculty, and business professionals to discuss the skills needed to move business into the future. The event served as a culmination of The Great Mind Challenge (TGMC), a competition during which SJSU student teams under the direction of Professor Larry Gee applied social business technology and concepts to real-world situations encountered by IBM Business Partner GBS. The span of generations in the room for the evening�s program was on display when IBMer Fabian Divito put up a chart illustrating the five eras of enterprise computing, laid out from left to right. To laughter from the audience, he quipped, �Don�t worry too much about the left half of the chart. Just know it happened.�
Don Edwards, a technology veteran who currently serves as head of IS for Alameda County Social Services noted, �I got started just to the left of where this entire chart picks up.�
So it made for a lively panel of diverse perspectives when Don joined tech industry experts Remy Malan from SugarCRM and Nanci Knight from IBM, along with Diane Pham, who recently graduated from San Jose State and took a job with the United Way, and Brian Orlando, a student member of the winning TGMC team. The students in the room were eager to learn from the experts, but the experts were just as eager to learn from the students. All were convinced that today�s graduate entering the workforce thinks and solves problems in ways fundamentally different from prior generations. With ubiquitous mobile and social technology, there is an inclination to share knowledge with and seek input from a wide open audience that comes naturally to today�s graduates. While those most adept at using social technology come from all generations, the message was clear: do not underestimate this difference across generations.
Diane�s experience joining the United Way illustrated the point. Little things that come naturally to her � which social media channel to use when, for instance � take more time with other workers in the organization. But business leaders are finding ways to harness social technology and the natural enthusiasm and creativity of today�s graduates to drive significant business value. Remy spoke of how the speed and attention to quality with which he can drive global projects today would not have been possible just a few years ago. Likewise, Don described at length the new opportunities to connect with and serve his clients that arise every day, from eliminating lines at local facilities to speeding resolution in the field with mobile devices in the hands of workers who can get closer to those they serve.
But how to encourage adoption of social technology among those not naturally inclined to use it? More importantly, how to ensure that adoption drives business value? Brian pointed to gamification as an answer. By designing a system whereby use of the technology rewards the user while aligning to business needs, everyone wins. There is evidence that such experimentation works. The 2012 IBM Tech Trends report found that those organizations ahead of the pack in applying advanced technology for strategic gain are nine times more likely than others to experiment with technology and encourage their employees to do so. They get people using new technology and building the skills they need even before formal projects are in place.
Building these skills � in social business, mobile computing, business analytics, and cloud computing � holds the key to renewed economic vitality. The Tech Trends report also uncovered a disturbing skills gap that threatens growth and is poised to get worse. In any of the four technology spaces mentioned above, only one in ten organizations has all the skills it needs to apply the technology for business advantage. Worse, among students and educators surveyed, a whopping 73% report a significant gap in their institution�s ability to meet market skill needs in those four spaces. Across the generations at work and the generation in school, more skill building needs to be done.
That�s why the San Jose State event was so exciting. This is a school and a group of students who are taking action and helping solve the problem. They know the real secret of social business is that people learn through interaction and exposure to a diverse set of experiences. They are applying that principle to the way they study the subject, actively participating in the IBM Academic Initiative to allow students and faculty to engage with experienced industry leaders around real world issues. And all of us � across generations � benefit by learning from each other and taking our skills to another level to drive greater success in our organizations.
Engaging and interacting more actively, therefore, will accelerate skill transfer across generations. But there is a way we can magnify our ability to learn from one another. Graham Mackintosh, in his keynote during the event, explained it in detail: social analytics. Describing smartphones as �human telemetry,� he pointed to an incredible amount of data now mapping people�s actions, movements, status, even their feelings. By analyzing this data, we are learning about one another at a terrific pace, uncovering better ways to serve consumers, faster ways to respond to emergencies, more effective ways to treat patients. And we have only scratched the surface. As Graham appropriately concluded his presentation, �It will be the next generation of business leaders � you � who will take this somewhere really exciting.�
Dan Hauenstein is the marketing and strategy leader for global IBM skill programs, including the IBM Academic Initiative, developerWorks, and the IBM Champion program. Follow Dan on Twitter @danhauenstein.
This post is contributed by Kim Madia, World Wide Product Marketing Manager for Infosphere.
Working at IBM, I have been fortunate enough to be a part of launching big data platforms to clients. Big data is more than simply a matter of size; it is an opportunity to find insights in new and emerging types of data and content, to make businesses more agile, and to answer questions that were previously considered beyond reach.
Living in the technology space, terms like sentiment analysis, analytics, and high velocity data are becoming familiar. However, it is interesting for me to see how those outside of the technology business are engaging in the big data phenomenon. Geoff Nunberg, teacher at the University of California at Berkeley, recently made a case for big data as the 2012 word of the year. Another example, The New York Times has ran articles about how the US held its first �big data presidential election� in 2012.
One topic that is picking up steam in popular media, and I expect will get even greater focus in 2013, is privacy. In this era of big data, understanding privacy couldn�t be more important. Privacy isn�t static and can�t be easily defined. Privacy also has different meanings across businesses, industries and cultures. Privacy rules therefore are a constant source of debate. For example, a person�s zip code might want to be kept private during a trip to a health care clinic but may want to be disclosed to a retail establishment.
Sometimes privacy is confused with security or anonymity. Though related, these terms are not the same. Privacy is defined as the ability to control use of information in different contexts.
Technology is available to help deliver privacy. Intelligent data masking inside big data platforms makes analytics possible while also keeping private information out of sight. A focus on privacy will fundamentally change how big data platforms are adopted. The end goal is to provide aggregated sensitive data to an analytics platform while protecting privacy. I believe in 2013 we will see more debate between IT professionals and governance regulators about how to create more effective privacy policies.
This post is by Erik Wiedenman, IBM Redbooks Manager
Well, I was sitting there the other day playing with blocks with my grand-nephew and the blocks made me think of Cognos Dynamic Cubes and how critical Cognos dashboards were to our transformation of a poorly coded, legacy metrics setup into a reliable, fast, consistent and flexible analytics environment.
OK, not really on the blocks part, but I�ve heard that blogs work better when they have a personal spin, so it seemed like a good way to start. But really, the part about the transformation is true, and as I drove into work this morning I was thinking about Cognos (while listening to a little Marshall Mathers), because I did get a real, tangible sense of value this week when I read the newly published Redbooks IBM Cognos Dynamic Cubes book.
In a previous job, I was a manager for metrics and analytics implementation. This was a wonderful job with the ability to really improve the business by getting and presenting data in ways that highlighted problems, suggested solutions and aided decision making. And as we got better and more sophisticated, we began to recognize not just problems but opportunities. I loved it.
However, we didn�t start out there. As any of you who�ve worked on metrics know, the path to good analytics is fraught with difficulties. We thought we had a good set of standard metrics, but �snap back to reality, oh, there goes gravity� � everybody liked our metrics, but nobody understood them and they often disbelieved them! How many times did we second guess our data when it didn�t show what we expected or wanted?
�That doesn�t look right to me? When was it loaded? Oh that�s old data from three days ago, this week�s ETL will show that we don�t really have a problem!�
�Can you rerun the reports, there must be something wrong? That�ll give us better results! What, next week?!?�
This was even more true in our case where we started from a legacy system with hard coded metrics (executed by a parade of interns years and years ago), and with data exponentially growing in size and complexity, in multiple unresolved databases or even embedded within the coding. Who even understood the existing reports?!?! We�d lost ourselves.
But �this opportunity comes once in a lifetime�. Thanks to significant executive and business owner support, a good plan and a great global team, �feet fail me not�, we were able to �move toward a new world order�. We got to all our data coming from rationalized databases with real time data flow and � critical factor - globally consistent Cognos management, reporting and dashboard capability. It was quite an exciting journey � and still is. I probably will blog a lot more about what we learned from our experiences, but in the interim here�s a great book that I wish I�d known about before we started: IBM Cognos Business Intelligence V10.1 Handbook. And really, the great part after a transformation like this is to be able to focus on improvements and actions around our real business information, not questions about the data, timing, running, etc.
In fact, I�m so happy with the progress we made, that here I am, in my next job, and I felt compelled to read the latest Cognos book (it�s really quite good) and even to use my time to edit the two Cognos books into additional documents � Solution Guides � which provides a good initial overview targeted to business leaders, management and professionals like myself. It�s worth your time to give them a quick look, because analytics is key to any great business, and in the end, success is our only option.
Years ago my father-in-law told me a story about a pal of his, a real estate investor who narrowly escaped a very risky deal. The man was about to finalize the purchase of a high-rise office building in Hawaii � in fact, he was in a conference room on the top floor of that building, waiting for the seller to arrive so they could close the deal.
To pass the time, the man went to the window to enjoy what would soon be his � or his paying tenants� � spectacular view. But as he approached the window he heard a faint, persistent, high-pitched tone, like a sound from a tiny tuning fork. And when he pressed his hand to the window he felt a slight but steady vibration.
It could have been nothing more than the wind. But the man went with his gut, delayed the purchase, and paid for a second set of building and site inspections. It turned out the building sat directly on a fault line � a fault line associated with the movement of subterranean volcanic magma.
My father-in-law�s friend walked away from the deal, feeling lucky. But IBM predicts that in the near future, this type of discovery won�t have anything to do with luck. One of this year�s IBM �5 in 5� predictions is that, five years from now, computers will hear what matters � e.g., that a swaying, healthy-looking tree is about to snap and fall into the road; that a customer you're talking with is unhappy, even if she�s smiling; that a baby�s cry means he�s sick, not hungry. (Being able to discern these subtleties will no doubt be valuable, but as the human attention span seems to approach zero as an asymptote, a computer that simply hears what we miss sounds like a must-have tool to me.)
Read about all five of IBM�s 2012 �5-in-5� predictions � each corresponding to one of the five human senses. Vote for the one you think will happen first. And don�t be afraid to admit which prediction (Being able to touch things through your phone? Digital taste buds?) makes you a little uneasy.
As organizations realize the power of both internal and external social networking tools, they are seeking employees with the skill set needed to advance social initiatives. Surprisingly, many organizations are discovering that their IT, research and development teams are a wealth of knowledge when it comes to working collaboratively.
For example, a popular software development methodology, Agile-Scrum, requires developers to stand up each morning and discuss the project they are working on, challenges they�re facing and how to overcome them. This gives developers the opportunity to talk through issues and take advantage of the collective skills of the group. In addition, when developers and IT professionals need an answer for a problem, they take advantage of internal and external resources � forums, blogs, Tweets or their coworker.
Data from the 2012 IBM Tech Trends Report indicates that more than two/thirds of the developers, students, professors and business leaders surveyed deem social business tools as imperative to their business success. The report actually revealed some pretty alarming statistics about the growing technical skill gap among today's professionals and students:
90 percent of those surveyed felt that their organization does not have the skills needed in business-critical technologies including social business, mobile, cloud computing or analytics.
More than 60 percent of IT/LOB decision makers and more than 73 percent of educators and students reported moderate to major skill gaps across all four technology areas.
And it's going from bad to worse: nearly half of the educators and students surveyed indicated major gaps in their institutions ability to meet IT skill needs in mobile, social, cloud and analytics. IBM announced a major new global skills initiative aimed at helping students, professionals and business leaders get the skills they need in social business and other key technologies.
Looking across your organization, how can you break down silos and help your employees embrace more digital, social and collaborative forms of communications?
Here are the top five skills each employee should acquire:
1. Define your social presence: Every time we engage in a social activity, we make choices that affect our presence. By presence, we mean achieving a position of distinction as a trusted business advisor, standing out in the marketplace as individuals and as an organization so that others can interact with us through our digital systems. The way you define your social presence will ultimately impact your eminence. You have the power to shape your eminence, through what you decide to share in the workplace and on the web.
2. Understand social communities and jump in: We all belong to communities in our daily, physical lives. With the advent of social media, we have the opportunity to join online, social communities that map back to our business and/or interest and connect with people across the world. Find and participate in social communities which help foster a greater sense of community belonging and inclusion.
3. Build your digital footprint or network by publishing your expertise: Work to build your "online brand" by cultivating your network and sharing your expertise. Internally, this will help coworkers find you quickly when a customer has a question or there�s an issue that comes up within your expertise.
4. Spend some time listening and engaging on social networks: There are a few steps within this step as you�ll want to listen and engage on the various external social networks that are important to you and your business. Run a Google search to see what others find when they search your name, check Twitter to see if you�ve received new @replies or direct messages and use relevant hashtags when posting.
5. Be a good digital citizen: A good digital citizen understands the responsibilities involved with using technology and digital media. Through awareness and thoughtfulness, you can be a good digital citizen and become a trusted and reliable resource.
These skills are easily acquired, but can take some time to fully develop. The first step in any social business is to familiarizing yourself with these skills and the technology available to help employees embrace their social profile.
Thanks again for making 2012 a spectacularly successful year for the IBM Software Blog. We'll be back in 2013 with more news, opinion and insights into the software that's changing the way the world literally works. Happy holidays!
The following is the sixth and final installment in our series on Advanced Data Visualization. Over the past three months, IBM visualization experts have explored new and emerging visual techniques and the underlying technologies you can deploy to better understand your data to transform insights into better business outcomes.
Frank van Ham is a well-known research scientist and an IBM Master Inventor with over a decade in experience in designing and deploying interactive information visualization. Some of his past projects include Many Eyes, a site for collaborative visualization and SequoiaView, a visual disk browser. Dr. van Ham currently works with the IBM Business Analytics division on integrating visualization into IBM's product portfolio.
Thanks to digital sensors, storage and processors we now live in a world that produces and stores a staggering amount of data, the vast majority of it in digital form. We often hear that all of this data has the power to transform many information-heavy industries, from health care to financial. However, most of this data is not useful in itself. The hardest challenge in dealing with so called 'big data' is not about scale or infrastructure, but about finding ways to refine it into useful information. In this blog post I will argue that it takes the combination of unique strengths from both humans and machines to successfully tackle this problem, and that visualization is the medium that ties the two together.
One possible route to attack 'big data' is to use the same computing power that has allowed us to gather all this data in the first place. We can use computer algorithms to refine this data for us and then present it in an understandable way. There's a plethora of algorithms that allow us to extract higher level features from raw data. For example, clustering algorithms allow us to identify larger groups of items that share a common property, statistical methods allow us to describe a the data in a set in terms of more abstract features and data mining methods allows us to extract often co-occuring events, for example. Commonly, we refer to the collection of all of these methods as 'data analytics'. Analytics is what allow us to wither down a large set of low-level factual observations into a smaller set of observations; however, they do not always provide us with information directly. Computers generally excel at fast and accurate data processing, but lack the context and creative skills to assemble these processed results into actionable information. In fact, solely relying on pure analytics to make decisions is dangerous for a couple of reasons:
More often than not analytics, return too much information and not all of it might be relevant. Unfortunately, computer systems often lack context to tell what information is relevant to you and what is not. At this point we still have the same problem as we did before (too much data), just at a smaller scale and higher level of abstraction.
The real world is not always easily captured in a nice mathematical model. Features in data exist at many different scales, with different amounts of certainty and algorithms sometimes make unwarranted and hidden assumptions to try and come up with a single answer. If your model is not detailed enough, analytic results might be incorrect; if your model is too detailed, analytic results might be unintelligible to a human operator.
Analytical or statistical summaries may sometimes hide what's actually going on in your data at a low level, again depending on the assumptions in the algorithm.
The following two charts try to illustrate this using two simple examples. Anscombe's quartet (left) presents four datasets that all have an equal number of points and virtually equal means and variance for both variables, as well as equal regression and correlation. Relying solely on aggregate statistics to describe these datasets without visually inspecting them would be grossly misleading. On the left I've plotted a small set of two dimensional points, some closer together than others. Suppose I want to know how many clusters of points are in this dataset. A human would probably say "it depends," while a clustering algorithm would say "3" (and yet another clustering algorithm might state "10"). Deferring critical decisions to �blind� analytics is dangerous in general and in all but the most straightforward cases you probably want a human in the loop to verify the results.
Unlike computer systems, the human brain is capable of putting information into context, making lateral jumps that connect two seemingly unimportant observations and provide creative hypotheses for an observed feature. This is (still) what makes us smarter than computers. In an ideal world, we would have humans place the results coming in from analytic processes in context and feed back their interpretations of the result into the processes themselves. Or in another analogy: Data mining algorithms provide us the tools to do the digging quickly, but deciding where to dig and what you do with the results is still very much a human decision. Or to quote Shyam Sankar in this TED talk : �You cannot algorithmically data mine your way to the answer. There is no �Find Terrorist� button.�
To realize this tight coupling between human operator and analytic tool, we need a medium that is suited to transfer information between both quickly and efficiently. Humans in general have a hard time interpreting large amounts of abstract information in numerical form because we only have limited working memory and are not naturally used to working with numerical representations. Instead, we have evolved to take in most of the information about the world around us in a visual manner. As a result, the part of our brains that do visual processing are well suited to spot outliers and detect patterns, without having to specify in advance what the patterns is.
Information visualization is a medium that uses computer algorithms to transform abstract data into visual imagery in a smart way, such that we can take advantage of our specialized "hardware". It allows us to quickly understand what is in a set of data and how the numbers relate. Note that I�ve deliberately designated visualization as a medium, not as a technique. Just like any medium, it takes skill to use it to communicate effectively and in a pleasant manner. In this post I�ve argued that visualization should be used to communicate analytic results from a computer to a human before they are used as basis for decisions. Other uses of the same medium involve communication of data from one human to another, for example to present data to another stakeholder. We will dig deeper into this interesting area of information processing in a number of future blog posts, so stay tuned!
Continue exploring visual analytics on IBM Many Eyes
Why stop the insight with this article? Visit IBM�s hub of visual analytics, IBM Many Eyes, and join over 100,000 like-mined visualization enthusiasts, academia and professionals. The Many Eyes web community democratizes data visualization by providing a simple three step process to create and interact with a visualization using your data set. Then share or embed your visualization across the web or your social network.
This post is courtesy of Ronnie Shelley, IAM Segment Manager for IBM Security.
Organizations of all sizes are facing the same dilemma today. The online tools and applications you use to open up communications with customers, enable a mobile workforce, and promote your offerings are the same ones used by cybercriminals to hack your web applications and steal your data. While large firms can combat these external threats by investing in robust security solutions and hiring the needed staff to manage them, it�s not as easy for the small organization with limited resources.
If you want to improve security without taking on the associated overhead, you may want to consider subscribing to Infrastructure as a Service or Software as a Service (SaaS) from third-party providers. Subscribing to a cloud-based service, rather than pur�chasing the hardware and software licenses required to run the technology in-house, can be a real boon to small and mid-sized organizations. They benefit from on-demand scalability, pay-per-use pricing, and relief from capital expenses for hardware purchases.
Cloud computing can provide a net gain in data security and system. The best cloud providers are usually well staffed and trained, with top-notch IT and security solutions at their fingertips. By offloading the management of some of your infrastructure to these providers, you�ll get access to advanced security solutions you may not be able to afford and manage on your own. You can focus on doing your business and let the cloud provider handle the security issues. In so doing, you could benefit from higher levels of protection and maybe an improved bottom line.
Yet, despite the clear benefits of cloud computing, protecting proprietary data and critical workloads remains a concern for organizations. These concerns can be resolved by choosing a provider with a solid, documented approach to safeguarding your data and applications. Questions to ask a potential cloud provider include:
Where and how will the data be stored?
How will your data be partitioned on shared servers and networks to ensure your information isn�t comingled with other clients� data?
Is the provider able to conduct audits and produce reports to demonstrate compliance?
What provisions are in place to protect your resources from a cyber attack?
How will the provider ensure your data is purged from their servers upon termination of the contract?
How will the access rights and activities of privileged users be managed?
These questions are not just academic. In all cases, even when using a third-party for hosting or cloud computing, the ultimate responsibility for protecting your company�s data falls on you � legally and in the eyes of the market. First, determine which workloads you�re comfortable placing in the cloud, and then be sure to select a cloud provider that has a strong history of providing secure hosting services. It is possible to improve security in the cloud. Visit the IBM Cloud Security Web site for more information.
Get more security news by following @IBMSecurity on Twitter.
I started with IBM in 1989. I still remember my first assignment. My manager called me and said �Next week a group of interns will be coming to IBM and I want you to present AS/400 networking capabilities to this group. By the way, if you need any resources, check out the AS/400 Redbooks in the IBM Library.� I had never heard of IBM Redbooks publications before, but that was not surprising because I was just starting my IT career. I went to the library (yes, we had a library in our office those days) and, on one of the shelves, saw these books that had the red covers. They were conveniently called �Redbooks.� I picked the one written for AS/400 and skimmed through the pages. I remember thinking to myself �These books are a great resource for product positioning, installation, and implementation experiences.� Of course little did I know that I would be authoring some � actually a lot � of these later in my career.
Fast forward 10 years. When I learned about a job opening in ITSO, it did not take long for me to decide about applying for the job. After a couple of interviews, I found myself managing my first Redbooks project. Those days, Redbooks and IBM Redpapers publications were the only deliverables ITSO produced, and these books were (and still are!) a great reference for implementers.
Fast forward another 13 years to the present day. I am still a Project Leader in ITSO, but ITSO now has many more deliverables in addition to our flagship Redbooks publications, such as:
These additions are all deliverables from the Redbooks brand; I�ve witnessed the transformation of Redbooks as a deliverable to Redbooks as a brand. I even contributed to creation of the last two deliverables.
The reason for this transformation is to respond to the evolving needs of our readers and address the end-to-end technical enablement requirements of a younger IT generation, especially in Growth Market Unit (GMU) based countries. These deliverables support IBMers and IBM Business Partners with pre-sales roles, and also IT Managers where as Redbook publications are more geared toward IT Delivery teams and implementers.
Another interesting fact about these deliverables is that most can be developed in an integrated development cycle, within the boundaries of a Redbooks project. For example, a residency team that is creating a Redbooks publication can also create videos, write blog posts, create a Solution Guide, or even throw in workshop material, based on the Redbooks. ITSO now has all the processes in place for all of these deliverables, and many examples of these have already been created.
Now we have another kid (or should I say �red�) on the block: Point-of-View publications (POVs) � brief, strategy-oriented documents that represent an author's perspective on a particular technical topic. Written by senior IBM subject matter experts, these publications examine current industry trends, directions, and emerging technologies. They further outline how readers can address business challenges with specific approaches, and explain the business value that a company or organization might gain by choosing the highlighted solution.
At about five pages in length, Point-of-View publications typically include solution highlights, a description of the problems being addressed, proposed solutions, and references for additional reading. They are available as PDFs for download. See the example below:
Several POV publications are already available on various topics:
Join me in welcoming this new "red deliverable� from ITSO and stay tuned for more POVs in the near future. I am interested in your thoughts. Let me know what you think of this new deliverable type and what topics would you like to see in the upcoming POVs. Enjoy our POVs!
Vasfi Gucer (email@example.com) is an IBM Redbooks Project Leader. He leads publications creation about Tivoli, WebSphere, and Cloud Computing.
The following is the fifth of a new six-part series on Advanced Data Visualization. Over the next three months, IBM visualization experts will explore new and emerging visual techniques and the underlying technologies you can deploy to better understand your data to transform insights into better business outcomes.
Graham Wills is the lead architect for IBM�s visualization engine. He has two decades experience in research and implementation of visualization systems in areas including statistical models, geo- and temporal- visualization, large-scale networks and coordinated views. He has published widely in the field and his recent book,Visualizing Timeis currently available on Amazon.
No, this isn�t a piece on modulo arithmetic, binary logic or the like. No need to call in the mathematicians and programmers. What we are going to discuss here is visualization design and how different visual features work together.
The figure below is an example of a well-designed chart. It�s a scatter plot of two numeric fields, with color used to encode a third field. When we look at the figure, we immediately notice two important features:
The points are spread out over the space, almost never even touching another point.
Most points are blue, four are green and one is red.
The data for this chart are in fact simulated data. The X and Y locations of the points are what is called �stratified random� � they are random, but only within certain bounds. In this case the data consist of 100 points placed on a 10x10 grid with a bit of random noise added to allow them to move a little. One of the reasons a scatter plot is such a great tool for numeric data is that we have a very good ability to assess distances from a fixed point � in this case, distances from the axes or boundaries of the data space. This chart draws a box around the data space and adds faint grid lines to make those comparisons even easier. That makes it easy to spot relationships between fields used for position on a scatter plot.
This is a general presentation rule � if you want to allow people to compare numeric values, the two best ways to do so are using aligned lengths (bars on a bar chart which all start at the X axis are a simple and powerful example) and by using aligned distances (like the scatter plot). In the chart we present above the regularity is immediately apparent. If we tried to use angles or color or something like that to show one of the fields, it would be much harder to spot that regularity.
The second feature of this chart is related to the field we use for color. Color is a seductively powerful way of encoding information. For a lot of human evolutionary history, it has been critical that we can identify items based on color. We have a strong ability to differentiate greens, a pretty good ability to differentiate shades of red and a relatively weak ability to differentiate blues. This is very probably because those are the colors of the foods that we and our ancestors ate. You are here, at least in part, because your great-great-(etc.)-grandparent was able to tell subtle differences between a red/green nutritious plant, and red/green poisonous one.
Color is a �grouping� function � we see colors in groups, not really as a continuous scale. Even if we present people with a chart that goes smoothly from blue to red (say), they will perceive it more in terms of groups of similar-colored items. For this reason, color is an excellent field to use for a categorical piece of data. Color does not also have a natural order; we can impose orders like blue/red, or heat scales; we can learn scales like those used in maps (blues get darker as the altitude goes below zero, browns get stronger as the altitude increases above the baseline); but these are not natural.
This chart is using color just as an indicator that a point belongs to a given group. This is the simplest and most effective use of color, and so this chart works: it represents the data not only truthfully, but also in a way that fits with our ability to interpret it.
We saw there was a pattern in the way the X and Y fields interacted � they were distributed regularly, more spaced out than we would expect. This chart is also clear in that it we do not draw false conclusions about the locations of the groups. The color of the points does not appear to have a dependency on their locations.
So far, so good. Take a moment to look at the chart on the left. This chart uses the same X and Y data, but instead of using color to map a field, we use symbol shape. Take a moment to look at the chart and compare it to the previous one.
This chart is effectively the same as the previous one. Although the field used for symbol shape contains different data, it is the same type of data (categorical) and, as in the previous chart, has three groups � a �default� group with most of the data, a group with 4 items (�green� previously, �square� here) and a singleton group (�red� previously, �plus� here).
Symbol shape works in a similar way to color. It is good for categories, has no particular order, and we process it mentally using a �grouping� function. People using charts based on symbols will say things like �the square points are mostly �� or �the bottom-right sector contains points from both plus and square groups�. The same language is used when working with charts using color.
So, off to our third example. Given our success with coding one field with color and another with symbol, it is natural for us to want to use both! If we have two numeric data fields and two categorical fields, it seems pretty clear that we can make a good chart using X and Y for the numeric values and color and shape for the categorical ones.
And, in the figure to the side here, this is exactly what we have done. Again, take a good look at the figure and compare it to the previous ones and see what conclusions can be drawn from it.
A famous quotation (much mangled in repetition) from American journalist H.L. Mencken (pictured at right) is �There is always an easy solution to every human problem--neat, plausible, and wrong�. This chart is not deceitful in purpose; it doesn�t misrepresent the data. It also follows good advice about drawing charts and for each mapping of the categorical field it is very plausible and very neat. In fact the only reason this chart fails is that at a fundamental level, combining shape and color just doesn�t work.
When we process symbol and color in our brains (or maybe just outside it � I�m not going deeply into the optic processing system in this article), we process them very separately. When we look at the 100 items in the first chart we instantly spot the unusual colors. If we had a million points we would do that identification just as fast. Similarly if we presented a million circles with just four squares and a single plus, we would immediately note and classify those unusual points. What we cannot do is process both at the same time. We cannot spot combinations of color and shape without detailed thought.
The third chart we showed works moderately well for comparing groups of color OR groups of symbol. The presence of the other encoding is distracting, but we can cope with that without much cognitive overhead. But if we wanted to deal with each separately, we could just use two charts more simply and more easily. The promise of combining both mappings in one chart is that we can spot patterns between them. But we cannot.
There is a critical feature of this chart that we cannot immediately spot � to find it we must carefully process each of the unusual items and investigate them sequentially. If you found the following feature in the chart � congratulations! It is not obvious and needs mental work to find. In this chart there is exactly one point that is both an unusual color and an unusual shape � the green square at the center bottom of the chart.
This is a critical piece of information. The red point is 1% of the data. If you assume color and shape are independent, then being unusual in color and shape are each 5% and so the combination is has a 0.25% probability of happening by chance (5% multiplied by 5%). This is four times as unusual, and it should be the most important point in the chart, and yet it is not visually obvious that this point is more than �slightly unusual�. This chart contains only 100 data points. The task becomes much more complex as the data and groups increase in number.
This article has been a cautionary tale. The human visual system is complex, and perhaps the strongest overall message to take away is that coding four or more fields into one chart is hard. It�s almost certainly best to avoid using two encodings like color/shape/size/orientation/texture for different fields if you have any interest in seeing relationships between those fields. Position, on the other hand, is very good for showing relationships. Maybe if you really need to see four fields, it might be better to use three for position and one for color? As we have seen, using two + two can lead to a chart that rates a solid �zero�.
Continue exploring visual analytics on IBM Many Eyes
Why stop the insight with this article? Visit IBM�s hub of visual analytics, IBM Many Eyes and join over 100,000 like-mined visualization enthusiasts, academia and professionals. The Many Eyes web community democratizes data visualization by providing a simple three step process to create and interact with a visualization using your data set. Then share or embed your visualization across the web or your social network.
The more information you have available to you, the better you'll be able to make informed decisions, spot trends, and make predictions. Right? Or does adding more information just mean more complexity � false positives and illusions?
What's for sure is there has never been more data available to us. 90% of the data in the world today, from all human existence, was created in the last two years. And it's growing. Today alone, 2.5 quintillion bytes (2.5 billion gigabytes) of data will be generated. Followed by another 2.5 quintillion bytes tomorrow. And the next day.
This is great news if more information means better informed decisions � terrifying news if it means the opposite.
Last November at IBM's Big Data conference Information On Demand, an unassuming statistician took the stage. His name was Nate Silver, and he was beginning to make a name for himself as an author. Later that month, he was making headlines the world over, having correctly called the outcome of the US presidential election in all 50 states.
Nate Silver's book The Signal and the Noise: Why So Many Predictions Fail � But Some Don't is a book about Big Data. It's filled with examples of analyzing data and mistakenly identifying what is thought to be signal (a meaningful pattern in data) with noise (random fluctuations). The examples include the ridiculous (the stock market gains an average of 14% when an NFC team wins the Super Bowl, and drops an average of 10% when an AFC team wins) and the worrisome (economists correctly predicted only 2 of the 60 recessions around the world in the 1990s).
Nate Silver makes the point that having more information in itself does nothing to improve our understanding of it. In fact, it can make matters worse. As the amount of available information increases, so do the number of hypotheses to understand that data.
Thankfully help is at hand � analytics.
In the IBM Redbooks Point-of-View publication Exploring the Potential of IBM Smarter Analytics Solutions, Jean Francois Puget and Baruch Schieber define how analytics applies to the process of using data to derive insights to make better decisions. Analytics are behind Nate Silver's impressive presidential election predictions, and behind the computer models that alerted meteorologists to the seriousness of super storm Sandy.
By applying analytics, you can assimilate, digest, and act on data. The Point-of-View publication cites plenty of examples of analytics at work � from helping Best Buy improve their advertising effectiveness, to Netherlands Railways optimizing their train schedule. Analytics can be used in simple form to present data in concise reports which aid decision making, through to its most complex form using analytics to collect, report, and ingest data to predict future trends and events.
Analytics is an area IBM has invested in for years, with over 10,000 technical analytics professional working for IBM, and over 9,000 consultants delivering IBM analytics solutions. IBM Smarter Analytics solutions connect people with trusted information so they can make real-time decisions and act with confidence in delivering better business outcomes. But Jean Francois Puget and Baruch Schieber explain it so much better: Exploring the Potential of IBM Smarter Analytics Solutions.
Don't delay � because here comes another 2.5 quintillion bytes of data that needs to be understood!
Martin Keenis an IBM Redbooks Project Leader. He works with technical experts to create books, guides, blogs, and videos. Follow Martin on Twitter at@MartinRTP.
This post was contributed by Brian Fitch, Product Manager for Network Protection.
The Tolly Group has just released a security efficacy report on the IBM Security Network IPS GX7800. The GX7800 was compared to open source SNORT version 22.214.171.124 and the latest (as of the time of testing) Sourcefire VRT (Vulnerability Research Team) updates.
The test consisted of a group of vulnerabilities with publicly available exploits. In the initial testing, the publicly available exploits were injected into network traffic. Both technologies performed well in detecting and blocking the attacks. The IBM GX7800 appliance blocked 99% of the attacks and SNORT blocked 91% of the attacks.
The testing then moved to testing against mutated attacks, or exploits that have a portion of their code changed. The code change does not affect the vulnerability targeted or how the vulnerability is exploited. This is important because it helps illustrate how successful technologies can be with regards to shielding the vulnerability from multiple exploit types as opposed to catching only a single exploit variant. A subsequent mutation of an exploit can be successful in exploiting the vulnerability as it now is undetected by a detection technology. The IBM GX7800 blocked 100% of mutated attacks as the Protocol Analysis Module recognized that these new attacks were still targeting the vulnerability, even if a portion of the exploit code had changed. The SNORT technology only blocked 52% of the mutated attacks.
The report also includes information on test configuration, equipment and technologies used, CVEs involved, as well as examples of exploit mutations. Performance capabilities of the GX7800 are also covered, including throughput metrics from both �drop� and �forward� operating modes of the appliance.
This post is courtesy of Mark Simmonds, Product Marketing for Data Governance and System z.
Previously I talked about the security of information and highlighted some of the capabilities among others of the InfoSphere Guardium solution for the System z platform. The latest release has added support for big data feeds like Hadoop. So your compliance, security and audit capabilities on data moving between System z and other platforms continues its pervasiveness . Being an enterprise wide solution supporting many different platforms and vendor databases you might expect it to be complex but it is architected in an elegant way that makes it seem so logical. The latest version incorporates enhancement that makes the fulfillment, installation, operation and support of the solution ever simpler. The best way to find out is to sign up for the webcast below (oh and you get a complementary white paper afterwards)�.
Moving on... when we hear the term 'security breaches' we are inclined to think about hackers and external threats trying to hack your mainframe or other data - and while that's true we often overlook what happens during the development and testing of applications. It only takes a member of the team to leave a test data report containing personal identifiable information lying on a desk or left open on a screen and a person's identity could be compromised. Many IT shops believe the only real way to really and thoroughly test applications or business processes is to use real data and after all, you can trust your own employees right? . Ehhhh No. Data can be breached maliciously or accidentally. So how can you protect your data from such incidents?
Masking or de-identifying the data renders it useless if stolen because personally identifiable information is no longer real - being replaced with fictitious generated data values. But that is only part of the problem. That data needs to maintain its referential integrity after it has been masked and it still needs to retain its behavioral characteristics such as structure of say a credit card, the built in check sums, zip, date of birth, social security number formats. Then you have the issue of what happens should you need to refresh the data.
I'm kicking off the week with another deep dive into the recently released 2012 IBM Tech Trends Report. Today, I've extracted the findings on Mobility, another of the critical emerging technologies that organizations are counting on to drive growth into 2013 and beyond...
Adoption & Investment:
49 percent of respondents have deployed Mobile
Top three barriers to adoption of Mobile according to those surveyed: Security (61 percent); integration of Mobile with existing infrastructure and data (44 percent); and difficulty extending existing applications to Mobile (38 percent)
69 percent plan to increase Mobile investment in the next two years, with 35 percent planning to increase it 10 percent or more
Over the next two years, 31 percent of respondents will start allowing BYOD � making it the norm for 76 percent of respondents
Only 9 percent of companies think they have all the Mobile skills they need: 25 percent of IT/LOB decision makers report a major Mobile skills gaps, with an additional 40 percent seeing moderate skills gaps (65 percent total)
45 percent of students and educators see major Mobile skills gaps, with an additional 32 percent seeing moderate skills gaps (77 percent total)
43 percent of respondents say their IT security policies don�t meet the needs of Mobile computing
50 percent are actively engaged in increasing security capabilities to existing Mobile applications
Growth market respondents appear to be lagging in Mobile deployment, with 35 percent citing current deployment versus 55 percent of mature markets
The shift toward BYOD is even more pronounced in growth markets, where 41 percent will allow BYOD within two years, bringing the total to 83 percent in those countries
This post was written by Anne Lescher, Product Marketing Manager with IBM Security Solutions.
One of the biggest challenges is protecting sensitive information, and one of the biggest fears is losing that information to hackers. Who has not worried about losing a backup tape or disc that holds millions of customer account numbers and reading about it in a news story that destroys your company�s reputation?
And yet we are equally fearful of the encryption technology that can protect our most sensitive information. We are afraid of the complex cryptographic algorithms and key exchange protocols, often comparing it to rocket science. We are equally afraid of the performance impact to our production workloads and online customer systems when accessing encrypted data. And finally, we are afraid of losing the encryption keys and thus losing all access to the data itself while trying to protect it.
Most of us no longer have any choice in whether we encrypt our data. As the number of security breaches continues to grow, regulations are increasingly adding more stringent protection controls for retail, healthcare and other industries, governments, and standards groups. These regulations are being enforced and punished with larger financial penalties. And that does not include the damage to your company�s market image and financial losses due to a data breach.
Ideally, we seek solutions that offer strong standardized encryption technology based on interoperable algorithms that can be implemented as transparently as possible to protect our information. Ideally, we desire solutions that support multi-vendor hardware self-encryption storage devices, that can interoperate with software data base access control solutions, and that can be managed by automated encryption key lifecycle management. These solutions must monitor and audit data protection to demonstrate compliance with regulations.
The good news is that there are industry standards groups that cooperate to deliver standardized encryption algorithms and key management interoperability protocols allowing security vendor products to work and play well together to protect your mission critical information.
IBM offers integrated hardware and software data security solutions that include:
IBM�s Tivoli Key Lifecycle Manager solution helps IT organizations better manage the encryption key life cycle. It enables them to centralize and strengthen key management processes with automated simplified capabilities that provide an intuitive user interface for configuration and management. It dramatically reduces operations complexity while facilitating compliance management of regulatory standards such as Sarbanes-Oxley and the Health Insurance Portability and Accountability Act (HIPAA). It also extends key management capabilities to both IBM and non-IBM products by leveraging open standards such as Key Management Interoperability Protocol (KMIP) to help enable flexibility and facilitate vendor interoperability.
The good news is that data security solutions can simplify the protection of your essential information. These solutions are a robust combination of integrated hardware and software with automated protection, monitoring, auditing and reporting to help you meet the stringent regulatory data requirements. They can simplify the protection of your data and take the rocket science out of data encryption.