 | Level: Intermediate Uche Ogbuji (uche@ogbuji.net), Consultant, Zepheira
07 Aug 2007 Explore obstacles that data architects often experience and learn strategies to work beyond them in this article. Build from small successes that begin with bridging departmental application data models to creating full enterprise integration projects. Apply these approaches to other types of software architecture, to enable your IT team to more efficiently handle changing requirements and IT approaches.
Introduction
Data architecture is an emerging profession that focuses on the statics (data
modeling) and dynamics (data flow) of information within systems, across multiple departmental systems, or even throughout an enterprise. Data architects are custodians of all forms of data, from relational databases to flat files, from documents and content to reports and transactional data. There is some overlap with enterprise architecture, which seeks to optimize overall business process through information technology, and information architecture, which looks for the best ways to present information as part of a user's experience (typically on the Web). Recently, organizations have been getting the message that it is essential to manage data in a holistic way, independently of the systems that presently process it, and so the role of the data architect has increased in prominence. Despite this, data architects face real difficulties in pursuing their profession at many workplaces. Most of these difficulties are similar to those that dogged other software architect disciplines as they were emerging, and present overall lessons for all such professions.
Change is the catalyst
Things are always a-changing in information technology (IT). In any given year, the executive
radar may focus on a portfolio of stand-alone N-tier applications, another year
enterprise resource planning (ERP) projects, and yet another year Service-Oriented
Architecture (SOA). Developers and managers come and go, businesses merge, and
industry trends shift. Keeping up with so much change has always been the hardest
undertaking of IT and the cause of many of its most infamous challenges, including
failed projects and cost overruns. Recent events have focused attention on the
importance of maintaining the value of an organization's data through such changes.
The success of Web technologies, including XML and the emergence of SOA have served as
carrots for getting executives to respect data architecture, and regulatory
imperatives such as the Sarbanes-Oxley law in the U.S. have served as sticks—auditors and compliance officers now routinely warn executives that they will probably have to retain and manage data records long beyond the useful lifetime of the applications that are processing these records. Doing so has always been good business, and now it is becoming settled law.
Data architects are increasingly getting decision makers to respect their broader
role within the enterprise. Unfortunately, they often find their profile and
responsibility raised without the resources or authority to meet expectations, and
this combination can be a perilous one, not just for an individual career, but also
for an organization's overall goals. Durable data architecture usually requires minor
changes to applications and business processes, and the position in the organization
chart rarely comes with the mandate for such change. In the past decade ERP projects
were closely paired with far more sweeping changes under the banner of business process reengineering (BPR), and these changes were mandated by executive fiat. Most data architects have no such executive decree to rely on, even though they ask far less of the organization than ERP. This is the stark reality, and the professional has no choice but to catalyze such change in a piecemeal fashion, one small victory at a time.
Roadblocks
It's almost a gratuitous exercise to catalog the slings and arrows a data architect
is likely to face advocating enterprise data models. In my discussions with peers, I find the list is almost universal, and all too familiar. Still it's useful to take stock of the roadblocks in order to develop an effective strategy for moving forward.
"It's just data"
The seams in data architecture always show during integration projects. Systems
integration has long been viewed as a merging of capabilities rather than a merging of
information. Program managers focus on code and design patterns rather than shared
data models. It's not unusual for a project integrating two applications to result in
a merged application that operates on the two original silos of data, as shown in
Figure 1. A lot of the resulting merged application framework is devoted to the
brokering of requests across these silos. In the "bad old days" of ad hoc
programming, decision makers were famous for dismissing the idea of software
architecture by saying "It's just code." These days, code gets a generous helping of
design and planning, and you're likely to hear instead, "It's just data." Some parts of the
spectrum, such as relational data, enjoy focused design, and this is a result of the
tireless advocacy of its professionals. Such attention is often not considered for
all the various sorts of data in a system. It will take more general advocacy to
spread this consciousness, but an architect who insists on such attention might be
sidelined, or more often, is at first accommodated, only to have her priorities
shelved if unanticipated circumstances put the squeeze on project goals, resources, or scheduling.
Figure 1. Data silos not always merged in integration projects
Dueling perceptions
Strangely, hand in hand with the "it's just data" problem, you often find dispute
over control of the data. Important data tends to come from applications that are
developed by one department, but ultimately, it needs to be shared by others. An
integration project with the goal of generalizing control of the data often runs into border disputes between interests. Without careful design, data can end up being locked into one department's point of view, lowering its value elsewhere, but it can be difficult to negotiate a neutral design, and the resulting disputes can sometimes scuttle the project.
Technical hurdles
Next to the organizational hurdles, the technical ones almost seem scant, but when it
comes to the task, data versioning, concurrency (locking), and life cycle management
are never simple. It's still very difficult to define and apply the business rules
that should shape the data and to provide the visualization technology to make that
shape accessible to the users of the data. All these technical problems are
exacerbated by dueling perceptions (different departments might want different locking or life cycle rules), and they require the sorts of resources to tackle that might not be forthcoming because "It's just data."
The way forward
You never get to your destination by focusing on the roadblocks. There are a variety of tactics a data architect can use to further his cause in the enterprise.
Enlisting champions
The simplest course of action in establishing data architecture is to find a friend.
All professional interactions are negotiations, and it's always important to win a key
decision maker to your side. Doing so not only gives you an advocate, but also
improves your chances of being taken seriously by other managers. There can sometimes
be a herd mentality among decision makers, and there is reluctance to be seen as the
source of negativity. You can look for managers who were veterans of successful
projects with strong data architecture in former jobs or even within their present
positions. Knowing the background of key decision makers is the ticket to maintaining
their interest as you explain the value you offer. Perhaps your champion could be a
manager whose group most clearly stands to gain from the project. An effective data
architecture should certainly seek to provide benefits across the organization, but
often it's clearer to show a group with one set of interests how your work can make it
more effective. There is nothing wrong or Machiavellian in honing your pitch to such
parties first. One dangerous tactic I've come across is to enlist a vendor as
champion. Integration projects are often undertaken by managers with a vendor in
mind, either for software or services to effect the integration. It may be tempting
to try to align data architecture with that vendor's plan for performance, but it is
very rare that such alignment is truly possible. Most software and integration
services are designed with the vendor's organizational model in mind, and most would prefer to push some degree of BPR upon your own organization than shape their wares to suit the unique needs of the client.
Building bridges
Different departments may have different points of view, but ultimately, they want to
find ways to work together. Helping them share data provides real value in terms of
cost reduction, cost avoidance, and, in some lucky circumstances, can even be a
revenue driver—for example, if your work presents not-previously-exploited
cross-sell and up-sell opportunities for customers. Take advantage of the desire of managers to take charge of business process by encouraging governance boards for key shared data, clearly communicating how data flow drives process. This may not be as hard as you think. Generally, the same managers who are reluctant to provide more tangible resources are happy to join a committee of practice. The close communication within such a cross-functional group will give you the sort of feel for the organizational practicalities that inform data architecture. Usually, the opportunity to build such bridges comes in association with a strategic project with a high profile, one with stakeholders in multiple departments.
Iteration, iteration...repeat
An enterprisewide project might seem a pipe-dream to most data architects, but the
road to such a project starts with more modest milestones. If you see opportunity in
some small, but strategic project—a new, promotional e-commerce Web site, for
example—you can make a focused pitch to the stakeholders, perhaps about the
importance of data that might be available from a seemingly distant source or of the
expected gains in productivity from streamlined data flow. Having a champion or two
would help a great deal in this undertaking. If you can muster a toehold in the
project plan, be forward-looking in your designs. If the project is successful, be
sure to go back to the stakeholders and highlight the actual benefits of your designs,
and even the future benefits in terms of possible integration with other systems.
They may become champions for the next project to build on your success. After a
string of such projects, you probably have a data model that serves the needs of
several departments, and more ambitious work becomes a possibility. You also have the
benefit of several iterations to address some of the technical difficulties, which can
allow you to extrapolate the solutions for greater scale.
 |
Summary
Almost every professional discipline to have emerged in recent memory has started off
facing problems resembling those discussed in this article. It's easy for the data
architect to feel downtrodden in the face of such issues, but it's important to focus
on the positives. Organizations are giving unprecedented attention to the role of
data across the enterprise, and despite initial difficulty, this can only mean good
things for the future of the profession. By seeking champions among management,
building bridges between departments, and scoring modest successes in carefully
selected projects, the professional data architect can cement this future by slowly turning their
promises of benefits for the enterprise into reality.
Resources Learn
Get products and technologies
-
Trial download: Rational
Data Architect. This enterprise data modeling and integration design tool from
Rational® software is designed to
help data architects design relational and federated databases, understand data
assets and their relationships, and streamline database projects.
- Build your next development
project with IBM trial software, available for download directly from developerWorks.
Discuss
About the author | 
| | Uche Ogbuji is partner at Zepheira, LLC, a
solutions firm specializing in the next generation of Web technologies. Mr.
Ogbuji is lead developer of 4Suite, an open
source platform for XML, RDF, and knowledge-management applications and lead
developer of the Versa RDF
query language. He is a computer engineer and writer, born in Nigeria, living
and working in Boulder, Colorado, U.S.A. Find out more about Mr. Ogbuji at his blog Copia. |
Rate this page
| |
