Welcome to Software Technology Magazine.

New methodologies and technical repository breakthroughs in Enterprise Architecture led the topics this June at E-Gov, the convention and seminar series in Washington, D.C. promoting electronic government throughout the United States. As new ones are discovered daily, E-Gov technologies and processes are being adapted for more efficient and effective government at reduced tax rates and for direct communication with citizens and businesses via one aspect of E-Gov, namely the Internet, Web portals, and Web services.

The most recent example of E-Gov in action, one that has the potential to affect nearly every American household, is the recent success of the Administration's DoNotCall.gov Web site. Upon opening at 8:30 a.m. EST, this Web site was registering the numbers and email addresses of upwards of three-quarters of a million applicants at the rate of over 108 registrations per second.

Efficient software development management through the application of new Enterprise Architecture techniques led to the construction of a capable and functioning site serving U.S. citizens in record time, according to the participants at E-Gov, which included Andrew Ide, CIO of White House Administration Staffs and Mark Powell, Chief Technology Officer for the Joint Chiefs of Staff. Now these methodologies and techniques are being applied to Homeland Security, according to Steve Cooper, CIO for the U.S. Department of Homeland Security.

The challenge for Homeland Security is to devise an architecture that is secure and aids rapid information-sharing and collaboration at all levels of government, as well as the private sector. The DHS National Enterprise Architecture is not just Federal, but reaches out to State and local governments. Optimal ways to include the private sector are being explored.

The Department of Homeland Security has an enormous challenge ahead of it, according to Cooper, because the DHS is a new Federal Government domain encompassing 22 formerly independent agencies with widely different computer- and information systems, not to mention government- and business processes. The new Department has already identified more than 2,500 "mission-critical applications or automated solution sets" and more than 50,000 items [data processing systems] that make up its IT infrastructure. Despite various levels of pre-existing architectures at many of the newly-integrated agencies, the sheer scope of the task is, of course, quite remarkable.

On the positive side, however, the process of taking inventory was estimated in April to be 40% to 50% complete. This initial inventory of its entire IT infrastructure is targeted for completion around the end of June and will constitute a critical step toward the ultimate creation of the Department's Nationwide Architecture.

The Department has further commenced an aggressive outreach effort led by a series of independent task forces to identify business processes common to its five Directorates, while two separate task forces have been studying infrastructure and application security. A third task force is studying security from a physical and business-process standpoint.

The challenge of building a robust Enterprise Architecture that is both open and secure has been one of the many key topics in the lectures and seminars of E-Gov during the week of June 9th through 12th. New EA techniques developed by OMB and the hallowed halls of university computer departments were selected, recommended, and applied by various Administration working groups, including the President's Critical Infrastructure Protection Board. The Board's perspective is that the Government has to rethink the way it creates IT architectures. Government organizations used to look at what they could do with them, as opposed to [also] looking at what [an adversary] could do with them. The constant introduction of new technologies is forcing officials to redefine that it means to have a secure architecture. Yet having an [efficient and effective] architecture as well as a secure one, while seemingly two conflicting goals, are not, according to the Board's chairman-elect, Howard Schmidt.

Handheld and wireless devices now have to be explicitly included in Federal architectures, Schmidt maintains, and so Enterprise Architecture designs and thought processes have to change. When IPv6 [Internet Protocol Version 6.x] is adopted, everything is connected and will have an IP address, and [Federal] architectures will be different.

The FAA, realizing that secure government agencies will never be able to get away from needing multiple layers of defense, has taken the first step toward making security a core component of its Enterprise Architecture by integrating its information systems security with the overall National Airspace System (NAS) and its IT and communications [sub] architectures. FAA task forces are examining and 'harmonizing' the administrative and mission-support areas. In a recent example, the FAA discovered, to its surprise, that putting its IS [information systems] security architecture on top of the NAS architecture [as an overarching structure] and then integrating the two added necessary constraints on the IS Security Architecture that would not have been included if the IS Security Architecture had been developed separately.

As the FAA experience shows, System Migration and Enterprise Application Integration often take center stage when a government or large corporate organization commences the rigorous planning and requirements determination necessary to formulate a successful Enterprise Architecture. The equivalent architectures of old, be they 'Grand Plan', 'Spiral Chaos', et al, were shaped by expensive, heterogeneous collections of mainframes and data centers; obscure, proprietary (non-open), and time-consuming application development tools; and incompatible vendor-specific communications protocols — all working in a closed environment where few were granted access, but those who were, were trusted with total access.

Many of these past patterns of operation and management remain, distorting and weakening otherwise capable enterprise infrastructures. As IT requirements and environments change, architectures that once made sense now become expensive monuments to the resistance to change. While architectures cannot adjust to every passing pop-software COTS, widget, or fad, classic and enduring designs can be constructed and evolved using the new breakthroughs in the flexible use and communication of enterprise-maintained and external data.

Intranet Portals and Web Collaboration Tools Help Align Enterprise Architecture

Enterprises and organizations have always had a problem aligning IT development with business goals. Management tends to talk in terms of business models, business processes and specific business activities, while the IT department talks about technology and how IT will be integrated into an overall architecture. At the summary or top management level, the result is a capability gap caused by the time taken to translate business requirements into IT systems. This translation approach risks losing definition in the process. In slow moving markets, these deficiencies can be tolerated. But in today's fast moving, highly competitive, low growth environment, businesses cannot afford such delays and lack of clarity.

A new method of making the link between business objectives and IT is needed. Today's environment demands a unified, synchronized approach that creates an overall business architecture. In this, the rules for constructing the business at the process, people, and organization levels are aligned, right from the start, with the rules governing IT structures and standards. A common Enterprise Architecture will let organizations set rules and standards for their businesses and better link the business and IT functions. This will make it easier to justify "the right" IT investments in terms of the appropriate business needs, as well as allow for rapid extensions to the business model and its supporting processes, such as the speedy integration of new acquisitions.

Hampered By The Enterprise Architecture

Since EA has long been seen by management as purely a matter for the IT department — where only IT departments had the relevant expertise — enterprise organizations saw IT applications as strictly back-office functions. The big change that made information and communications technology a business issue was the advent of the Internet (and for governments, the potential for paperwork reduction and E-Gov). Suddenly, government and private sector organizations could talk to customers, suppliers, and stakeholders quickly, cheaply and easily. For the first time in history, business models were being driven by computer-mediated applications — and attaining unprecedented business success. In the rush to the Web, many organizations perceived, for the first time, that their antiquated "legacy" enterprise architectures were hindering business and new-process development, rather than supporting it. The increasing need to respond fast to new mandates, sudden environmental changes (e.g., 9/11), and competition caused many enterprise architectures to fall behind in their ability to serve business goals.

Today, large-organization IT-related delays and inflexibility continue to hinder business growth. CRM projects, real-time operations, and business process outsourcing plans all suffer from technology-related hold-ups and in-adaptability. Managers rarely knew how "to fix the disconnect" between the quality of IT infrastructures and the need for strategic change.

The business architecture portion of Enterprise Architecture removes the artificial distinction between business and IT planning. It tackles the rules for how a business is put together and the enterprise architecture that dictates the shape of the IT environment at the same time, and brings them into alignment from the start. Both are architectural concepts, and they are much more likely to work smoothly together if they are linked from the outset. When both architectures are devised at the same time, the organization can plan for the optimum use of IT investments. The enterprise architecture will avoid producing IT systems that are expensively over-specified to cover every eventuality, or unable to adapt to serve new demands and urgent government or business needs.

Capability Breakthroughs Simplify the Development of Enterprise Architectures

Current trends toward external portals and Web Services, unification of heterogeneous and geographically-dispersed networks (multiplying beyond all bounds of reason), and the new ease of real-time data collection and analysis/application enable — and indeed compel — corresponding changes in the ways that an enterprise adds new, heretofore unimaginable, capabilities — and structures those capabilities into its strategic and functional processes.

New user application functions and software technology tools (e.g., Web Services and Enterprise Architecture design and integration tools) are driving IT organizations to undergo their most fundamental and rapid change in decades. Developers and IT/software departments no longer have the luxury of specialization — not only because of technical obsolescence, but also by reason of the acceleration of competitive and security-development timeframes: the disciplines of business process development, database- and data systems management, and network overhaul and integration are unarguably blending into a single function. The 'front office' and the 'back office' of enterprise computing are consolidating into the interface door. Faced with such changes, enterprise IT managers and builders must now adopt a comprehensive architect's perspective.

Enterprise success, opportunity, and effectiveness now depend on leadership in conceiving, implementing, and using new kinds and sources of information made available by new requirements, technologies, and circumstances. PC technologies, for example, made finer granularity possible — businesses and governments could customize manufactured items, design custom-tailored pharmaceuticals to specific diseases, and deliver efficacious and desirable services, rather than inundating customers or citizens with ineffective and unsatisfactory "one size fits all" mass issues.

Advances in Software Tools Support the Breakthroughs

In the eternal tradeoff of creativity and innovation vs. time- and cost-efficiency through standardization, there will always be a need for compromise — as the pendulum of practicality swings back and forth between the extremes. In software development houses in the past, lip service was paid to the goal of software reuse, but it was difficult for developers — even within a single organization — to identify and share code that could performed common functions without, eventually, some sort of tweaking and customization.

Object-oriented technologies improved the definition of function and interaction among the modules of any given application, enabling some commercial traffic in modules for such horizontal needs as I/O and other universal elements of applications. Only toward the turn of the new century, though, did modules become self-disclosing — enabling them to answer the calling module's question, "Can you do this for me?"

Within those objects or modules, however, purpose-built data representations required a hodgepodge of specialized data connections, owing to vendor-specific software designs. The transforming effect of Web services, though, amounted to going through the enterprise, identifying common components of what people need and do while acquiring needed software items and equipment in quantity, to be combined as needed for the smooth functional operation of the whole, just as all office building appliances use the same basic utilities and connections (or are provided appropriate adapters or transformers), rather than requiring redundant incompatible power systems. Because Web Services — and the Enterprise Architecture middleware/intranet equivalents — provide results from inputs, functional specifications, and external sources without regard to the means of accomplishing them, difficulties and complexities of processing and provision of data and information are transcended, greatly reducing costs and accelerating an Enterprise Architecture's — and every IT project's time-to-completion.

Pervasive open networks have also opened the IT office space to the outside world, broadening workers' access to resources but also exposing them to unprecedented threats. Endless discussion has addressed the need for boundary security, but intrinsic flaws in security architectures and their applications leave architects and administrators with an unattractive choice: either shutting their doors to subsets of their customers, or leaving them open to skilled invaders.

New Open-Architecture Software Engineering Tools Support More Effective Architectures

The standardization of protocols and data formats has converged on what Microsoft Corp., under the rubric of Windows DNA, identified as the tripod of application integration: TCP/IP for transport, HTTP for interaction and XML for representation. The last of these, XML, challenges development toolmakers to integrate flexible and powerful tools for XML authoring, inspection and dynamic transformation into their tool sets.

Altova Inc.'s XMLSpy, and increasingly TIBCO's TurboXML and SoftQuad's XMetal, combine versatile editing power with broad support for varied database platforms, multiplying that power by easing integration with Java and C++ programming. While vendors have not yet marketed an integrated environment whose XML tools are in the same class of comprehensiveness and ease of use, Oracle Corp.'s JDeveloper is praiseworthy for its use of XML throughout its own structure for ease of customization. Emphasizing the breakdown of traditional discipline boundaries, tools such as Corel Corp.'s Ventura 10 combine programmable XML transformations with more traditional publishing power.

At the Enterprise Architecture level, and with services being produced and consumed across the enterprise boundary, modeling and testing take on new importance. For example, Microsoft's Visual Studio .Net deserves praise for its integration of Web service hosting and testing facilities into the development cycle, and tools such as Popkin Software and Systems Inc.'s System Architect offer more capable process simulation functions.

As information and function interact up and down the entire supply chain, it's vital for application code to have internal controls on what resources should be available to what tasks. Mainframe developers may get a grim satisfaction from the industry's painful rediscovery of disciplines that seemed to have been left behind by the luxury of "one user, one machine," but in the wake of 9/11 and many Internet hacker worms, security issues today are more complex than ever. The security that's always been available in Java, and that's far more accessible and easy to tailor in Java 2 Enterprise Edition than it was upon Java's debut, sets the standard—but the security features available to developers on Microsoft's .Net platform deserve more attention.

Information Architectures and Data Management Advances

The not-so-easily-reached goals for an enterprise data architect are to manage information in ways that make it available to all those who can benefit or help the organization benefit from it while keeping it private from the unauthorized, maintaining accuracy in owned data, and allowing data values to be properly interpreted in the context of their own genesis/history and in comparison to related data. There's long been a tension between offices maintaining operational databases and 'warehoused' data. Centralized data warehouses have historically been too outdated and inaccessible to provide information about what's going on right now. Operational data repositories, maintaining information on an organizations current activities (and environment), tend to be joined at the hip with specific applications and controlled by individual business units. Big can mean slow, and immediately relevant can mean isolated and inaccessible. Advances in data management technology, however, now allow that trade-off to be re-examined.

While raw speed and capacity continue to march forward inexorably — the Transaction Processing Performance Council's TPC-C database benchmark results show each of the top-five single-server benchmarks now processing more than 400,000 transactions per minute — these kinds of throughput gains in database systems and servers can make consolidation of operational servers more feasible than in the past. Fist-sized terabyte drives and growing capacities are fundamentally changing the traditional design principles behind data storage and warehousing — namely, the need to use a separate warehouse database loaded during an offline window from multiple operational sources. Put together with E-Gov and the simplicity of Web Services, these breakthroughs have immediate distributed processing and network consolidation implications for an organization's Enterprise Architecture.

Oracle Corp. has for years led the way in data storage leadership, with its ongoing addition of data warehousing features such as bit-mapped indices to its core transactional database. Oracle even killed its stand-alone OLAP database Express (traditionally used for data warehousing) in favor of a new online analytical processing extension to Oracle9i. IBM's DB2 and Microsoft SQL Server are also effective tools in this application, with their support for pre-computed queries (materialized views) and OLAP-influenced SQL functions such as rollup and cube.

Information "Rapid Response" becomes a New Requirement within Enterprise Architecture

Processing and analyzing data in near real time at the decision-maker's level is viewed as the Holy Grail of enterprise data management. More organizations, particularly government ones such as the Department of Homeland Security, are discovering that real-time information capabilities are crucial to implementing the organization's strategy.

Doing more analysis directly on the operational database using in-place processing is the wave of the future. Transactional databases continue to gain functionality in this area, and closely coupling operational and warehouse processing prevents the schema and semantic mismatch that can happen when the two systems are maintained separately. Along with greater volumes of data, organizations are working to more quickly extract and respond to information. This dynamic is partly driven by portal deployments, since portals make data staleness painfully obvious.

Real-time data analysis requires a more fluid approach to data storage than has been traditionally taken. Data in transit, data stored on a client's system or data stored in an internal database is now viewed as a broader understanding of "database." These 'federated', or conceptual, databases are often scattered across the organization as well as with customers, suppliers, Web Service data repositories, and public information sources, and are identifiable primarily only as a set of focal topics of mutual interest among the data sources.

Dynamic data gateways are key strategic tools in the EA process, as are more robust data collection, data validation, and data culling tools at the start of the customer interaction chain. With today's demand for real-time information, data cleansing and aggregation can no longer be left until weeks after data collection occurs. Web services are proving to be an effective way to link disparate data systems to loosely coupled federated databases. In the Web services model, data is available in real time, mapped to a common, interoperable data format (XML) and supported through the entire client/mid-tier/database application stack.

XML technology breakthroughs are already profoundly changing the data landscape as a vehicle for data and information blueprinting within Enterprise Architecture. Every major transactional database is undergoing major internal surgery to natively support XML Schema data types and XML Query as a query language. Microsoft's forthcoming Office 2003, for example, provides data creators with more powerful ways to generate structured, highly reusable data than ever before. While practical databases continue to store data in a variety of incompatible ways, XML-influenced trends of data fluidity, in-flight conversion and transformation, and self-describing data constructs (i.e., the co-location of data with its structural description) have markedly changed how organizations manage and manipulate data.

Tools for Building a Strong Information and Data Architecture

Enterprise Architectures must therefore incorporate designs for portal, Web Service-based, and mobile application deployments to keep employees in close contact with data. Database replication techniques will support 'federated' warehousing and data sourcing for broad distribution while enforcing data quality rules and requirements. Consolidated 'single access point' methodologies and/or bio-identification technologies will permit security architectures to be implemented as close to the data as possible by using, at minimum, row- and column-level permissions.

Middleware and access-via-Internet (e.g., SOAP and UDDI) tools can enable departments to build databases by building and maintaining subsets of larger operational databases while enforcing process and business rules at all points of data entry and at all stages of the data life-cycle using Enterprise-Architecture-identified shared business logic.

Data does not become information, however (much less useful information) unless it is interpreted in context. New tools for folding warehouse and analysis tasks into operational databases to allow all data to be manipulated in one place aid in contextual reporting. OLAP, data mining, data visualization and statistical analysis are more highly effective (although underused) when applied as strategically-justified techniques within an over-arching architecture. Web Service architecture tools allow data from ERP (Enterprise Resource Planning) and similar vertical applications to be integrated into line-of-business information systems far more easily than ever before.

Training and Top Management Support Makes Enterprise Architecture Work

In more mature organizations, Enterprise Architecture is unfortunately viewed as impractical "conceptual rocket science" causing extended delay in IT rollouts and meaningful only to "the Tech guys." But building a top-notch enterprise architecture isn't just about getting networks, servers and other technical resources to mesh. It's also about integrating corporate strategies and business architectures and developing the human resources — the knowledge, experience, skills and perspective — needed to execute them.

An Enterprise Architecture cannot succeed unless the organization ensures that its executive team fully understands the effort is about providing business management and actionable information at the highest levels — and therefore the career development opportunities that go with them. Executive involvement in the business architecture portions of EA must include the appropriate overview training so that management teams have an appropriate buy-in and feel it enhances their personal and professional growth. In a redesigned performance management system and 'management control system', support is coordinated and garnered from the line managers as well as the general manager level; for example, in providing HQ overhead to cover the Enterprise Architecture portion of training costs, so line manager bottom lines are not penalized but rather rewarded on performance via the improved business functionality that an effective EA supporting field offices would provide.

Commitment from top management (or at the Federal Secretarial level) is necessary for evaluating and justifying an organization's return-on-training investment. To the extent possible, use of e-learning systems can be promulgated so that fewer execs need be absent from full work.

Second, incentive and encouragement can be established via EA points toward an 'emerging leaders' program and by helping senior management understand the value of the training investment.

Building a Business Architecture

A customer- and stakeholder-oriented style of architecture can be defined at both the business and IT levels. Organizations pursuing a customer-oriented strategy would base its architecture on a generic style for that purpose, recognizably similar to the architectures used by other organizations and enterprises targeting the same value discipline, but could then add in ideas and capabilities of their own, reinforcing strengths and differentiation through a distinctive business model, cogent business processes and a carefully-designed IT infrastructure. The resulting business architecture sets the rules as to how business and IT functions are not only put together, but also unearths how they are — need to be — interrelated.

As part of the business architecture portion of the EA, each of the organization's divisions or business units must look at its business plan, look at its resources and look at the operational mandates or customer market which it serves and try to forecast what different skills it needs. Consequently, the required skill sets for full-time Enterprise Architecture can no longer be a collection of academic and IT credentials — successful enterprise architects must be business-focused first. It's not difficult to get narrow, specific technical people — or even to get management college graduates or MBAs, but finding professionals with skills and experience in both camps can be challenging.

Finally, a Board member, Inspector General, or other oversight executive must take responsibility as an outside reviewer to ensure that the Enterprise Architecture is sound and is leading to meaningful results. The CIO of an enterprise or organization must be made to yield its Enterprise Architecture to be a jointly shared responsibility between top management and IT — rather than ending up another stovepipe or fiefdom. And ultimately, the challenge for building a good Enterprise Architecture organization if to find people having both business and IT savvy and who can make efficacious decisions under conditions of complexity and ambiguity and while having only incomplete information.

To summarize key E-Gov Enterprise Architecture recommendations and findings:

To avoid hobbling the business processes, an enterprise must define its business model and enterprise architecture at the same time.

Switching to a unified business architecture takes time and effort — enterprises that can afford to pace themselves should take migration and sequencing gradually.

Enterprise Architecture is still seen by some managements as a 'back-office' activity. Its role in meeting new resource-short mandates and enabling business growth is not fully understood.

Business and IT convergence in a legacy organization is usually limited to functional specification handovers or an ad-hoc project-based necessity.

Top management must ensure that Enterprise Architecture shifts from being a purely IT concept and activity to be an expression of the enterprise's business architecture — an information management extension of mission and goals, strategy, processes, and programs.

The organization's CIO/CTO must understand that IT no longer exclusively owns the Enterprise Architecture.

Enterprise architects should create a business architecture linking all aspects of the business model with the Enterprise Architecture. Where performed, strategic planning activities should apply scenarios linked to the "strategic drivers" (value discipline pursued by the organization) to identify the appropriate generic architecture style.

An Inspector General professional or Board-level executive should be appointed to provide an overall outside review of the fit of the enterprises' Enterprise Architecture with its business architecture, linking all aspects of the business model.

Although emerging advances breakthroughs at every level are enhancing Enterprise Architectures for E-Gov and enabling the Department of Homeland Security and others to meet unprecedented missions and mandates, the business process changes needed to support the switch to a unified Architecture takes time and effort — and may initially incur considerable costs. Only those organizations or enterprises where the need for change is most pressing should commence a formal Enterprise Architecture overhaul immediately. For others, a more gradual and orderly migration or transition ('sequencing') is more appropriate, based on the organization's major IT systems and lifecycles, a la Capability Maturity Model (CMM), etc. Realigning the way this vital top-level strategic planning is done over the business planning period of the next two to four years can be quite challenging — but in today's volatile world, may mean the difference between achieving the mission — or ignominious failure and collapse.