Architectural principles, and the designs and technologies used to implement them, form the bedrock of BI initiatives but are invisible to business users. Attention to architecture ensures BI success.
Table of Contents
When building capabilities for business intelligence (BI), most enterprises focus on the elements that are visible to the business users: functionality in query/reporting tools and BI applications, training on these tools and applications, and the impact of BI on critical business processes. Far too little time is spent on "behind the scenes" or "hidden" aspects of BI: the critical underpinnings that ensure a robust implementation capable of delivering insight in a reliable, scalable and flexible manner. The architecture of the individual components, as well as the overall BI solution, can make or break a BI effort. In this Spotlight, we examine the issues and technology components that are key to delivering a BI architecture that will meet users' immediate requirements and ensure long-term value to the enterprise.
A significant majority of the IT effort expended in a BI project is consumed by data integration issues. Designing a repeatable process by which data is acquired from operational systems, transformed, integrated and delivered to the data warehouse is technically challenging. In addition to issues of data security, ownership and quality, the proper selection of technology for data integration is critical, but not obvious. Project teams continue to employ different tools for the task, centering around extraction, transformation and loading (ETL) tools and application integration suites. These two classes of technology continue to converge, leading to contention between proponents of each, as described in "ETL and Application Integration Suites Convergence Continues."
In addition to debates over ETL and application integration suites, interest and hype is on the rise for the virtual data federation, which some vendors label "enterprise information integration." "Virtual Federation Will Augment BI Data Architecture" explains how this style of data integration technology will augment proven approaches in some interesting ways, rather than revolutionize the data architecture for BI.
In delivering the data foundation for their BI initiatives, project teams focus on finding the data they need, extracting it from its sources and delivering it to the data warehouse or other analytic structures. They tend to overlook the quality of that data, a major reason for the failure of BI and data warehousing projects, and a significant operational challenge for large enterprises. "Data Quality 'Firewall' Enhances Value of the Data Warehouse" suggests how enterprises can address data quality issues in the BI context. "Banks and Asset Managers Are Enhancing Data Quality" presents the results from a recent study of data quality issues and practices in the financial services industry.
Limited awareness of the value of metadata in the BI context creates challenges for many enterprises. Rigorous focus on capturing, managing and providing visibility into metadata is required. Without this focus, the project teams implementing the architecture, tools and applications forego the benefits of reuse and affect the analysis. In addition, business users are hampered in their ability to understand the lineage and quality of the data. In "Best Practices for Managing Data-Warehousing Metadata," we explore these issues, as well as options for getting the most value from metadata in the BI domain.
Scalability and Interoperability
As BI deployments grow in complexity and breadth, the ability to scale the architecture increases in importance. Many deployments perform poorly and are unreliable because production work loads with large numbers of users push the architecture to its limits. Enterprises need to focus on scalability when evaluating tools and technologies for BI because poor performance is one of the fastest ways to kill business-user interest and acceptance of a BI project. "Scalability and BI: What You Need to Know Now" details the critical scalability-oriented points to analyze when evaluating BI tools.
As organizations take a more-service-oriented approach to building and deploying business applications, they need BI tools and applications that can operate with each other and with non-BI applications and infrastructure components. The impact of Web services in this area is evolving slowly, as described in "Web Services for BI Applications Have Yet to Be Exploited."
Business Activity Monitoring
As more businesses strive for the ideal of the real-time enterprise, there is growing interest in reducing the latency of BI delivery. Making faster decisions based on more real-time information can benefit enterprises seeking faster and more-efficient operational processes. Business activity monitoring (BAM) is a style of BI applications that harnesses real-time events in the context of business operations. "How the Pieces in a BAM Architecture Work" provides an overview of the components of BAM applications and how they are related.
"ETL and Application Integration Suites Convergence Continues" — Extraction, transformation and loading tools and application integration suites are merging slowly, with vendor convergence preceding true technology convergence. By Ted Friedman, Bill Gassman and Roy Schulte
"Virtual Federation Will Augment BI Data Architecture" — The virtual federation style of data integration technology will be used to augment, rather than replace, common approaches to data acquisition for business intelligence. By Ted Friedman
"Data Quality 'Firewall' Enhances Value of the Data Warehouse" — A "firewall" approach can prevent poor-quality data from entering the data warehouse. By Ted Friedman
"Banks and Asset Managers Are Enhancing Data Quality" — Results from a Gartner survey indicate steps that financial services providers are taking to improve data quality. By Mary Knox
"Best Practices for Managing Data-Warehousing Metadata" — Scope, standards and integration are the most-important best practices for implementing a metadata management plan for BI. By Michael Blechar
"Scalability and BI: What You Need to Know Now" — When selecting BI tools and applications, enterprises must carefully evaluate vendors' approaches to achieving scalability. By Howard Dresner
"Web Services for BI Applications Have Yet to Be Exploited" — Web services have the potential to improve interoperability between BI tools and related applications, but real-world proof points are rare. By Bill Hostmann
"How the Pieces in a BAM Architecture Work" — An architecture for business activity monitoring is delivered via multiple technologies working in concert. By Bill Gassman