Difficulties of Implementing Data Warehouses
Some significant operational issues arise with data warehousing:
construction, administration, and quality control. Project management—the
design, construction, and implementation of the warehouse—is an important and
challenging consideration that should not be underestimated. The building of
an enterprise-wide warehouse in a large organization is a major undertaking, potentially
taking years from conceptualization to implementation. Because of the
difficulty and amount of lead time required for such an undertaking, the
widespread development and deployment of data marts may provide an attractive
alternative, especially to those organizations with urgent needs for OLAP, DSS,
and/or data mining support.
The administration of a data warehouse is an intensive enterprise,
proportional to the size and complexity of the warehouse. An organization that
attempts to administer a data warehouse must realistically understand the
complex nature of its administration. Although designed for read access, a data
warehouse is no more a static structure than any of its information sources.
Source databases can be expected to evolve. The warehouse’s schema and
acquisition component must be expected to be updated to handle these
evolutions.
A significant issue in data warehousing is the quality control of data.
Both quality and consistency of data are major concerns. Although the data
passes through a cleaning function during acquisition, quality and consistency
remain significant issues for the database administrator. Melding data from
heterogeneous and disparate sources is a major challenge given differences in
naming, domain definitions, identification numbers, and the like. Every time a
source database changes, the data warehouse administrator must consider the
possible interactions with other elements of the warehouse.
Usage projections should be estimated conservatively prior to construction of the data warehouse and should be revised continually to reflect current requirements. As utilization patterns become clear and change over time, storage and access paths can be tuned to remain optimized for support of the organization’s use of its ware-house. This activity should continue throughout the life of the warehouse in order to remain ahead of demand. The warehouse should also be designed to accommodate the addition and attrition of data sources without major redesign. Sources and source data will evolve, and the warehouse must accommodate such change. Fitting the available source data into the data model of the warehouse will be a continual challenge, a task that is as much art as science. Because there is continual rapid change in technologies, both the requirements and capabilities of the warehouse will change considerably over time. Additionally, data warehousing technology itself will continue to evolve for some time so that component structures and functionalities will continually be upgraded. This certain change is excellent motivation for having fully modular design of components.
Administration of a data warehouse will require far broader skills than
are needed for traditional database administration. A team of highly skilled
technical experts with overlapping areas of expertise will likely be needed,
rather than a single individual. Like database administration, data warehouse
administration is only partly technical; a large part of the responsibility
requires working effectively with all the members of the organization with an
interest in the data warehouse. However difficult that can be at times for
database administrators, it is that much more challenging for data warehouse
administrators, as the scope of their responsibilities is considerably broader.
Related Topics
Privacy Policy, Terms and Conditions, DMCA Policy and Compliant
Copyright © 2018-2024 BrainKart.com; All Rights Reserved. Developed by Therithal info, Chennai.