Data warehouse

Data warehouse

     Data warehouse is data management and data analysis

     Goal: is to integrate enterprise wide corporate data into a single reository from which users can easily run queries

            Benefits

   The major benefit of data warehousing are high returns on investment.

   Increased productivity of corporate decision-makers

            Problems

   Underestimation of resources for data loading

   Hidden problems with source systems

   Required data not captured

   Increased end-user demands

   Data homogenization

   High demand for resources

   Data ownership

   High maintenance

   Long-duration projects

   Complexity of integration

Main components

          Operational data sources - - > for the DW is supplied from mainframe operational data held in first generation hierarchical and network databases, departmental data held in proprietary file systems, private data held on workstaions and private serves and external systems such as the Internet, commercially available DB, or DB assoicated with and organization‘s suppliers or customers.

          Operational datastore(ODS) - - >  is a repository of current and integrated operational data used for analysis. It is often structured and supplied with data in the same way as the data warehouse, but may in fact simply act as a staging area for data to be moved into the warehouse.

          query manager - - > also called backend component, it performs all the operations associated with the management of user queries. The operations performed by this component include directing queries to the appropriate tables and scheduling the execution of queries

          end-user access tools - - > can be categorized into five main groups: data reporting and query tools, application development tools, executive information system (EIS) tools, online analytical processing (OLAP) tools, and data mining tools.

          Data flow

          Inflow- The processes associated with the extraction, cleansing, and loading of the data from the source systems into the data warehouse.

          upflow- The process associated with adding value to the data in the warehouse through summarizing, packaging , packaging, and distribution of the data.

          downflow- The processes associated with archiving and backing-up of data in the warehouse.

          Tools and Technologies

The critical steps in the construction of a data warehouse:

          Extraction

          Cleansing

          Transformation

after the critical steps, loading the results into target system can be carried out either by separate products, or by a single, categories:

          code generators

          database data replication tools

          dynamic transformation engines

For the various types of meta-data and the day-to-day operations of the data warehouse, the administration and management tools must be capable of supporting those tasks:

            Monitoring data loading from multiple sources

            Data quality and integrity checks

            Managing and updating meta-data

            Monitoring database performance to ensure efficient query response times and resource utilization

            Auditing data warehouse usage to provide user chargeback information

            Replicating, subsetting, and distributing data

            Maintaining effient data storage management

            Purging data;

            Archiving and backing-up data

            Implementing recovery following failure