โ Scribed by Wojciechowski, Artur
No coin nor oath required. For personal study only.
External data sources (EDSs) being integrated in a data warehouse (DW) frequently change their data structures (schemas). As a consequence, in many cases, an already deployed ETL workflow executes with errors. Since structural changes of EDSs are frequent, an automatic reparation of an ETL workflow after such changes is of a high importance. In this paper we present a framework for handling the evolution of an ETL layer. To this end, structural changes are monitored and stored in a Metabase. An erroneous execution of an ETL workflow causes a reparation of the ETL activities that interact with the changed EDS, so that the repaired activities can work on the changed EDS schema. The reparation of the ETL activities is guided by several customizable reparation algorithms. The proposed framework was developed as a module external to an ETL engine, accessing the engine by means of API. The innovation of this framework are algorithms for semi-automatic reparation of an ETL workflow.
๐ SIMILAR VOLUMES
In the last years, the problems of using generic storage techniques for very specific applications has been detected and outlined. Thus, some alternatives to relational DBMSs (e.g., BigTable) are blooming. On the other hand, cloud computing is already a reality that helps to save money by eliminatin