Data Quality: Dimensions, Measurement, Strategy, Management, and Governance

✍ Scribed by Rupa Mahanti

• Publisher: ASQ Quality Press
• Year: 2019
• Tongue: English
• Leaves: 528
• Category: Library

✦ Synopsis

This is not the kind of book that you ll read one time and be done with. So scan it quickly the first time through to get an idea of its breadth. Then dig in on one topic of special importance to your work. Finally, use it as a reference to guide your next steps, learn details, and broaden your perspective.
from the foreword by Thomas C. Redman, Ph.D., 'the Data Doc'

Good data is a source of myriad opportunities, while bad data is a tremendous burden. Companies that manage their data effectively are able to achieve a competitive advantage in the marketplace, while bad data, like cancer, can weaken and kill an organization.

In this comprehensive book, Rupa Mahanti provides guidance on the different aspects of data quality with the aim to be able to improve data quality. Specifically, the book addresses:
-Causes of bad data quality, bad data quality impacts, and importance of data quality to justify the case for data quality
-Butterfly effect of data quality
-A detailed description of data quality dimensions and their measurement
-Data quality strategy approach
-Six Sigma - DMAIC approach to data quality
-Data quality management techniques
-Data quality in relation to data initiatives like data migration, MDM, data governance, etc.
-Data quality myths, challenges, and critical success factors

Students, academicians, professionals, and researchers can all use the content in this book to further their knowledge and get guidance on their own specific projects. It balances technical details (for example, SQL statements, relational database components, data quality dimensions measurements) and higher-level qualitative discussions (cost of data quality, data quality strategy, data quality maturity, the case made for data quality, and so on) with case studies, illustrations, and real-world examples throughout.

✦ Table of Contents

Title Page
CIP Data
Table of Contents
List of Figures and Tables
Foreword: The Ins and Outs of Data Quality
Preface
Acknowledgments
Chapter 1: Data, Data Quality, and Cost of Poor Data Quality
The Data Age
Are Data and Data Quality Important? Yes They Are!
Data Quality
Categorization of Data
Master Data
Reference Data
Transactional Data
Historical Data
Metadata
Data Quality: An Overview
How Is Data Quality Different?
Data Quality Dimensions
Causes of Bad Data Quality
Manual Data Entry
Inadequate Validation in the Data Capture Process
Aging of Data/Data Decay
Inefficient Business Process Management and Design
Data Migration
Data Integration
Data Cleansing Programs
Organizational Changes
System Upgrades
Data Purging
Multiple Uses of Data and Lack of Shared Understanding of Data
Loss of Expertise
Lack of Common Data Standards, Data Dictionary, and Metadata
Business Data Ownership and Governance Issues
Data Corruption by Hackers
Cost of Poor Data Quality
The Butterfly Effect on Data Quality
Conclusion
Summary of Key Points
Key Terms
Categories of Data
Cost of Poor Data Quality—Business Impacts
Causes of Bad Quality Data
Chapter 2: Building Blocks of Data: Evolutionary History and Data Concepts
Introduction
Evolution of Data Collection, Storage, and Data Quality
Database, Database Models, and Database Schemas
Relational Database
Data Models
Normalization
Data Warehouse
Dimensional Modeling, Fact, Dimensions, and Grain
Star Schema, Snowflake Schema, and OLAP Cube
The Data Hierarchy
Common Terminologies in a Nutshell
Summary
Chapter 3: Data Quality Dimensions
Introduction
Data Quality Dimensions—Categories
Data Quality Dimensions
Data Specifications
Completeness
Conformity or Validity
Uniqueness
Duplication
Redundancy
Consistency
Integrity
Accuracy
Correctness
Granularity
Precision
Ease of Manipulation
Conciseness
Objectivity
Data Coverage
Relevance
Interpretability
Believability, Credibility, and Trustworthiness
Reputation
Time-Related Dimensions (Timeliness, Currency, and Volatility)
Accessibility
Security
Traceability
Data Reliability
How to Use Data Quality Dimensions
Data Quality Dimensions: The Interrelationships
Data Quality Dimensions Summary Table
Summary
Chapter 4: Measuring Data Quality Dimensions
Measurement of Data Quality
Data Quality Dimensions Measurement: Subjective versus Objective
What Data Should Be Targeted for Measurement?
Critical Data Elements
Metadata and Data Quality Measurement
Data Quality: Basic Statistics
Data Quality Dimensions: Measurement
Measuring Completeness
Measuring Uniqueness
Measuring Validity
Measuring Accuracy
Measuring Consistency
Measuring Integrity
Measuring Volatility
Measuring Currency
Measuring Timeliness
Measuring Data Lineage
Measuring Data Coverage
Measuring Relevance
Measuring Accessibility
Measuring Data Security
Measuring Data Reliability
Measuring Ease of Manipulation
Measuring Conciseness
Measuring Objectivity
Measuring Interpretability
Measuring Believability, Credibility, and Trustworthiness
Assessing Reputation
How to Conduct Data Profiling
Manual Data Profiling
Data Profiling Using Spreadsheets
SQL Scripts for Profiling Data
Use of a Data Profiling Tool
Summary
Chapter 5: Data Quality Strategy
Introduction
What Is a Data Quality Strategy?
Data Strategy versus Data Quality Strategy
Why Is a Data Quality Strategy Important?
Data Quality Maturity
Data Quality Maturity Model
Level 1: The Initial or Chaotic Level
Level 2: The Repeatable Level
Level 3: The Defined Level
Level 4: The Managed Level
Level 5: The Optimized Level
Scope of a Data Quality Strategy
Data Quality Strategy: Preplanning
Phases in the Data Quality Strategy Formulation Process
Planning and Setup
Discovery and Assessment
Prioritization and Roadmap
Data Quality Strategy Document and Presentation
Overview
Assessment Approach
Findings and Recommendations
Initiative Details
Implementation Roadmap and Financials
Characteristics of a Good Data Quality Strategy
Data Quality Strategy Implementation
Who Drives the Data Quality Strategy?
Chief Data Officer (CDO)
CDO Capabilities and Responsibilities
Reporting Structure
Data Quality Strategy—A Few Useful Tips
Conclusion
Chapter 6: Data Quality Management
Data Quality Management
Data Quality Management—Reactive versus Proactive
Data Quality Management—Six Sigma DMAIC
Define
Measure
Analyze
Improve
Control
Data Quality Assessment
Data Access and Preparation
Data Profiling and Analysis
Results and Recommendations
Root Cause Analysis
Data Cleansing
Parsing and Standardization
Matching, Linking, and Merging
Data Enrichment or Data Augmentation
Data Validation
Presence Check Validation
Format Validation
Range Validation
Lookup Validation
Data Type Validation
Data Quality Monitoring
Data Migration and Data Quality
How to Ensure Data Quality in Data Migration Projects
Data Quality Assessment
Data Cleansing Approach
Solution Design—Data Mapping
Coding, Testing, and Implementation
Data Migration Assessment
Data Migration Technology
Data Integration and Data Quality
Data Integration Challenges
Data Warehousing and Data Quality
Master Data Management and Data Quality
Master Data Profiling
Master Data Integration and Data Quality
Data Governance and Master Data Management
Metadata Management and Data Quality
Six Sigma DMAIC and Data Quality Management—Example
Six Sigma Define Phase
Six Sigma Measure Phase
Six Sigma Analyze Phase
Six Sigma Improve Phase
Six Sigma Control Phase
Key Principles of Data Quality Management
Conclusion
Summary of Key Points
Data Quality Management
Six Sigma
Data Quality Assessment
Root Cause Analysis
Data Cleansing
Data Validation
Data Quality Monitoring
Data Migration
Data Migration and Data Quality
Data Integration
Data Integration and Data Quality
Master Data Management (MDM)
Master Data Management (MDM) and Data Quality
Metadata Management
Chapter 7: Data Quality: Critical Success Factors (CSFs)
Introduction
Data Quality Myths
Myth 1: Data Has to be Absolutely Perfect
Myth 2: Data Quality Is Data Accuracy
Myth 3: All Data across the Enterprise Need to Be of the Same Quality
Myth 4a: Good Data Remain Good Forever
Myth 4b: Once Our Data Is Cleaned, It Will Remain Clean
Myth 5a: Data Quality Tools Can Solve All Data Quality Issues
Myth 5b: Data Quality Tools Are Plug-and-Play
Myth 6: Data Quality Is Not My Responsibility
Myth 7: With Data, No News Is Good News
Myth 8: Our Data Are Too Complex
Myth 9: Our Data Are Very Different from Others
Myth 10a: Data Quality Is a Luxury That Can Wait
Myth 10b: Data Quality Is a “Nice to Have”
Myth 11: Return on Investment (ROI) for Data Quality Initiatives Is Difficult to Calculate
Myth 12: Data Quality Is Expensive
Myth 13: Data Quality Issues Are Only Due to Data Sources or Data Entry
Myth 14: Data Quality Checks Are a Part of the Extract, Transform, and Load (ETL) Process
Myth 15: ETL Tools Offer Much the Same Functionality as Data Quality Tools
Myth 16: Data Quality Is Data Cleansing
Myth 17: Data Quality Is Data Assessment
Myth 18: If Data Works Well in the Source Systems, the Same Should Apply in Target Systems
Myth 19: Data Migration Will Fix All Your Data Quality Issues
Myth 20: Data Quality Is Too Risky
Myth 21: Data Quality Is the Same as Data Governance
Data Quality Challenges
Responsibility for Data and Data Quality, and Cross-Functional Collaboration
Intuition-Driven Culture
Lack of Awareness
Resistance to Change
Funding and Sponsorship
Return on Investment
Reasons for Failure of Data Quality Initiatives
Data Quality—Critical Success Factors
Leadership and Management Support
Sponsorship
Organizational Culture
Managing Change
Business User Involvement
Tools and Technologies
Skill Sets, Knowledge, and Staffing
Accountability and Ownership
Teamwork, Partnership, Communication, and Collaboration
Education and Training
Success Stories
Robust Data Quality Strategy
Data Governance
Track Return on Investment
Key Considerations While Establishing a Data Quality Program
Think Big, Start Small
Balancing the Extremes
Set Expectations
Get a Few Quick Wins
Acknowledge That It Will Not Be Easy
Buy Data Quality Software after Understanding the Business Process and Related Data
Multiple Solution Options
Monitor Progress
Commitment to Continuous Improvement
Conclusion
Summary of Key Points
Challenges
Key Considerations
Critical Success Factors
Chapter 8: Data Governance and Data Quality
Introduction
Evolution of Data Governance
Data Governance
Data Governance Misconceptions
Misconception #1: Responsibility for Data Governance Lies with IT
Misconception #2: Data Governance Is Restricting
Misconception #3: Data Governance Is a Project
Misconception #4: “Data Governance” Is a Synonym for “Records Management”
Misconception #5: Data Governance Is All about Compliance
Misconception #6: Data Governance Is about Documented Policies
Misconception #7: Data Governance Is Siloed by Business Units or Department
Misconception #8: Data Governance Is a Published Repository of Common Data Definitions
Data Governance versus IT Governance
Why Data Governance Fails
Data Governance and Data Quality
Data Governance, GDPR, and Data Quality
Data Governance and PII
Data Governance Framework
Why Is a Data Governance Framework Needed?
Data Governance Framework Components
Rules and Rules of Engagement
Data Policies
Processes
People
Roles and Responsibilities
Technology
Data Governance—Top-Down or Bottom-Up Approach
Conclusion
Summary of Key Points
Data Governance
Data Governance Misconceptions
Data Governance Framework
Data Stewardship
Data Principles
Appendix A: Data Quality Dimensions: A Comparison of Definitions
Appendix B: Abbreviations and Acronyms
Bibliography
Index