i
ii
United Republic of Tanzania
TANZANIA DATA QUALITY ASSURANCE FRAMEWORK FOR OFFICIAL STATISTICS
Prepared by: National Bureau of Statistics Tanzania
and Office of Chief Government Statistician Zanzibar
JUNE, 2024
iii Table of Contents List of Tables ..................................................................................................................................... v FOREWORD .................................................................................................................................... vi ABBREVIATIONS AND ACRONYMS .................................................................................. viii DEFINITIONS OF THE TERMS .................................................................................................. x CHAPTER ONE ................................................................................................................................ 1 BACKGROUND INFORMATION .............................................................................................. 1 1.0 Background ............................................................................................................................... 1 1.1 Rationale for a Data Quality Assurance Framework ................................................... 2 1.2 Objectives of Quality Assurance Framework ............................................................... 3 1.3 Scope and Coverage ......................................................................................................... 4 CHAPTER TWO ............................................................................................................................... 5 NATIONAL STATISTICAL SYSTEM IN TANZANIA ........................................................... 5 2.0 Coordination of National Statistical System ...................................................................... 5 2.1 NSS Current Situation ...................................................................................................... 6 CHAPTER THREE ........................................................................................................................... 8 DATA QUALITY ASSURANCE FRAMEWORK PRINCIPLES AND REQUIREMENTS8 3.0 Introduction .............................................................................................................................. 8 3.1 Level A: Managing the Statistical System ..................................................................... 8 3.1.1 Principle 1: Coordinating the National Statistical System .................................... 9 3.1.2 Principle 2: Managing Relationships with Data Users, Data Providers and Other Stakeholders ................................................................................................................. 12 3.2 Level B: Managing the Institutional Environment ..................................................... 19 3.2.1 Principle 4: Assuring Professional Independence ................................................ 19 3.2.2 Principle 5: Assuring Impartiality and Objectivity .............................................. 22 3.2.3 Principle 6: Assuring Transparency ....................................................................... 25 3.2.4 Principle 7: Assuring Statistical Confidentiality and Data Security .................. 27 3.2.5 Principle 8: Assuring Commitment to Quality ..................................................... 30 3.2.6 Principle 9: Assuring adequacy of resources ........................................................ 36 3.3 Level C: Managing Statistical Processes ...................................................................... 38 3.3.1 Principle 10: Assuring methodological soundness .............................................. 38 3.3.2 Principle 11: Assuring cost-effectiveness ............................................................... 42 3.3.3 Principle 12: Assuring appropriate statistical procedures .................................. 46 3.3.4 Principle 13: Managing the Respondent Burden .................................................. 50 3.4 Level D: Managing Statistical Outputs ........................................................................ 53
iv 3.4.1 Principle 14: Assuring Relevance ........................................................................... 53 3.4.2 Principle 15: Assuring Accuracy and Reliability .................................................. 56 3.4.3 Principle 16: Assuring timeliness and punctuality .............................................. 59 3.4.4 Principle 17: Assuring accessibility and clarity .................................................... 62 3.4.5 Principle 18: Assuring coherence and comparability .......................................... 66 3.4.6 Principle 19: Managing metadata .......................................................................... 69 CHAPTER FOUR ........................................................................................................................... 71 IMPLEMENTATION OF QUALITY ASSURANCE FRAMEWORK .................................. 71 4.0 Quality Management ............................................................................................................. 71 4.1 Institutional Arrangement ............................................................................................. 71 4.2 Quality Unit ..................................................................................................................... 73 4.2.1 Internal Quality Management Team ...................................................................... 73 4.2.2 Quality Statistics Technical Working Group ........................................................ 73 4.2.3 Quality Assurance Managers .................................................................................. 75 4.2.4 Quality Assurance Directors ................................................................................... 75 4.2.5 NSS Quality Governance Board .............................................................................. 76 4.3 Training of Staff ............................................................................................................... 76 4.4 External and Internal Communication ........................................................................ 76 4.5 Development of Implementation Strategy and Action ............................................. 76 4.6 Assessment and Reporting ............................................................................................ 77 4.6.1 Quality Indicators ..................................................................................................... 78 4.6.2 Quality Reports ......................................................................................................... 78 4.6.3 Obtaining Feedback from Users ............................................................................. 78 4.6.4 Conducting Assessments ......................................................................................... 78 4.6.5 Labeling ...................................................................................................................... 79 4.7 Assuring Continuous Quality Improvement .............................................................. 79 ANNEX I: GENERIC STATISTICS BUSINESS PROCESS MODEL (GSBPM) ................... 80 ANNEX II: PROCESSES DURING DATA QUALITY ASSESSMENT ................................ 81 References ........................................................................................................................................ 83
v List of Tables Table 2.1 SWOC Analysis ............................................................................................................. 7 Table 3.1 Principle 1: Coordinating the National Statistics System ...................................... 10 Table 3.2 Principle 2: Managing Relationships with Data Users, Data Providers and Other Stakeholders ......................................................................................................................... 13 Table 3.3 Principle 3: Managing Statistical Standards ............................................................ 17 Table 3.4 Principle 4: Assuring Professional Independence .................................................. 20 Table 3.5 Principle 5: Assuring Impartiality and Objectivity ................................................ 23 Table 3.6 Principle 6: Assuring Transparency ......................................................................... 26 Table 3.7 Principle 7: Assuring Statistical Confidentiality and Data Security .................... 28 Table 3.8 Principle 8: Assuring Commitment to Quality ...................................................... 31 Table 3.9 Principle 9: Assuring adequacy of resources .......................................................... 37 Table 3.10 Principle 10: Assuring methodological soundness ............................................ 39 Table 3.11 Principle 11: Assuring cost-effectiveness ............................................................. 43 Table 3.12 Principle 12: Assuring appropriate statistical procedures ................................ 47 Table 3.13 Principle 13: Managing the Respondent Burden ................................................ 51 Table 3.14 Principle 14: Assuring Relevance ......................................................................... 54 Table 3.15 Principle 15: Assuring Accuracy and Reliability ................................................ 57 Table 3.16 Principle 16: Assuring timeliness and punctuality ............................................ 60 Table 3.17 Principle 17: Assuring accessibility and clarity .................................................. 63 Table 3.18 Principle 18: Assuring coherence and comparability ........................................ 67 Table 3.19 Principle 19: Managing metadata ......................................................................... 70
vi FOREWORD In today's data-driven world, the quality of information we rely upon is paramount to the success of our endeavors. As we navigate through the complexities of modern business landscapes, ensuring the accuracy, reliability, and integrity of our data becomes not just a priority but a strategic imperative. It is with great pleasure that Tanzania National Statistics Offices i.e the National Bureau of Statistics (NBS) and Office of the Chief Government Statistician (OCGS) introduce Data Quality Assurance Framework for the United Republic of Tanzania. This framework represents a significant milestone in our journey towards harnessing the full potential of Tanzania's data assets. It is a testament to our unwavering commitment to excellence and our recognition of the pivotal role that high-quality data plays in driving informed decision-making, fostering innovation, and achieving our strategic objectives. At its core, this framework embodies a set of principles, processes, and practices designed to uphold the highest standards of data quality throughout its lifecycle. It is the result of extensive collaboration, drawing upon the collective expertise and insights of individuals across the National Statistical System (NSS). From data stewards and analysts to Information Technology professionals and business leaders, each contribution has been instrumental in shaping this comprehensive approach to data quality assurance. Quality Assurance Framework for official statistics is a dynamic and adaptive system that evolves in response to changing business needs, technological advancements, and emerging best practices. It serves as a cornerstone for fostering a culture of data excellence, where every member of NSS is empowered to take ownership of data quality and contribute to its continuous improvement. As we embark on this journey, we invite each and every one to embrace the principles embodied in this framework and to play an active role in championing data quality within their respective domains. Together, let us leverage the power of high-quality data to drive innovation, enhance operational efficiency, and deliver exceptional value to our customers and stakeholders.
vii NBS and OCGS would like to express sincerely appreciation to all stakeholders who have contributed to the development of the Tanzania Data Quality Assurance Framework (TDQAF). Their dedication, expertise, and commitment to ensure the integrity and reliability of this framework have been invaluable. Special thanks are extended to the Government of the United Republic of Tanzania for committing resources, leadership and guidance throughout this process and World Bank for their technical and financial support. We also extend our gratitude to all staff members who have participated in developing quality assurance framework. We appreciate the contribution made by technical experts from Office for National Statistics of United Kingdom(ONS-UK) for reviewing this framework and provided valuable inputs and recommendations.
Dr. Albina Chuwa Statistician General National Bureau of Statistics
Salum Kassim Ali Chief Government Statistician Office of The Chief Government Statistician, Zanzibar
viii
ABBREVIATIONS AND ACRONYMS
API
Application Programming Interface
CAPI
Computer Assisted Personal Interviewing
COFOG
Classification of the Functions of Government
COICOP
Classification of Individual Consumption by Purpose
CPC
Central Product Classification
CPI
Consumer Price Index
CSPA
Common Statistical Production Architecture
DQAF
Data Quality Assurance Framework
EAC
East African Community
ESS
European Statistical System
FYDP
Five Year Development Plan
GAMSO
Generic Activity Model for Statistical Organizations
GDDS
General Data Dissemination System
GDP
Gross Domestic Product
GSBPM
Generic Statistics Business Process Model
HRP
Human Resource Planning
HS
Harmonized System
ICT
Information and Communication Technology
IMF
International Monetary Fund
ISCO
International Standards Classification of Occupations
ISIC
International Standard Industrial Classification
ISO
International Organization for Standardization
IT
Information Technology
IQMT
Internal Quality Management Team
LGAs
Local Government Authorities
MDAs
Ministries, Departments and Agencies
MOU
Memorandum of Understanding
NBS
National Bureau of Statistics
NGO
Non- Government Organization
NQAF
National Quality Assurance Framework
NSAs
Non-State Actors
NSDS
National Strategic Development Statistics
ix
NSO
National Statistics Office
NSS
National Statistical System
OCGS
Office of the Chief Government Statistician
ONS-UK
Office for National Statistics-United Kingdom
PE
Personal Emoluments
PISCs
Public Institutions and Statutory Corporations
QSTWG
Quality Statistics Technical Working Group
RISDP
Regional Indicative Strategic Development Plan
ROSCs
Reports on the Observance of Standards and Codes
SADC
Southern Africa Development Community
SDDS
Special Data Dissemination Standard
SDGs
Sustainable Development Goals
SDMX
Statistical Data and Metadata eXchange
SNA
System of National Accounts
SSPS
Sector Strategic Plan for Statistics
SWG
Statistical Working Group
SWOC
Strengths, Weaknesses, Opportunities and Challenges
TDQAF
Tanzania Data Quality Assurance Framework
TDV
Tanzania Development Vision
TNADA
Tanzania National Data Archive
TQM
Total Quality Management
TSMP
Tanzania Statistical Master Plan
UN
United Nations
UN-NQAF
United Nations National Quality Assurance Framework
URT
United Republic of Tanzania
WB
World Bank
ZADEP
Zanzibar Development Plan
x DEFINITIONS OF THE TERMS Data Ecosystem: The entire network of data collectors, producers, analysts, and users in both state and non-state actors in Tanzania.
National Statistical System (NSS): The group of statistical organizations and units (statistical agencies) within a country that develop, produce, and disseminate official statistics on behalf of the Tanzania Government.
National Statistics Offices (NSOs): These are the National Bureau of Statistics of the United Republic of Tanzania and the Office of the Chief Government Statistician of Revolutionary Government of Zanzibar. NSO has a coordination role within the national statistical system and they are responsible for the development, production and dissemination of official statistics across multiple statistical domains.
Official Statistics: Statistics that are produced, validated, compiled and disseminated by the National Statistical System in accordance with the United Nations Fundamental Principles of Official Statistics or compiled by Statistical Units for their specific domain and certified by NBS or OCGS.
Quality: The degree to which a set of inherent characteristics of an object fulfils requirements. In the context of statistical organizations, the object is the statistical output or product, the process, the institutional environment or the statistical system.
Quality Assurance: Set of planned and systematic activities for ensuring quality in the processes by which products are developed, which starts from the designing stage, going through the implementation stage up to the post-collection stage to create products and services that meet the needs, expectations and requirements of customers.
Quality Assessment: the part of quality assurance that focuses on an assessment of how well quality requirements are fulfilled
Statistical agencies: These encompass members of the NSS, encompassing the NBS, OCGS, and other producers of official statistics. Statistical agencies other than the NBS
xi normally have other main purposes and tasks than the production of official statistics and only a section or a small group of people within the institution produces statistics.
Statistical Unit: An organizational entity within a government ministry, department, or agency, other than the NSO that develops, produces and disseminates statistics.
UN-NQAF: Checklist which sets out dimensions and elements under which quality of statistics can be assessed. The checklist identified 19 quality principles which cascades down to 87 requirements and down further to 257 elements.
1 CHAPTER ONE BACKGROUND INFORMATION 1.0 Background In recent years, the emerging of new data sources, widen the range of data providers and statistics producers, fueled by technological advances and new demands for detailed and timely data for policy and decision-making in the context of the 2030 Agenda for Sustainable Development and National Development Agenda. Many National Statistics Offices (NSOs) around the world have adopted national codes of practice and/or statistical quality assurance frameworks to capture the principles and best practices, and to safeguard public trust on statistics they produce. These principles and frameworks target national statistical organizations to meet the needs of users including that of the government. Statistics must be of good quality to ensure that it is complying with all the required dimensions of quality in order to be generally accepted as fit for purpose. The quality of statistical information covers a number of different dimensions including aspects such as relevance; accuracy; integrity; methodological soundness; timeliness; accessibility; and serviceability. These dimensions are overlapping and inter-dependent and the relative importance placed on any one of them depends on who is using the statistic and what it is to be used for. Moreover, producers of statistics have been producing statistics that are sometimes conflicting. The situation has over the years brought difficulties for planners and decision makers in deciding which dataset to use as evidence-base. Furthermore, the increase in demand for both traditional and non-traditional data for policy and development agenda influenced the Government of Tanzania to review the mandate of NBS and OCGS to produce relevant, timely and reliable user-driven official statistics that facilitate informed decision-making. The Government of Tanzania mandated the NBS and OCGS to coordinate the NSS and make better provision in relation to their functions and related matters.
The NBS, OCGS and other stakeholders of the National Statistical System (NSS) prepare the guideline for assuring the quality of data in the country. This document sets out a framework for assuring and assessing the quality of official Statistics that are produced by the NSS. The development of Tanzania Data Quality Assurance Framework adopted