TTThhheee UUUnnniiittteeeddd RRReeepppuuubbbllliiiccc ooofff TTTaaannnzzzaaannniiiaaa
DDDaaatttaaa QQQuuuaaallliiitttyyy AAAsssssseeessssssmmmeeennnttt FFFrrraaammmeeewwwooorrrkkk
fffooorrr TTTaaannnzzzaaannniiiaaa MMMaaaiiinnnlllaaannnddd
NNNaaatttiiiooonnnaaalll BBBuuurrreeeaaauuu ooofff SSStttaaatttiiissstttiiicccsss
MMMiiinnniiissstttrrryyy ooofff FFFiiinnnaaannnccceee
DDDaaarrr eeesss SSSaaalllaaaaaammm
JJJaaannnuuuaaarrryyy,,, 222000111444
National Bureau of Statistics
VISION “To become a one-stop centre for official statistics in Tanzania”
MISSION “To produce quality official statistics and services that meet needs of natio nal and international stakeholders for evidence-based planning and decision making”
The United Republic of Tanzania
Data Quality Assessment Framework for Tanzania Mainland
National Bureau of Statistics Ministry of Finance Dar es Salaam January, 2014
Abbreviations and Acronyms CDOS Concepts and Definitions for Official Statistics in Tanzania CoP European Statistics Code of Practice (Eurostat) GSBPM Generic Statistics Business process Model HR Human Resources ICT Information and Communications Technology MTEF Medium Term Expenditure Framework NBS National Bureau of Statistics NSS National Statistical System OCGS Office of the Chief Government Statistician QGSP Quality Guidelines for Statistical Production in Tanzania SMSG Statistical Methods, Standards and Guidelines TSMP Tanzania Statistical Master Plan GSBPM Generic Statistical Business Process Model STATCAP Statistical Capacity Building
Contents Abbreviations and Acronyms .................................................................................................... 1 Introduction ................................................................................................................................ 1 Links between the Framework and Other Policy Documents ................................................... 2 How the Data Quality Assessment Framework was Developed ............................................... 3 Using the Framework ................................................................................................................. 5 Part 1: The National Statistical System ..................................................................................... 6 Part 2: Agencies responsible for the production and dissemination of official statistics .......... 9 Part 3: Programme design (GSBPM phases 1 to 3) ................................................................. 27 Part 4: Programme implementation (GSBPM Phases 4 to 7) .................................................. 35 Part 5: Post collection and evaluation (GSBPM Phases 8 and 9) ............................................ 44 Attachment 1: Overview of the Elements of the UN Generic National Quality Assessment Framework ....................................................................................................... 48 Attachment 2: The Generic Statistical Business Process Model (GSBPM) ............................ 49
1
Introduction
This document sets out a framework for assessing the quality of official statistics that are
produced by the National Statistical System (NSS) of Tanzania. It is based on a Generic
National Quality Assurance Framework (NQAF) that was produced under the auspices of the
United Nations Statistical Commission and which was approved in 2012. This framework has
been used because it is aligned with other in ternational quality frameworks such as the IMF’s
Data Quality Assessment Framework (DQAF) and the European Statistics Code of Practice
(CoP). It is also based on widely accepted definitions of quality as applied to official statistics
as well as the Fundamental Principles of Official Statistic s, that were adopted by the United
Nations Statistical Commission in 1994 and which were formally endorsed by the UN
Economic and Social Council in 2013.
There is a general consensus that the quality of statistical information covers a number of
different dimensions including aspects such as: relevance; accurac y; integrity;
methodological soundness; timeliness; accessibility; and serviceability. These dimensions are
overlapping and inter -dependent and the relative importance placed on any one of them
depends on who is using the statistic and what i t is to be used for. The Data Quality
Assessment Framework includes aspects of all the dimensions listed above and is designed to
be comprehensive. It will allow users of official statistics in Tanzania to understand how any
particular statistic has been produced, what processes and mechanisms have been applied to
ensure that it is fit for a specific purpose and what controls have been put in place to give the
user confidence that it is as objective, accu rate and reliable as possible. The Framework
describes different standards and guidelines that have been applied to the organisation and
management of the agency that has produced the statistics as well as the measures that have
been put in place to ensure the quality of the production process.
Because data quality of official statistics depends on all aspects of the data production process as well as the organisation and management of statistical agencies and the National Statistical System as a whole, th is Framework is comprehensive. It covers all levels of the National Statistical System , that is, from the governance of the system as a whole, th rough the management of the data producing agencies, to the controls applied to the collection of data, compilation of s tatistics, dissemination of different statistical products and the mechanisms used for archiving and evaluation.
Need for a Data Quality Assessment Framework Official statistics that are produced and disseminated by any national statistical agency are used for many different purposes. It is important to ensure that users of the statistics are not at a disadvantage, compared with the producers. Just by looking at the statistic itself most users 2
do not have enough information to determine whether or not it i s fit for any particular purpose. Without knowing how the statistic was compiled, what data sources were used, how they were collected, what concept s and classifications were applied and how the statistics were calculated, they are unable to say whether it can be used with confidence. In particular, in order to trust the statistics they need to be assured that proper scientific methods were used and that the published numbers are objective, reliable and timely. They also need to have confidence that the published statistics have not been manipulated in any way. The Data Quality Assessment Framework is designed to provide the information users need to make these judgements.
This Framework sets out how and under what conditions , data are collected, statistics are calculated and report s are made available to users. This publication provides users with a detailed description of how the statistical agency is managed, the rules that it operates under, the methods it uses and the procedures it applies to ensure th at official s tatistics are of good quality. It also describes to what extent statistical practices in Tanzania are in line with international recommendations and good practice. It provides a general description that covers all official statistics and all statistical operations. More detailed information describing the processes and procedures used to produce particular sets of statistics and different time series of data – the metadata – is also published and users are invited and encouraged to consult this information to answer more specific questions.
Links between the Framework and Other Policy Documents
The Data Quality Assessment Framework is part of the overall governance of the National
Statistical System of Tanzania and should be read in conjunction with other important
national statements of statistical policy that govern and direct its activities. In particular it is
linked with and builds on a number of other documents including the following:
• The Statistics Act 2002 and the proposed new Statistics Legislation
• The Tanzania Statistical Master Plan TSMP 2009/10 – 2013/14
• Concepts and Definitions for Official Statistics in Tanzania (CDOS), Second Edition,
June 2011
• Statistical Methods, Standards and Guidelines (SMSG), February 2012
• A Handbook of Quality Guidelines for Statistical Production in Tanzania (QGSP),
November 2012
Other frameworks and processes that have an important link to quality assurance and which
are referred to in the Framework include the annual budgets, work plans and reports of the
National Bureau of Statistics Human Resource Management Strategies, and policies
governing areas such as the procurement and use of Information and Communications
Technology (ICT).
3
How the Data Quality Assessment Framework was Developed
The Data Quality A ssessment Framework has been developed from two important
international generic models for statistical activities. The first is the Generic National Quality
Assessment Framework (NQAF) prepared by an expert team and put together by the UN
Statistics Division. The NQAF consists of 342 quality elements or statements that have an
impact on the quality of official statistics. The elements are organised into 19 themes or
topics that reflect different aspects of the statistical production process. Within each theme ,
the elements are categorised as operating at five different levels as follows:
• Actions and activities that apply at the level of the National Statistical System as a
whole. These are mainly concerned with the coordination of statistical activities
between different producing agencies.
• Actions and activities that concern the overall management of the agencies that are
responsible for the collection, compilation and dissemination of official statistics .
These are the activities that generally ensure the integrity of statistics.
• The design of specific individual statistical programmes or activities , covering
activities such as identifying the need for different statistics, the design of the
statistical programmes and the development of data collection instr uments such as
questionnaires and manuals.
• The implementation of individual statistical programmes or activities, covering aspects
such as data collection, processing, analysis and dissemination.
• Activities within individual statistical programmes or activities at the post collection
stage, that is, those mainly concerned with archiving and evaluation.
The second generic model or framework that has been used in Tanzania is the Generic Statistical B usiness Process Model (GSBPM). This model, which has been developed and approved by the UN describes all the different steps or phases that are involved in producing official statistics. It is generic in that it applies to different kinds of statistical process es, although not all phase s are relevant in all cases . The model includes nine separate phases, each broken down into a number of steps. Phases 1 to 3 are concerned with the design of statistical programmes, phases 4 to 7 cover implementation and phases 8 and 9 involve archiving and evaluation. More informat ion about NQAF is provided in Attachment 1 and about GSBPM in Attachment 2.
Both the NQAF and GSBPM are comprehensive covering all the main issues affecting official statistics and all stages of the statistical process. As a result, the generic framework is quite lengthy and cumbersome to use ; it also includes a sub stantial amount of repetition. For example, a number of the same quality concerns apply to more than one theme or topic. 4
In order to simplify the generic framework and to make it more applicable to Tanzania, the different quality concerns have been reorganised and are now listed simply by the level they operate at. For example, measures to ensure that statistical activities are designed and carried out without any external interference apply at t he level of the agency and, to some extent, the whole of the National Statistical System. Other concerns, for example, ensuring that different statistical processes use agreed standard classifications , operate at the level of individual programmes and activities.
The Tanzania Data Quality Assessment Framework, therefore, is divided into five main
levels as follows:
• Part 1: The operation of the National Statistical System as a whole, focusing mainly on
issues of coordination
• Part 2: The management of agenci es responsible for the collection, compilation and
dissemination of official statistics and the application of controls to ensure the integrity
of statistics and other policies that apply to all their activities
• Part 3: The design of specific individual st atistical programmes or activities, covering
aspects such as the specification of needs, the design of the process and the
development of the data collection instruments
• Part 4: The implementation of individual statistical programmes or activities, coverin g
data collection, data processing, data analysis and the dissemination of the various final
products
• Part 5: Activities within individual statistical programmes or activities at the post
collection stage, focusing especially on archiving and evaluation.
Within each of these five parts, a number of elements are listed, covering different aspects
of quality assurance, as follows.
• A brief description of the quality element;
• The NQAF quality element on which it is based;
• The document, guideline or process that sets out the standards that are required to be in
place or be observed;
• Notes setting out how the element is assured, or what else needs to be done to put this
in place.
Each part of the Framework, with the exception of the first, is divided into a num ber of sections that set out the main areas covered. These are related to the 19 topics listed in the NQAF, but with some modifications to fit the Tanzanian situation. A number of sections are repeated across all five parts, while some are specific to that particular level only. Overall there are 269 elements allocated across a total of 49 sections as indicated in Table 1.
5
Table 1: Design of the draft data quality assessment framework for Tanzania Description of each part Number of Sections Number of Elements 1 The National Statistical System 1 9 2 Agencies responsible for the production and dissemination of official statistics 17 125 3 Programme design ( GSBPM phases 1 to 3) 13 52 4 Programme implementation (GSBPM Phases 4 to 7) 10 61 5 Post collection and evaluation (GSBPM Phases 8 and 9) 8 22 Total 49 269
Using the Framework Both the providers and users of official statistics will use the Framework in a number of different ways. For the producers of statistics, especially managers and staff responsible for different statistical processes, the Framework provides a means of assessing the quality of the different processes and products they are responsible for and identifying where improvements and changes are needed for harmonisation of data production process and systems. The Framework will be important in developing medium-term strategic plans as well as annual work programmes. It can also help identify priority areas where improvements and investments in quality are needed.
At the level of the National Statistical System, the Framework has an important role in building trust in official statistics and in strengthening coordination. Producers of statistics will apply the framework to ensure that their products are recognised as meeting the requirements of official statistics. This process will be formalised as and when proposed new Statistics Act becomes law.
This is a public document to assist the users of official statistics, to check that the quality standards of the agencies and processes are being met. Over time it will also be possible for users to make statistical agencies aware of areas where quality needs to be improved.
It is also important to note that the Framework will be subject to revision at least every three
years, and more frequently as and when major changes are made. For example, when a new
classification or standard is introduced, the Framework will need to be updated to reflect this.
Changes will also be needed to reflect the process of continuous quality improvement,
through investment in capacity, technological change and improvements in the efficiency and
effectiveness of agencies.