45
Requirement
Element to be assured
Description
production systems.
solutions that increase effectiveness and efficiency.
Integrating Administrative Data in Tanzania
NSS
The statistical business architecture of the statistical agency is based on
international standards and tools such as the GSBPM, the Generic
Activity Model for Statistical Organizations (GAMSO), the Common
Statistical Production Architecture (CSPA), and the Statistical Data and
Metadata eXchange (SDMX).
46 3.3.3 Principle 12: Assuring appropriate statistical procedures Principle 12 underscores the significance of implementing effective and efficient statistical procedures across the entire statistical production chain. These procedures form the foundation of quality in statistical outputs. This principle emphasizes the importance of methodological rigor and adherence to established procedures to ensure the reliability and accuracy of statistical data at every stage of production. This principle has five quality requirements explained in Table 3.12.
47 Table 3.12 Principle 12: Assuring appropriate statistical procedures Requirement Element to be assured Description 12.1 Statistical processes are tested before implementation.
The testing strategy is developed as part of the design phase of the statistical business process model Pre-tests and pilots need to be conducted for each statistical activity and results be documented. Pilot reports serve as evidence for the practice Data capture procedures and data collection tools and instruments such as electronic systems are tested to ensure simplicity and minimal intrusion on privacy, and are adjusted if required before their implementation Survey questionnaires are tested using appropriate methods (e.g., pilot survey, focus groups, etc.). Collection systems for administrative and other data are tested before use Data treatment and data processing procedures are tested and adjusted, if required and possible, prior to their actual application Test results are taken into account in the implementation of the production process and are approved In the case of integrating data from one or more sources, the quality of the linkage procedures is tested. 12.2 Statistical processes are well established and regularly monitored and revised as required
The statistical agencies have documented procedures and guidelines that contain recommendations for appropriate methodologies to be used at different steps of the statistical production process Recommendations from pre-test and pilot results as well as evaluation and quality assessment reports are useful inputs for revision of such processes. Documentation reports of changes serve as the evidence example Rebasing reports of National Accounts Statistics and CPI Documentation of production processes should follow the GSBPM A policy for archiving data and statistics is in place and is followed Statistical procedures employ internationally recognized statistical techniques Data from all data sources are reviewed and validated to identify potential problems, errors and discrepancies such as outliers, missing data and miscoding
48 Requirement Element to be assured Description When coding is done through an automated process, a team of well-trained coders is assigned to verify the automated coding and to handle un-coded cases
The effects of data editing and imputation are analyzed as part of assessing the quality of the data collection All statistical databases are designed and arranged in a way that allows and facilitates data linkage, using unique identifiers for statistical units as appropriate while ensuring data security and privacy
12.3
Procedures are in place to
effectively
use
administrative
and
other
data sources for statistical
purposes
.
Statistical agencies use tools and guidelines to assess the
quality of the data of administrative and other data
sources
Guidelines and manuals on managing administrative
and other data sources need to be in place. The presence
of metadata documentation and interventions on data
utilization workshops in the Communication Strategy as
evidence
Appropriate processes and so ware applications for the
collection,
processing
and
analyses
of
data
of
administrative and other data sources have been
developed and implemented
Owners or holders of administrative and other data
sources inform the statistical agencies of any changes in
the data production process
Metadata related to administrative or other data sources
are available to the statistical agencies, including
concepts
and
definitions,
classifications,
coverage
compared to target population and other quality aspects
Documentation exists that describes how data from
administrative and other sources meet the statistical
requirements in terms of definitions, concepts and
coverage, among other things.
12.4
Revisions of statistics follow
standard and transparent
A revision policy that follows international standards
and recommendations exists and is made public
Involvement of key stakeholders and documentation of
the revision process need to be done. Revision reports
and stakeholder consultation reports serve as evidence
Guidelines for revisions exist and are followed
Revisions of the published statistics are accompanied by
49 Requirement Element to be assured Description procedures
metadata that provide necessary explanations
Indicators expressing the amount and types of revisions
are computed and evaluated for improvement
12.5
Metadata
and
documentation of methods
and
different
statistical
processes
are
managed
throughout the processes
and shared as appropriate.
There is a policy on metadata documentation linked to
the statistical production processes
There should be standard methodology/ tools for
documentation
of
metadata
and
methods.
These
methodologies and tools should be made available to
users for their reference. Data dissemination platforms
such as TNADA, TISP serve as evidence
The policies and standards for maintaining and updating
metadata are followed.
Work on preparing statistics and their related metadata
should be done in parallel
Metadata
are
captured
throughout
the
statistical
business process following the GSBPM and stored in a
metadata management system
Statistical methods and processes are documented in
such a way that allows for the recreation of the entire
statistical production process
50 3.3.4 Principle 13: Managing the Respondent Burden Individuals, households or businesses that provide the data upon which statistical products are based are fundamental contributors to the quality of data and information. The requirement to collect data should be balanced against production costs and the burden placed on respondents. Mechanisms to maintain good relationships with providers of data and to proactively manage the respondent burden are essential to improving quality. This Principle has four quality requirements which are elaborated in Table 3.13.
51 Table 3.13 Principle 13: Managing the Respondent Burden Requirement
Element to be assured
Description
13.1
The
range
and
detail
of
requested information is limited
to what is necessary.
Availability
and
suitability
of
existing
surveys
and
administrative or other data sources are explicitly considered
before suggesting a new survey.
Questionnaires are designed in an optimum
level that captures necessary required
information from respondents
Before establishing a new survey, it is examined whether the
required data can be produced with less respondent burden
by modifying or amending an existing survey or by linking
the new survey with an existing survey (integrated survey
system).
The collection of each data item of a survey has to be
explained and justified.
The collection of any data items that are identical or similar to
those collected in another surveys is limited to what is
considered necessary for verification and possible data
linkage purposes.
When possible, surveys or parts of the information to be
collected in the surveys are extracted or derived from
available administrative registers.
The burden on respondents is measured and included in a set
of quality indicators and in the quality reports.
13.2
Mechanisms are in place to
promote the value and use of
statistics to respondents.
Information
packages
that
provide
respondents
with
important and necessary information about the survey and
that explain the value of official statistics are made available.
Presence of consent statement that state the
importance and objectives of the survey;
and respondents' obligation to participate in
the survey
Respondents are provided with the final reports or result of
the census or sample survey in which they participated.
Initiatives
with
community
groups,
schools,
business
advocates and others are undertaken to raise awareness of the
value of official statistics.
Electronic products are developed that give necessary
statistical information to businesses and individuals, and
these products are promoted through initiatives with
52 Requirement
Element to be assured
Description
communities and respondents.
Social media is being used to promote participation in surveys
and censuses.
There are standard practices to obtain feedback from
respondents and to respond to their requests and complaints
in a regular manner.
13.3
Sound methods including IT
solutions are used in surveys to
reduce or distribute respondent
burden.
Appropriate sampling techniques are used to minimize
sample sizes to achieve the target level of accuracy.
The introduction of CAPI in data collection
reduces
the
respondent's
burden
considerably in terms of time taking and
asking an inappropriate question
Sample surveys are coordinated to distribute the burden on
respondents.
Multiple modes of collection are offered to respondents,
including electronic surveys.
Collection of data is done at the most appropriate time of the
day and the year.
13.4
Data sharing, data linkage and
use of administrative and other
data sources are promoted to
minimize respondent burden.
Documentation of data already available within the NSS,
including archived data, exists and is shared.
Strategies and intervention on systems
integration and the use of non-traditional
data sources (including big data) are part of
the priority areas within TSMP II. This
serves as evidence of effort on minimizing
the respondent's burden
Procedures and technical tools for data sharing and data
linkage within the NSS (e.g. formal agreements, web services,
common databases) exist.
Data repositories are shared between statistical agencies for
production of official statistics and in compliance with
confidentiality policies.
Information of the quality of data to be linked exists (e.g. on
coverage and linkage possibilities).
Use of administrative and other data as an alternative to
survey data for producing official statistics is promoted
throughout the NSS
53
3.4
Level D: Managing Statistical Outputs
Statistics serve the needs of national governments, research institutions, businesses, the
general public and the international community. The output quality is measured by the
extent to which the statistics are relevant, accurate and reliable, timely and punctual,
readily accessible by and clear to users, and coherent and comparable across geographical
regions and over time. The quality principles in managing statistical outputs are:
a) Assuring relevance
b) Assuring accuracy and reliability
c) Assuring timeliness and punctuality
d) Assuring accessibility and clarity
e) Assuring coherence and comparability
f) Managing metadata
3.4.1 Principle 14: Assuring Relevance
Statistical information should meet the current and/or emerging needs or requirements of
its users. Without relevance, there is no quality. However, relevance is subjective and
depends upon the varying needs of users. The challenge is to weigh and balance the
conflicting needs of current and potential users to produce statistics that satisfy the most
important and highest-priority needs within the given resource constraints. This Principle
is explained by four requirements as elaborated in Table 3.14.
54
Table 3.14
Principle 14: Assuring Relevance
Requirement
Element to be assured
Description
14.1
Procedures are in place to
identify
users
and
their
needs and to consult them
about the content of the
statistical work program.
There is the legislation or some other formal provision
which includes an obligation to consult with the main
users of the statistics.
There should be a register of users’ groups and a
regular consultation program with them. Establishment
and operationalization of Sector Working Group and
presence of Communication Strategy can serve for such
purpose
Structured and periodic consultation processes (e.g.,
advisory council and committees or working groups)
with key stakeholders and users are in place to review
the content of the statistical program and the
usefulness of existing statistics, and to identify
requirements for new statistics.
Feedback from user support service, center or hotline is
analysed to understand and identify user needs.
Data on the use of statistics (for example web analytics,
number and types of downloads, subscribers of
reports) are collected and analysed to improve
statistical outputs.
14.2
Users’
needs
and
requirements are balanced,
prioritized and reflected in
the work program.
Users’ priority needs are met and reflected in the work
program of the statistical agency.
Recommendations from users’ consultations need to be
taken into consideration during design of statistical
work program. User consultation minutes during
planning and designing of statistical program serve as
evidence
Procedures are in place to prioritize between different
users’ needs in the work program and strategic goals.
The data on the use of statistics are analysed to support
priority setting.
A periodic evaluation of the work program is carried
out to identify emerging needs and lower priorities.
There are processes in place to monitor and consult
with stakeholders the relevance and practical utility of
existing statistics (with respect to scope, level of detail,
cost, etc.) according to current and emerging user
needs.
14.3
Statistics based on new and
existing data sources are
being developed in response
An innovation laboratory to consider and experiment
with new data sources to meet emerging information
needs is established.
There should be a strategy and guidelines of using
different sources for statistics production. TSMP II