35 Requirement Element to be assured Description 8.8 Risk analyses addressing the quality of important statistical products and processes are performed. Risk and quality management are closely coordinated (e.g. by institutional arrangements and regular meetings if responsibilities for these activities are placed differently) Quality assessment report should be presented and discussed at length to wider range of stakeholders Risks linked to core recommendations and principles of the NQAF (e.g. for lack of independence and confidentiality breaches) are analysed and measures taken if needed to improve compliance Risk analyses addressing the quality of different stages of the statistical production are conducted according to the GSBPM. Risk analyses addressing the quality of important statistical products such as population statistics and censuses, national accounts and Consumer Price Index (CPI) are performed (e.g. risk of poor accuracy expressed by errors, poor timeliness and lack of comparability)
36 3.2.6 Principle 9: Assuring adequacy of resources The financial, human, and technological resources available to NSOs and Statistical Units should be adequate both in magnitude and quality, and sufficient to meet the needs regarding the development, production and dissemination of statistics. This principle is explained by three requirements elaborated in Table 3.9.
37 Table 3.9 Principle 9: Assuring adequacy of resources Requirement Element to be assured Description 9.1 Financial, human and technological resources are sufficient to implement the statistical work and development program. A resource mobilization strategy such as a NSDS is in place. NSOs should have resource management and mobilization policies or strategies that guide on mobilizing and use of such resources. The evidence documentation includes Human Resource Planning (HRP) and Personal Emoluments (PE) The annual work plan is feasible given the available resources. Costs (staff costs and other costs) of each stage of the production processed are measured. 9.2 Planning and management principles are aimed at the optimal use of available resources. Information technology is employed to increase efficiency. Cost and benefit analysis should be done to optimize the use of resources for achieving greater results Standardization, integration and automatization of statistical production and dissemination are pursued to increase efficiency of operations and to save costs. 9.3 The statistical agencies’ use of resources is reviewed. Indicators on the use of human and financial resources are monitored centrally and regularly reported to management. There should be regular reviews of the budget and implementation of work plans, for example mid- term reviews The use of human resources is evaluated annually based on established guidelines and procedures. The evaluation covers allocation, performance and training needs of staff. Staff opinion/satisfaction surveys are conducted regularly.
38
3.3
Level C: Managing Statistical Processes
International standards, guidelines and good practices are fully observed in the statistical
processes used by the NSOs and Statistical Units to develop, produce and disseminate
official statistics, while constantly striving for innovation. The credibility of the statistics is
enhanced by a reputation for good management and efficiency. The relevant principles to
be assured under this level focused on balancing between quality of statistics, cost and
respondent burden. Managing statistical process has four principles:
a) Assuring methodological soundness
b) Assuring cost-effectiveness
c) Assuring appropriate statistical procedures
d) Managing the respondent burden
3.3.1 Principle 10: Assuring methodological soundness
In developing and compiling statistics, NSOs and Statistical Units should use sound
statistical methodologies based on internationally agreed standards, guidelines or best
practices and consistent with established scientific principles. Effective and efficient
statistical procedures should be implemented throughout the statistical production chain.
The following five quality requirements are recommended for assuring methodological
soundness, see Table 3.10.
39
Table 3.10
Principle 10: Assuring methodological soundness
Requirement
Element to be assured
Description
10.1
The methodologies applied by the
statistical agencies are consistent
with
international
standards,
guidelines and good practices and
are regularly reviewed and revised
as needed.
Organizational
structures
for
the
development
and
application of sound statistical methods are commensurate
to the needs.
NSOs should have manuals which guides on the
processes of statistics production and these
guidelines
need
to
be
harmonized
with
international standards. For example, statistical
classifications (ISIC Rev. 4, SNA 2008, COICOP
etc.)
There are review and reporting processes in place that allow
the management of the statistical agency to be assured that
sound methodological approaches have been adopted and
applied throughout the production process.
The methodologies of surveys and the use of administrative
data and other sources of data are evaluated periodically.
Sampling design is based on sound methodology
Proper follow-up procedures are planned and implemented
in the case of non-response.
Statistical editing procedures and imputation methods are
based on sound methodology.
When statistical modelling is used in the statistical
production process (e.g. for seasonal adjustment), the
validity of model assumptions is carefully considered and
the impact on final estimates evaluated.
Statistical agencies review the methods used by external
partners for the compilation of data and the production of
statistics.
10.2
The
statistical
agencies
recruit
qualified staff and have regular
programs
to
enhance
their
methodological skills.
Staff of the statistical agency are recruited based on their
academic background, qualifications and experience.
Recruitment of statistical staff should consider
professionalism and there must be a program to
train these staff regularly. Supporting evidence
includes the Scheme of Services; Training Needs
Assessment and Training Plan
Appropriate qualifications requirements are specified for all
posts.
Training and development programs are in place to ensure
the
staff
acquires
and
continuously
update
their
methodological knowledge.
Staff skills are regularly updated so that staff is able to
utilize new data sources and tools, and able to easily change
positions.
40
Requirement
Element to be assured
Description
Attendance of staff at relevant training courses and/or to
national or international conferences is encouraged.
10.3
Statistical agencies are to choose the
data source with regard to accuracy
and reliability, timeliness, costs, the
burden on respondents and other
necessary considerations.
The use of alternative sources of data, including existing
surveys and census, administrative data, “big data” or other
sources of data is constantly evaluated.
Data collection should be planned only when,
and for items which are, necessary at a certain
time. Choice of data source should consider all
data
quality
dimensions
including
cost-
effectiveness
Quality has to be assessed when using administrative data
or other data sources. Ideally, when using administrative
data, it should be assured that:
•
The population is consistent with the statistical
output requirements;
•
The classifications are appropriate;
•
The underlying concepts are appropriate;
•
The records are complete and up to date;
•
The geographical coverage is complete and the
measurement
units
are
appropriately
defined/identified.
When using other data sources (such as big data), the
specific methodological challenges such as the ones linked
to the statistical population and the veracity and volatility of
such data have to be considered.
10.4
The registers and the frames for
surveys are frequently evaluated
and adjusted.
A systematic approach is in place for updating the survey
frames to ensure accurate coverage of the target population.
NSOs must maintain and update sampling
frames of households and establishments
For all surveys the appropriate statistical population frames
are updated regularly.
Information gathered during the conduct of surveys is used to assess and improve the quality of the frame, especially its coverage and the quality of the contact variables and the auxiliary information (variables used in the sampling design). 10.5 The statistical agencies cooperate with the scientific community to improve methods and promote Collaboration with the scientific community is in place, for example through conferences, workshops, task forces, and training/courses, to discuss relevant methodological and There should be a formal collaboration agreement between NSOs, R&D and the scientific community
41 Requirement Element to be assured Description innovation in the development, production and dissemination of statistics
technological developments, e.g. in exploiting new data sources. There are agreements with academic institutions on cooperation and the exchange of qualified personnel. Staff collaborate on methodological issues with colleagues at international level. Regular participation and presentations at relevant national and international conferences is encouraged for exchange of knowledge and experiences. National and international conferences, seminars, workshops, or similar events with the participation of the scientific community are organized by the statistical agencies.
42 3.3.2 Principle 11: Assuring cost-effectiveness Principle 11 emphasizes the responsibility of NSOs and Statistical Units to ensure the effective and efficient utilization of resources. It is imperative for NSOs and Statistical Units to demonstrate the extent to which objectives have been met, the attainment of results at a reasonable cost, and the alignment of outcomes with the primary purposes of the statistics. This principle is managed by six quality requirements which are elaborated in Table 3.11.
43
Table 3.11
Principle 11: Assuring cost-effectiveness
Requirement
Element to be assured
Description
11.1
The costs of producing
all individual statistics
are
measured
and
analysed,
and
mechanisms are in place
to
assure
cost-
effectiveness
of
statistical
activities
or
processes.
There is a system for registering cost and time used for all statistical
products, and it should be possible to estimate time used on the main
processes.
Cost per unit individual should determine for
each
statistical
activity
and
should
be
calculated for alternative methods of data
collection. Concept notes and proposal may
serve as evidence
The costs of producing the statistics are well documented at each stage of
the production process, and regularly reviewed and analysed across
statistical products to assess the effectiveness of their production
Cost–benefit analyses are carried out to determine the appropriate trade-
offs in terms of data quality.
The cost-effectiveness of every statistical survey is assessed.
The need for each survey variable to be collected is justified.
There is an ongoing review process that considers whether a particular
program is still operating in the most cost-effective way to meet its stated
requirements.
Data collection instruments are designed to minimize coding and editing
cost and time.
11.2
Procedures
exist
to
assess
and
justify
demands
for
new
statistics
against
their
cost.
Demands for new statistics are regularly registered and assessed by
statistical experts with respect to the proposed methodology and
associated costs, and discussed by management, based on inputs from
users and in cooperation with other stakeholders.
User Satisfaction Survey, user feedbacks and
user-producer forums can be used as a means
of
justifying
demands
for
new
data.
Assessments and cost and benefit analysis
reports need to be prepared
Before contemplating a new data collection, there are mechanisms to
review whether already available data sources can be utilized with
minimal impact on their purpose and quality.
When introducing new statistics, a cost-benefit analysis is conducted.
11.3
Procedures
exist
to
assess
the
continuing
There are regular discussions by management on the usefulness of all
statistics, based also on inputs from users such as the results of user
satisfaction surveys.
NSOs must have a prioritization policy or
guideline
for
statistics
production.
44
Requirement
Element to be assured
Description
need for all statistics, to
see
if
any
can
be
discontinued to free up
resources.
The usage of different statistical products including statistical databases
is monitored and assessed to evaluate their relevance.
Questionnaires
for
periodic
surveys
are
reviewed regularly before undertaking of
survey
Users and stakeholders are informed and consulted about the possible
discontinuation of statistical outputs.
11.4
Modern information and
communication
technologies are applied
to
improve
the
performance
of
statistical processes.
An appropriate IT strategy exists and is regularly reviewed and updated
to improve effectiveness and efficiency of the statistical processes.
NSOs have to ensure the use of ICT tools in
planning,
designing
and
implementing
statistical activities at all stages. Primary
statistical data collection are conducted using
CAPI. Most of statistics are disseminated
through websites, dashboards and social
media
The IT architecture and hardware infrastructure are regularly reviewed and updated, and possibilities for innovation and modernization are identified.
Routine clerical operations and statistical processes (e.g. data capture, coding, data editing, data validation, data exchange) are automated where possible and are regularly reviewed.
Centralized IT and methodological units exist and provide possibilities for pooling of resources and investments. 11.5 Proactive efforts are made to improve the statistical potential of administrative data and other data sources. Statistical agencies provide input in the legislative process to obtain and maintain access to administrative and other data sources for statistical purposes, if needed.
NSOs should have strategic actions to improve administrative data and other non- traditional data sources. TSMP II prioritizes improvement of administrative data as among NSS focus Appropriate arrangements (e.g. service-level agreements or national legislation) with owners or holders of administrative data and other data collections are made and updated as needed, specifying the access to and flow of data and metadata and other relevant aspects An assessment of possible administrative data sources is carried out prior to launching any new survey. Data-linking and integration methods are pro-actively pursued while ensuring data security and privacy. Quality reports for administrative and other data used for official statistics are established by the responsible statistical agency in cooperation with the data owners or holders. 11.6 The statistical agencies define, promote and implement integrated and standardized The statistical agencies have developed strategies to move to a more integrated and standardized statistical production system within their organization. NSOs should have a guideline which document all procedures to be followed during statistics production. Evidence is shown through Manual for Strengthening and The statistical agencies promote, share and implement standardized