28
Programme design (GSBPM phases 1 to 3) Quality Element UN NQAF Element Link to existing policy, guidelines or practice How the quality element is assured and recommendation (if any) 3.1: Managing relations with data users and providers 3.1.1 Processes are in place for consulting with user groups when new statistical programmes are developed or existing statistical programmes are reviewed
2.19 SMSG Consultation processes are put in place when new statistical processes are being designed 3.1.2 Processes are in place for monitoring the relevance and practical utility of existing statistics to users
2.10 Not in place Needs to be developed and put in place 3.1.3 Processes are in place for users to advise agencies about their emerging needs and priorities
2.21 SMSG This is done routinely by NBS when new major surveys or censuses are planned, it is also part of the TSMP process 3.1.4 Processes are in place for maintaining close cooperation with the most important groups of users
2.24 TSMP Sector Working Groups Consultation processes are in place 3.2: Managing standards for the design of statistical programmes
3.2.1 Users and data providers are involved in developing and approving standards
3.7 TSMP Sector Working Groups Consultation takes place when a new strategic or master plan is prepared 3.2.2 A statement of conformity to corresponding international or national standards is included in standards.
3.8 SMSG and CDOS More documentation is needed 3.2.3 Divergences from the corresponding international or national statistical standards are documented and explained.
3.9 SMSG and CDOS More documentation is needed 3.2.4 Where new standards are introduced, detailed links to previous standards are made available 3.11 SMSG and CDOS Formal process for updating standards needs to be put in place together with the need for documentation
29
Programme design (GSBPM phases 1 to 3) (ct’d) Quality Element UN NQAF Element Link to existing policy, guidelines or practice How the quality element is assured and recommendation (if any) 3.3: Assuring the confidentiality and security of the statistical data generated by the programme 3.3.1 Statistical agencies identify in advance what data and microdata are to be disseminated and identify any risks that confidential data about individual respondents could be identified
7.7 SMSG Statistics Act 2002, June 2010 Dissemination and Pricing Policy This is done to a limited extent, but is not fully documented 3.3.2 Appropriate procedures are identified in advance to ensure that data and microdata are anonymized 7.8 SMSG and June 2010 Dissemination and Pricing Policy
This is done to a limited extent, but is not fully documented
3.4: Improving the quality of the design of the statistical programme
3.4.1
At the design stage, a quality assurance plan is
prepared that describes the standards to be used, any
formal obligations and quality control actions to
prevent, monitor and evaluate errors and to control
different stages of the statistical process.
8.12 SMSG This is for major surveys and censuses and is part of the programme design stage 3.4.2 Trade-offs affecting quality are systematically examined
8.15 SMSG This is done but more documentation is needed 3.5: Making sure that there are adequate resources for the statistical programme
3.5.1 An assessment is made to ensure that the programme is feasible given the available resources
9.7 SMSG This is done for major surveys and censuses and is part of the programme design stage 3.5.2 The programme makes as much use as possible of existing methods and tools
9.8 SMSG As far as possible methods and tools are standardised within NBS 3.5.3 The programme uses data from existing sources, wherever appropriate
9.9 SMSG This is done but more documentation is needed
30
Programme design (GSBPM phases 1 to 3) (ct’d) Quality Element UN NQAF Element Link to existing policy, guidelines or practice How the quality element is assured and recommendation (if any) 3.6: Making sure that the statistical methods used are sound and in line with international recommendations and good practice 3.6.1 The scope of the programme and other aspects such as the concepts, definitions, classification and basis of recording are in accordance with applicable international standards.
10.11 SMSG and CDOS Documentation is needed 3.6.2 Explicit consideration is given to trade-offs between accuracy, cost, timeliness and provider burden
10.12 SMSG and CDOS Documentation is needed 3.6.3 Alternative sources of data, including the availability of existing survey data or administrative records is explicitly considered so as to minimize new data collection
10.13 SMSG Documentation is needed 3.6.4 There is proper justification for each question asked and appropriate pre-testing of questions and questionnaires takes place
10.14 SMSG This is the practice 3.6.5 A systematic approach is in place for updating the survey frame to ensure adequate coverage of the target population.
10.15 SMSG This is the practice of the NBS following each population census 3.6.6 There is proper consideration of sampling and estimation options and their impact on accuracy, timeliness, cost, respondent burden, and data comparability
10.16 SMSG This is the practice 3.6.7 Where administrative records are used, it is verified that the population is consistent with the statistical output requirements; the classifications are appropriate; the underlying concepts are appropriate; and the records are complete and up to date 10.17 SMSG and CDOS This is the practice of NBS but more documentation is needed
31
Programme design (GSBPM phases 1 to 3) (ct’d) Quality Element UN NQAF Element Link to existing policy, guidelines or practice How the quality element is assured and recommendation (if any) 3.6.8 The data collection process is designed to reduce the respondent burden and to increase response rates
12.6 SMSG Documentation is needed 3.6.9 The soundness of the use of administrative data sources is assessed
12.8 SMSG This is done on an ad-hoc basis and a more systematic approach is needed 3.6.10 The data collection instruments (mainly questionnaires) are designed to minimize coding cost and time
12.9 SMSG Documentation is needed 3.6.11 The questionnaire design allows for automated data capture
12.11 On an ad hoc Documentation is needed 3.6.12 Data capture is designed to improve accuracy and timeliness by making use of automated data capture and data capture in the field where relevant
12.12 SMSG This is being done where feasible, but more documentation is needed 3.6.13 Edit rules are streamlined in order to validate the data entered and allow for error corrections and quality improvement during the data capture exercise
12.13 SMSG Documentation is needed 3.6.14 Follow-up activities are implemented to collect information for non-response adjustment
12.14 SMSG This done only for major censuses 3.6.15 A sound methodology is in place for imputation that has been developed based on scientific principles
12.15 SMSG This is done on an ad-hoc basis and a more systematic approach is needed 3.6.16 Staff involved in data collection are consulted before developing training materials
12.16 SMSG Experienced field staff, but interviewers are not consulted
32
Programme design (GSBPM phases 1 to 3) (ct’d) Quality Element UN NQAF Element Link to existing policy, guidelines or practice How the quality element is assured and recommendation (if any) 3.7: Making sure that the statistical programme is cost effective and efficient 3.7.1 Clear and documented justification for each specific programme exists
11.12 SMSG and TSMP Documentation is needed 3.7.2 Before contemplating a new data collection, a review of whether current data sources can be used is undertaken.
11.13 SMSG and TSMP Documentation is needed 3.7.3 On-going review processes are in place to consider whether a particular programme is still operating in the most cost-effective way
11.14 TSMP and Mid-term Reviews This is done when a new strategic or master plan is prepared or as part of a formal mid-term review 3.8: Managing the burden on respondents 3.8.1 Mechanisms are in place to assess the necessity to undertake a new statistical survey.
13.6 SMSG Documentation is needed 3.8.2 Surveys apply sound methods to reduce or distribute the response burden.
13.7 SMSG Documentation is needed 3.8.3 Surveys apply statistical standards to make it easier to respond to them.
13.8 SMSG and CDOS Documentation is needed 3.8.4 Respondents are provided with information about the survey and how the information they provide will be handled and used
13.9 SMSG Documentation is needed 3.9: Making sure that the statistical programme is and remains relevant
3.9.1
The survey objectives are fully documented based on
the needs of the users
14.14
SMSG
This is a practice
33
Programme design (GSBPM phases 1 to 3) (ct’d) Quality Element UN NQAF Element Link to existing policy, guidelines or practice How the quality element is assured and recommendation (if any) 3.9.2 User satisfaction is regularly measured and systematically followed up.
14.15 TSMP User satisfaction surveys are carried out on a regular basis 3.10: Making sure that the statistics produced by the programme are accurate and reliable 3.10.1 A quality assurance plan for the programme is prepared that sets out actions to prevent, monitor and evaluate non-sampling errors
15.7 Not in place Needs to be developed 3.10.2 Where relevant, the revision policy follows standard and transparent procedures in the context of each survey.
15.8 No revision policy in place This is done on an ad-hoc basis 3.11: Making sure that the statistics produced by the programme are timely and available to users with a minimal delay 3.11.1 Explicit consideration is given to overall trade-offs between timeliness and other dimensions of quality during the programme design stage.
16.11 SMSG Documentation is needed 3.11.2 The possibility and usefulness of releasing preliminary data is considered, taking into account the data’s accuracy
16.12 SMSG Documentation is needed 3.11.3 Contingency plans exist to deal with emerging problems that could delay the release of data.
16.13 SMSG Documentation is needed 3.11.4 Realistic schedules are defined for the production processes
16.14 SMSG Documentation is needed 3.11.5 The maximum acceptable amount of time that can elapse - between the end of the reference period and the availability of the data – is specified and is known to staff and users. 16.15 SMSG and June 2010 Dissemination and Pricing Policy This is done on an ad-hoc basis and a more systematic approach is needed
34
Programme design (GSBPM phases 1 to 3) (ct’d) Quality Element UN NQAF Element Link to existing policy, guidelines or practice How the quality element is assured and recommendation (if any) 3.12: Making sure that the statistics produced by the programme are accessible and clear to users 3.12.1 Consideration has been given to trade-offs between accessibility and confidentiality during the programme design stage
17.18 SMSG Documentation is needed 3.12.2 A strategy has been developed and agreed with stakeholders for the release of data, metadata and, where possible, microdata from the data collection
17.19 Not in place A strategy needs to be developed 3.12.3 Processes are in place to ensure that metadata are documented according to standardized metadata systems, and are regularly updated.
17.20 SMSG NBS has standards for metadata or a systematic way of recording and archiving metadata. However, more documentation and guidelines are needed 3.13: Making sure that the statistics produced by the programme are coherent and can be compared with other sources and over time 3.13.1 The common repository of concepts, definitions and classifications is consulted when designing a new individual statistical programme/domain
18.12 CDOS Documentation is needed 3.13.2 Before a new statistical programme is designed, any major related statistics are analysed 18.13 SMSG This is done on a regular basis and a more systematic approach is needed