35
Part 4: Programme implementation (GSBPM Phases 4 to 7) Part 4 of the Framework covers t he implementation phases of individual stati stical programmes; i t is concerned with activities such as data collection, data processing, data analysis and the dissemination of the various final products to users. It includes 61 different quality elements in 10 different sections as follows. 4.1 Managing relations wit h data users and providers , focusing mainly on dissemination; 4.2 Managing standards for the implementation of statistical programmes; 4.3 Assuring the confidentiality and security of statistical data when the results and micro-data are disseminated; 4.4 Improving the quality of the implementation of the statistical programme; 4.5 Making sure that the statistical programme is implemented in a cost effective and efficient way; 4.6 Making sure that the outputs from the statistical programme remain relevant to users; 4.7 Making sure that the statistics produced by the programme are accurate and reliable; 4.8 Making sure that the statistics produced by the programme are timely and available to users with a minimal delay; 4.9 Making sure that the statistics produced by the programme are accessible and clear to users; 4.10 Making sure that the statistics disseminated by the programme are coherent and can be compared with other sources and over time.
36
Programme implementation (GSBPM Phases 4 to 7) Quality Element UN NQAF Element Link to existing policy, guidelines or practice How the quality element is assured and recommendation (if any) 4.1: Managing relations with data users and providers 4.1.1 User support services are in place so that prompt assistance is given to users to help them access and interpret the data.
2.25
June 2010
Dissemination and
Pricing Policy
User support services are available from NBS but only on an ad-hoc
basis.
4.1.2
Information on the methodology of statistical
processes and the quality of statistical outputs is
available and provided.
2.26 SMSG and CDOS Metadata are prepared and available for some, but not all statistical processes. Need to extend to all statistical processes 4.1.3 Processes are in place for assuring the confidentiality of data from individuals, businesses or other entities in administrative records
2.27 Statistics Act 2002 and SMSG Documentation is needed 4.2: Managing standards for the implementation of statistical programmes 4.2.1 The agency uses appropriate conceptual frameworks, such as the SNA, that provide a basis for consolidating statistical information
3.12 SMSG and CDOS Documentation is needed 4.2.1 The managers of statistical programmes are held accountable for applying appropriate statistical standards.
3.14 SMSG and CDOS Documentation is needed 4.2.3 Plans for the development and application of new standards are communicated to managers responsible for statistical programmes well in advance.
3.16 SMSG and CDOS This is not done systematically 4.2.4 Statistical programmes collect and retain information at the fundamental or most detailed level of each standard classification in order to provide maximum flexibility in aggregation and to facilitate reclassification as needs change 3.17 SMSG and CDOS Documentation is needed
37
Programme implementation (GSBPM Phases 4 to 7)(ct’d) Quality Element UN NQAF Element Link to existing policy, guidelines or practice How the quality element is assured and recommendation (if any) 4.2.5 Statistical products are accompanied by, or make explicit reference to, readily accessible documentation on the statistical standards used
3.18 SMSG and CDOS Documentation is needed 4.2.6 Periodic reports to senior management are prepared about the extent to which statistical standards are used by the statistical programmes/domains
3.19 SMSG and CDOS This is done on an ad-hoc basis and a more systematic approach is needed 4.2.7 The statistical standards used are communicated to all potential data users and the public
3.20 SMSG and CDOS This is done but a more systematic approach is needed 4.3: Assuring the confidentiality and security of statistical data 4.3.1 Appropriate processes are in place to assess the risk that individual respondents can be identified from the public release of statistics or of microdata, and procedures are applied to eliminate or minimize this risk
7.9 SMSG and June 2010 Dissemination and Pricing Policy Documentation is needed 4.3.2 Where there is a risk of identification of individuals from the public release of statistics or microdata, and this risk is considered to be above a minimum level, then, depending on the sensitivity of the data, the data or microdata are not disseminated
7.10 June 2010 Dissemination and Pricing Policy Documentation is needed 4.3.3 All procedures that are taken to eliminate or adequately reduce the risk of identification are properly documented and made available as part of the metadata related to the statistical dataset
7.11 SMSG and June 2010 Dissemination and Pricing Policy This is done on an ad-hoc basis and a systematic approach is needed 4.3.4 Users are made aware that procedures to eliminate the risk of identification have been implemented, and that this could lead to a loss of information
7.12 SMSG Documentation is needed
38
Programme implementation (GSBPM Phases 4 to 7)(ct’d) Quality Element UN NQAF Element Link to existing policy, guidelines or practice How the quality element is assured and recommendation (if any) 4.3.5 Appropriate physical and information technology security procedures ensure the protection of unit records
7.13 SMSG and June 2010 Dissemination and Pricing Policy Documentation is needed 4.4: Improving the quality of the implementation of the statistical programme 4.4.1 Mechanisms are in place to assure the quality of the data collection and editing processes
8.16 SMSG This is done but a systematic approach is needed 4.4.2 The costs of each stage of the production process are measured
9.10 SMSG and Program Budget This is a practice 4.4.3 Appropriate management processes are in place to adequately monitor the quality of the data collection and data processing
10.19 SMSG Documentation is needed 4.4.4 Adequate measures are in place to encourage accurate responses, to follow up on non-response, and to deal with missing data.
10.20 SMSG Documentation is needed 4.4.5 Appropriate arrangements are in place to make both internal and external consistency checks of data, with corresponding correction or adjustment strategies
10.22 SMSG Documentation is needed 4.4.6 Information on costs and efficiency of operations are assembled to inform future design decisions
10.23 SMSG Documentation and a standardized way of costing are needed 4.4.7 Management information is available to manage and monitor all aspects of the collection phases
10.24 SMSG Needs to be done a systematic manner 4.4.8 Contingency plans exist in case problems emerge 10.25 SMSG Documentation is needed
39
Programme implementation (GSBPM Phases 4 to 7)(ct’d) Quality Element UN NQAF Element Link to existing policy, guidelines or practice How the quality element is assured and recommendation (if any) 4.4.9 For all major data collections, assessments are done in respect of: coverage of the population against the target population; the sampling error; non-response rates; and any other serious accuracy or consistency problems
10.26 SMSG Documentation and a systematic way of doing this is needed 4.5: Making sure that the statistical programme is implemented in a cost effective and efficient way 4.5.1 The use of information and communications technology is optimized for data collection, processing and dissemination
11.15 SMSG It is done partially, there is a need to extend to all programs 4.5.2 Every effort is made to minimize the reporting burden, consistent with the principal purposes for which the statistics will be used
11.16 SMSG This is done but a systematic approach is needed 4.5.3 Routine clerical operations are automated wherever possible.
11.17 SMSG Documentation is needed 4.5.4 Training materials and manuals are carefully designed and are prepared so as to improve the skills of field staff
12.17 SMSG This is a practice 4.5.5 The list of respondents’ contact information is regularly updated
12.18 SMSG and CDOS This is done for business establishments and some other programs 4.5.6 All the main statistical activities are communicated and advertised in order to increase awareness and trust among respondents.
12.19 SMSG Documentation is needed 4.5.7 Staff involved in data processing are trained and are familiar with the principles of confidentiality of data
12.20 SMSG and Statistics Act 2002 This is a practice
40
Programme implementation (GSBPM Phases 4 to 7)(ct’d) Quality Element UN NQAF Element Link to existing policy, guidelines or practice How the quality element is assured and recommendation (if any) 4.5.8 The format of different databases is compatible with different statistical software that is used for compilation and data analysis
12.21 SMSG This is a practice 4.5.9 Data capture and data collection instruments are tested and adjusted prior to the actual field operation or data collection process
12.22 SMSG This is a practice 4.5.10 Unique record codes and identifiers are created for the purpose of record linkage with other sources of information
12.23 SMSG This is a practice 4.5.11 Proper follow-up procedures are planned for filling data gaps and handling inconsistencies.
12.25 SMSG Documentation is needed 4.5.12 Data editing is repeated after each stage of data processing, including imputation
12.26 SMSG This is a practice but a systematic approach is needed 4.5.13 When weighting is required, a weight is associated to each sampled unit in the database after the data processing is completed.
12.27 SMSG This is a practice but documentation is needed 4.5.14 Activity and cost indicators are produced and properly documented in order to be used in monitoring and managing both current and future collection processes
12.28 SMSG This is done but a systematic approach is needed 4.5.15 Standard practices for responding to respondents’ requests and complaints have been defined and are followed
13.10 Not yet in place Needs to be developed
41
Programme implementation (GSBPM Phases 4 to 7)(ct’d) Quality Element UN NQAF Element Link to existing policy, guidelines or practice How the quality element is assured and recommendation (if any) 4.6: Making sure that the statistical programme is and remains relevant 4.6.1 Editing and other statistical quality control measures are consistent with the needs of the most important users
14.16 SMSG This is done but a systematic approach is needed 4.6.2 Statistical products are aligned with users’ needs 14.17 SMSG This is a practice 4.6.3 Users are informed about known gaps between the statistical concepts used in the statistical programme and the concepts that are of most interest to users
14.18 SMSG This is done but a systematic approach is needed 4.6.4 Metadata are described and made available to users
14.19 SMSG Documentation is needed 4.7: Making sure that the statistics produced by the programme are accurate and reliable 4.7.1 Statistical procedures for analysis and compilation employ internationally recognized statistical techniques.
15.9 SMSG This is a practice 4.7.2 Data sources are systematically checked, and the data that are used are compared with data from other sources. 15.10 SMSG Documentation is needed 4.7.3 Results are compared with other sources of information in order to ensure their validity
15.11 SMSG Documentation is needed 4.7.4 Periodic reports on data quality are prepared and published
15.12 SMSG This is done but a more systematic approach is needed 4.7.5 Methods and tools for preventing and reducing non- sampling errors are in place and are used
15.13 SMSG Documentation is needed 4.7.6 Sampling and non-sampling errors are measured, evaluated and systematically documented
15.14 SMSG More on non-sampling errors needs to be done