2nd Open Data Hackathon Info Day The National Open Data Portal and other Data Sources 3rd September 2018 Dimitris Michail (PAPD) Christos Rodosthenous (OUC)
Open Data Open Data is data that can be freely used, re-used and redistributed by anyone - subject only, at most, to the requirement to attribute and sharealike. (Open Data Handbook) Open Government Data is data collected by Public Service Bodies within the scope of their public tasks, which is released as Open Data.
Open Data: Demand and Supply Roles of Coordinating Authority Political Support ++ ++ Supply Accessibility Affortability Acceptability Responsiveness PSI L.O. Network Facilitate Demand Awareness Motivation Citizen Rights Consumer Voice PAPD Coordinating Authority
Open Data in Cyprus
National PSI
Re-Use Law,
2006
National
Action Plan
2013-2017
Launch of 1st
edition of
data.gov.cy
New National
PSI Re-Use
Law, 2015
2006
2013
2014
2015
2017 2018
National
Action Plan
2017-2021
Launch of 2nd
edition of
data.gov.cy
Analysis of
Current
Situation
Identification
of Users’
Requirements
Implementation
Design based
on OS
Software
March 2017
June 2017
January 2018
September 2017
Data.gov.cy Development
Evolution of the National Open Data Portal www.data.gov.cy 2014 2016 2018
Main Functions of CY Open Data Portal
Main Functions
Source of Information
Dataset Search
Host Support Material
Data Requests
Showcase Applications
Data Suppliers
80+ Central Government PSBs
1 Municipality (pilot)
1 Semi Governmental Organization (Pilot)
Examples of Datasets
Real Time Met Stations Data
Real Time Air Quality Data
Real Time Parking Availability (Nicosia)
POI (theatres, museums, stadiums, parks, government buildings etc)
Catalogues: Forest Fires, Public Procurements, Earthquakes, etc
Geospatial: e.g. forests, planning zones, municipality boundaries, etc
Registries: e.g. companies registry, nurseries registry,
Statistics: GDP, trade, tourism, immigration, etc
7 Format Transformation - Compatibility with Technical Framework o Machine Readable o Open Formats CSV instead of XLS, PDF, DOC 480 datasets 1350 700 datasets Carried out by the PSI Team (not an automated process) Assessment of all published datasets o Approve / Discard o Organize (Merged to one or Split to many) o Checks for Metadata Completeness +300 datasets Identification / Preparation & Publication of New Datasets Migration of Data and Open Formats
The Cyprus Open Data Portal Deployed using DKAN opensource platform [https://getdkan.org/]
DKAN Installations
92 documented worldwide installations Source: http://docs.getdkan.com/en/latest/introduction/dkan-sites.html
Organizing and searching data Filter data per: Topic Organization Format Tag License Type
Visualizing Data using the Portal Interface
Grid format
Graph format
Map format
Access to Data and Metadata using APIs The data.gov.cy provides access to the portal data and metadata using four (4) APIs: Datastore API: The DKAN Datastore houses any CSV files that have been uploaded and then imported into the Datastore and provides an endpoint to access them CKAN API: Harvest and index the Portal from other CKAN-based portals (CKAN harvester) Open Data APIs: Two Open Data Standards APIs: DCAT-AP και Project Open Data Dataset REST API: Access all Portal content nodes
Datastore API - Endpoint
Datastore API - Parameters Parameters •resource_id (mixed) – id (string) or ids (array) of the resource(s) to be searched against. •filters (mixed) – array or string of matching conditions to select •q (string) – fulltext search •offset (int) – offset this number of rows •limit (int) – maximum number of rows to return •fields (array or comma separated string) – fields to return (default: all fields in original order) •sort (string) – comma separated field names with ordering •join (array) – array of fields to join from multiple tables •group_by (array) – array of fields to group by Aggregation functions •sum (string) – field to compute the sum •avg (string) – field to compute the average •min (string) – field to compute the maximum •max (string) – field to compute the minimum •std (string) – field to compute the standard deviation •variance (string) – field to compute the variance Source: https://docs.getdkan.com/en/latest/apis/datastore-api.html
Datastore API - Example https://www.data.gov.cy/api/action/datastore/search.json?resource_id=b0de1b05-88e7-4350-96d0-ad1266bf2b35&limit=5&language=en
Portal Statistics (Period: 5 May 2018 – 28 August 2018) Visitors: 10,600 Visitors’ Countries : 83 % Visitors from Abroad : 35% Page Views = 74,000 Page Views per Day = 630 % Access via API = 22%
Examples of Datasets
Location and Measurements of E/M Radiation of Radio-communication Stations Publisher: Department of Electronic Communications Themes: Science and Technology Formats: CSV, API (json) Frequency: bi-annual Tags: EM radiation
Real-Time Recordings of Meteorological Stations Publisher: Department of Meteorology Themes: Science and Technology Formats: API (XML) Frequency: Hourly Tags: καιρός, κλίμα Related datasets: Weather Forecasts by Area
Real-Time and Historical Recordings of Air-Quality Stations (3 datasets) Publisher: Department of Labour Inspection Themes: Environment Formats: API (xml) – real time data CSV, API (json) – historical Frequency: Hourly, Daily Tags: ατμόσφαιρα
Companies, Partnerships and Business Names Registry Publisher: Department of Registrar of Companies and Official Receiver Themes: Economy and Finance Formats: CSV, API (json) Frequency: Monthly Tags: companies registry Related Datasets: statistics on registered companies, trade marks, etc
Public Transport Datasets Collection of 7 datasets (one per Bus Service Operator) that include: OSEL EMEL OSYPA OSEA Publisher: Departments of Public Works Themes: Transport Formats: csv, shp, kml, json Tags: bus, δημόσιες μεταφορές EMEL Routes (shp) ZINON Nicosia Small Buses INTERCITY
Municipality of Nicosia Datasets Collection of 40+ Datasets that Include: Parking Availability Upcoming Events Parks and Municipal Gardens POIs Libraries Theatres Galleries Museums ….. Themes: Regions, Cities and Zoning Formats: XML Frequency: Real Time, As Required Tags: Nicosia
Collection of Geospatial Data Collection of 180+ Layers from the: CY Cadastral Map CY Topographic Map CY Hydrographic Map CY Administrative Map + CSS, hospitals, fishing shelters, natural trails, picnic sites, etc…. Publisher: mainly Department of Lands and Surveys Themes: Regions, Cities and Zoning, Environment Formats: wms, wfs, gml, shp, soap, kml Tags: γεωχωρικά
Fuel Prices API (requires registration) Cyprus Post API Earthquakes Datasets and APIs Forest Fires Datasets Police Statistics on Crime Public Procurements Catalogue Government Budget Election Results Public Servants (characteristic, sick leave, pay) • Tourism Statistics • Night Pharmacies • Water Quality Checks Results • Real Estate Statistics • Immigrations and Asylum Statistics • Ministerial Council Decisions • Archeological Sites and Museums • Post Codes Catalogue • Pharmaceutical Products Prices Other Interesting Datasets
Dimitris Michael Open Data Team Τμήμα Δημόσιας Διοίκησης και Προσωπικού Ministry of Finance Tel: +357 22 60 1523 / 1515 | Fax: +357 22 602763 Web: www.data.gov.cy | Twitter: @OpenDataCY Christos Rodosthenous Open Data Team Information, Technology and Library Unit Open University of Cyprus Tel: +357 22 411783 Web: www.data.gov.cy | Twitter: @OpenDataCY