rdata.work

Data Api

General

Tools

eXtensible Business Reporting Language (xbrl.org)

XBRL (eXtensible Business Reporting Language) is a freely available and global standard for exchanging business information. XBRL allows the expression of semantic meaning commonly required in business reporting. The language is XML-based and uses the XML syntax and related XML technologies such as XML Schema, XLink, XPath, and Namespaces. One use of XBRL is to define and exchange financial information, such as a financial statement. The XBRL Specification is developed and published by XBRL International, Inc. (XII).

Java

  • JAXB Java Architecture for XML Binding
  • XBRLAPI XBRLAPI provides a free, open source Java implementation of an API that supports analysis of XBRL data sources, both taxonomies and the business reports that are based upon them. XBRLAPI is released under the lesser GPL license.

R

SDMX

sdmx.org

sdmxtools.org

SOAP Requests

PX Web API of Nordic Countries (mainly SWE and FIN)

Standards

REST

OData

SOAP

RCurl example

library(RCurl)

headerFields =
  c(Accept = "text/xml",
    Accept = "multipart/*",
    'Content-Type' = "text/xml; charset=utf-8")

body = '<?xml version="1.0" encoding="utf-8"?>
 <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:hs="http://www.holidaywebservice.com/HolidayService_v2/">
 <soapenv:Body>
 <hs:GetHolidaysAvailable>
 <hs:countryCode>UnitedStates</hs:countryCode>
 </hs:GetHolidaysAvailable>
 </soapenv:Body>
 </soapenv:Envelope>'
url <- "http://www.holidaywebservice.com/HolidayService_v2/HolidayService2.asmx?wsdl"

curlPerform(url = url,
            httpheader = headerFields,
            postfields = body,
            verbose = TRUE)

Lists of SDMX data sources

International Organisations

UN Data Portal

Standards
SDMX, SOAP

European Central Bank (ECB)

Eurostat

Standards
SDMX, REST

International Labour Organization (ILO)

Standards
SDMX, REST

International Monetary Fund (IMF)

OECD

Standards
SDMX, SDMX-JSON, REST

UNESCO (UIS)

UNIDO

World Bank (WB)

National Providers (OECD member countries)

AUT Statistics Austria

Standards
CSV data, JSON meta data

STATcube - Statistical Database

SuperSTAR is the name of a software package comprising a number of applications for the processing of large amounts of data. The application provided by our homepage is specially designed for Internet application and is called SuperWEB.

DNK Statistics Denmark

Standards
JSON, XML

FIN Statistics Finland

Standards
XLSX, CSV, JSON, JSON-STAT, SDMX, PC-Axis

NOR Statistics Norwary

Standards
JSON

SWE Statistics Sweden

Standards
JSON

AUS Australian Bureau of Statistics (ABS)

Standards
SDMX

BEL National Bank Belgium (NBB)

Standards
SDMX, JSON, REST, DOTSTAT

CAN CANSIM Web Data Service (WDS)

Standards
SDMX 2.0, SOAP, XML
Authentication
IP address

DEU Federal Bank (Bundesbank)

Standards
SDMX-ML, REST

DEU DESTATIS

Standards
REST

Table Format (TabellenExport)

  • registration: standard customer (50 Euros)

Quader Format (DatenExport)

Contact

Tel 1 (Dirk Hollmann, technical)
+49(0)611 / 75-3403
Tel 2 (administrative)
+49(0)611/ 75-4555
E-Mail
genesis-online@destatis.de

ESP Instituto Nacional de Estatística

Can this be included in RJSDMX

Standards
SDMX, REST

FRA INSEE

GBR Office for National Statistics (ONS)

Geographic coverage

Most 2011 Ward Hierarchy datasets do not include Scotland or Northern Ireland on the system, so this is usually England and Wales (level 2) rather than UK (level 0).

Formats

The ONS OpenAPI supports XML and JSON outputs for both discovery and delivery operations.

SDMX validity

A dataset (or slice) is represented in SDMX by two responses: Structure and Data. The ONS OpenAPI, for convenience, allows you to receive the two responses in a combined message, but as this is not valid in SDMX an artificial ONSDataPackage entity is used to encapsulate them. If you are using SDMX-based tools (e.g. from Eurostat) you should always use the separate outputs (/dsd for the structure and /set for the data).

The Structure part of an SDMX message can contain Key Families, Concepts and Codelists, these three together comprise a DSD (Data Structure Definition). Parts of the DSD can be returned separately, or in a single response thing using dataset/[datasetid]/dsd.

Limit output

Slicing
One way of limiting the amount of data is via slicing. For each dimension you want to restrict the items for, add dm/[dimension name]=[item code] to the query string
/dataset/QS611EW.xml?context=Census&geog=2011WARDH&dm/2011WARDH=E06000045&totals=false&apikey=yourkeyhere

IRL StatCentral.ie

ITA ISTAT (SDMX)

Standards
SDMX
Italian National Institute of Statistics
Via Cesare Balbo, 16 00184 – Rome phone +39 06 4673.1
Dissemination and Communication of Statistical Information Directorate
Patrizia Cacioli phone +39 06 4673.2048-2049 fax +39 06 4673.2240 email dcdc@istat.it via C. Balbo, 16 00184 Roma
IP registration
http://sdmx.istat.it/registr_ut_sep/
requested IPs
  • 78.250.131.57 (available)
  • 78.41.129.246
  • 78.41.129.7
test IP

MEX Instituto Nacional de Estadística y Geografía (INEGI)

Standards
SDMX, REST, JSON

NLD Statistics Netherlands (CBS)

Standards
OData, REST, JSON

OData API

References

USA Bereau of Economic Analysis (BEA)

Standards
REST, NDJSON

Spain Instituto Nacional de Estadística

Standards
REST, JSON

National providers (OECD non-member countries)

Brazil

Commercial Providers

Bloomberg

Published

07 April 2015

Category

datascience