Users of the data downloaded from the various sources identified face a number of serious problems if they are trying to combine data from different databases. The web site will try to identify some of these problems and when possible suggests ways in which the users can overcome them. We will also make suggestions that the data providers might wish to take into account to make their users' lives easier and more productive. One of the goals of this web site is to try to convince data providers to provide downloads in standard and useable formats such as txt, csv and xml and to try to at least standardize on the use of ISO two character codes for linking countries in different databases.
Reference Data -- One of the key problems concerns reference data (lookup tables) for users trying to link data from different sources, or often even from the same source. A simple example is the difficulty linking country data from more than one source -- nearly all of the data providers produce data identified by country name -- unfortunately some of the names of the countries in the different data bases are often spelled differently, and when codes are provided, even if many are identical, few organizations have codes that are entirely consistent with other organizations, all of which makes linking between two tables laborious. To access discussion of these reference data problems click here.
Data Formats -- Many of the organizations listed permit downloading of data as Excel or ASCII text files. However many others do not make downloads available or only provide .pdf format -- in these cases data often has to be entered by hand into a spreadsheet or database before it can be used in conjunction with data from other sources.
Data Completeness -- Some excellent web sites have been started by some international organizations, but have suddenly ended due to the end of a project or for unexplained reasons.