Definitions of Data
Usually for interoperability, we try to follow World Bank national definitions applied to cities, or the relevant state of the art of data science.
Theoretical definitions may exist for smaller data-sets. But we always default to practical usable definitions. i.e. what is the data point designed to measure?
Where do you get your data from?
This is the number one question we get asked by far.
The main answer to this is over 5,000 major sources gathered by 2ThinkNow in over 100 languages. We have hundreds of thousands of minor sources.
From this we create a 'view'- a commercial understanding of what the most 'accurate' numbers are given the state of the data science at that time.
We also have done proprietary original research - 2ThinkNow have done R&D subsidised by Australian government incentives to complete gaps in data-sets. This separates our data from other data-sets.
Because of the size of our data set, our proprietary original research, and our knowhow (since 2007), our data will likely be more accurate than other sources.
Customers report over 98% accuracy.
Specific Sources
All sources are specified with data purchase where quoted directly. Or 2ThinkNow listed as source where we have done calculations.
Statistics
In order of priority: Local statistical agencies, state/national statistical agencies, university studies, whitepapers, research sources, sometimes organisations like U.S. Census, EuroStat (note that this is mostly NUTS not cities) or OECD but less frequent, our own proprietary estimates to complete gaps and original research, among other sources.
Censuses in emerging countries may reflect official numbers not actual numbers. (reality).
2ThinkNow include notes, are aware of statistical limitations, and can guide you.
Commercial Activity (Business)
We use meta data from commercial activity (such as firm openings or closures).
Also, original research. We use this meta data from commercial activity combined with proprietary algorithms to infer values. This may include statistics to train the algorithms.
Population Demographics
Original research. We use advanced algorithms, this may include statistics to train the algorithm, and public meta data from personal activity (in aggregate).
2ThinkNow do not hold or track data on individuals for data gathering.
Original Research
We do a number of other original research, where we are listed as the source. This will be explained and noted when data is purchased.
Other
Other sources are wide and varied. They include media, trade organisations, trade fairs. All of these are directly quoted.
Data Guarantee and Support
2ThinkNow guarantees all data against errors or omissions. We will investigate, and if necessary, replace any data with errors or relevant omissions, based on better available data.
The ability to gather city data is always improving, so the state of the art moves forward.
Less than 2% of our data set is reported as containing errors on each engagement, and in many cases the % reported is zero % errors. (no support needed).
In either case, you are covered by our Data Guarantee for any support.
Time Series
Most core data sets we have proprietary methods to complete gaps in data sets and provide data-sets going back 10 or more years. These will be estimates using our proprietary methods. Time series back to 2012, and in some cases, 2009 generally available.
Why can't I find the data myself?
First, there is no central city data authority. There is no government portal with all the cities data (unlike national data). Data is scattered.
Often, open data portals do not hold useful data.
Most data projects are under-funded, or only funded some years. And, many data sets exist only while an expert employee compiles them, and cease to exist after.
And, where open data portals exist they are not complete, not exhaustive, and often reflect compromises of politics. Yes, there are many political considerations or compromises that block data accuracy. In one example two rival Asian nations could not agree on means of measurement of railway track (presumably a fairly straightforward item to measure).
How much harder is it to agree how to measure contentious areas?
Also, our work is highly specialised data analysis and data science.
Data is specialised - just like accounting, medicine or law. 2ThinkNow specialise in cities data. You wouldn't ask a company tax accountant to do a forensic accountants job.
Many other firms do not make this distinction, assuming, data is data.
Finally, 2ThinkNow have done city data globally (not just one country or bloc) for over a decade and have specialised methods and tools.
And yes, here are more reasons:
A) you have to know what variable you are finding
B) the data is often wrong in its raw form (or at least not clear)
C) there are often multiple competing answers (cities can have 5-7 population figures)
D) it will cost you more to find the data than 2ThinkNow
E) we have pre-existing data-sets for cross-checking, anyone else will not. This means most other data sets have lower accuracy.
F) time! Cities data is notoriously difficult to find.
G) our data is current years, not old and ancient 2012 data.
H) translating the data's meaning from multiple sources and harmonising it takes many iterations, something beyond the scope of a single project.
We could continue.
Or, 2ThinkNow data is accurate cities data, at a fair price.
Purchase Data
Average Price per City
Prices are per city. Minimum cities apply.
Level 2 Data Point