Geographical searches with Latitude and Longitude Part I
Calculating the distance between two geographical locations is a common feature of Web 2.0 applications. This is usually a two-step process. Firstly, obtaining the latitudes and longitudes of the locations from their addresses, a process known as geocoding. Secondly, calculating the great-circle distance between these two points and doing something meaningful with this value.
In this first of a 2-part article, we'll be looking at the design and concept of the solution, and in the second part, we'll examine the actual implementation.
Geocoding
The problem of geocoding is one that has been quite comprehensively addressed by several companies, amongst them MapQuest and Yahoo.
Usage of these REST web services is fairly straightforward - you supply either a zip code or an address, and a latitude and longitude is returned in XML format. All that's needed is to parse the output XML to obtain the lat/long.
We will therefore examine how to calculate the distance between two points and how to integrate this with Lucene to produce some cool geographical searches.
The Design
Suppose we have a database of events, each with a location address. Given any arbitrary address, we need to use Lucene to produce a list of events which are within 50 miles of this address.
We start by creating the Lucene index of the events, with each event corresponding to a Document. In addition to the event-related Fields such as name, date, address, organizer, etc we will also attempt to geocode the event's location address into latitude and longitude and add these as Fields.
When a search is performed, the input address will also be geocoded into latitude/longitude. The rectangle (or circle if you wish) containing the required search area will be calculated (50 miles from the given address). This rectangle will then be supplied to a custom Query class LatLongQuery, which will do the job of scoring the hits such that only documents with a latitude/longitude within the desired area will return, and that documents closer to the center will have a higher score.
Calculating distance between two points
To calculate the distance between two points, we need to calculate the great-circle distance between the two points because the Earth is not flat but spherical.
http://www.meridianworlddata.com/Distance-Calculation.asp contains a simple formula for calculating great-circle distance (but has up to 10% error). You're welcome to use a more accurate formula if you wish.
An approximate formula for distance bet. lat and long is d = sqrt(x^2 + y ^2) where x = 69.1(lat2 - lat1) and y = 69.1 * (lon2 - lon1) * cos(lat1/57.3)
We want to find max and min lat/long to obtain the bounding rectangle, which basically means alternately setting x and y = 0
To be continued...
We've mapped the application to the Lucene model, decided we're going to use a custom Query and Scorer to perform the actual searching, and also looked at how to calculate great-circle distance.
In the second part of this article, we'll examine the actual implementation for noteworthy points.
Credits
The code and concept used in this article was developed whilst working for a search engine for realtors, Homethinking.com. You can see the code in action.
Kudos to Niki, Homethinking Head-Honcho for allowing me to release this code.