Skip to content

Automatically downloads a database of known IP addresses as well as other location data and creates a lucene index for spatial searching for IP addresses within a specific range (or other criteria). Stores lat/lon, zip, city, country, and IP addresses for fast lucene search. Index is 1GB upon completion.

krickert/IPAddressZipCodeStateCountryLuceneJavaSearch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

If you have maven and windows, you jsut need to tweak a little bit: the bootstrap script just installs 2 jar files to your local repository.

To run:
1) bootstrap.sh
2) mvn test
3) run.sh

This will:
1) Download the geospatial database of IP addresses with lat/lon data
2) Create a spatial lucene index for searching by a number of fields

For speed, it uses a lot memory - right now I'm setting it to 4GB but it'll push it up to 3GB.  You can make this A LOT smaller. Take a look at src/main/resources/project.properties for all the configuration options you can use.

TODO:
1) Just return the IDs and retrieve the data from a database
2) Get this working awesome with Postgresql's http://www.gissearch.com/
3) Learn more about this stuff
4) Tune this.  It takes about 5-10 minutes to make on my mac, but to write out 1 GB I know it can go faster.

It'll create a directory called "index" which will contain all the lucene segment files needed.

How it works:
1) Checks to see if you want to download a new index
2) Downloads new data if needed
3) Uses SuperCSV to parse the data and put it in a queue for processing
4) Spawns off a configurable number of threads and grabs the data from the queue and writes it concurrently to the solr index writer
5) When the threads are done, the index writer commits the index
6) Happy geo searching.


What this can be used for:
* Website feature: "who's near me online?"
* Fraud: don't allow IPs from specific regions 
 

About

Automatically downloads a database of known IP addresses as well as other location data and creates a lucene index for spatial searching for IP addresses within a specific range (or other criteria). Stores lat/lon, zip, city, country, and IP addresses for fast lucene search. Index is 1GB upon completion.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published