Seems mod_geoip2 was worth the effort

Started by Gregg, August 28, 2009, 09:54:52 PM

Previous topic - Next topic

Gregg

Yes I admit at this time there are only 6 downloads of this module, for me it was worth it even if the count was at 0. Seeing use by others than I makes it that much better.

Yes, I'm bored :lol:

mario

For me it still works fine. On the company server it is funny to see that some customers come over a proxy in many countries.
Against your bornes: Write the tutorial how even I can compile apache  :P

Sob

Current mod_geoip has one little disadvantage. It doesn't support IPv6. And not supporting IPv6 is bad, mmmkay... :)

I played with it a little and came with this:

http://web.hisoftware.cz/sob/download/mod_geoip2_1.2.5-ipv6-1.patch
http://web.hisoftware.cz/sob/download/GeoIP-1.4.6-win-ipv6-1.patch

When additional data files are added to Apache's config:

GeoIPDBFile "c:\path\to\your\GeoIPv6.dat"
GeoIPDBFile "c:\path\to\your\GeoLiteCity.dat"

Then it works fine for countries and does something for cities.

I have only one production IPv6 range to test with and it doesn't return any city info for it. For 6to4 it returns something, but complete nonsense. I tried to fake some addresses using GeoIPScanProxyHeaders and X-Forwarded-For and it shows something, but always somewhere in US, even for those where I'm sure they're somewhere else. So this part doesn't seem to work right. When looking at the csv version of GeoLiteCity database, it doesn't seem to include IPv6 at all. But in that case I don't get why it finds something.

You're welcome to join and play.

Gregg

Hi Sob,

"For 6to4 it returns something, but complete nonsense."

To be quite honest, I get complete nonsense at times in ipv4.

I have no outside IPv6, only internal so I cannot test this right now. However, I am working up a build for 2.3 that will build all modules (fcgid/macro/log_rotate/etc) in tree. As long as the mmn in ap_mmn.h keeps changing, modules have to be recompiled. Might as well do them during the compile of Apache.

When I get to geoip, I'll give these a try. If it doesn't hurt, I see no problem adding it to the bleeding edge alpha's collection of included modules. I'll talk it over with Mario also once I think I know what I am talking about  :D

Thanks for the patches.

Sob

Only GeoLiteCity.dat returns nonsense for IPv6 addresses. I did a little more reading since the last time and now I'm sure there's no IPv6 support in that database. Only available with IPv6 support is country database GeoIPv6.dat and it works correctly. So I removed the changes for city database. I also changed country part to avoid unnecessary lookups in wrong database:

http://web.hisoftware.cz/sob/download/mod_geoip2_1.2.5-ipv6-2.patch

It should be safe and correct. Except maybe for scanning proxy headers, I'm not sure if they would contain bare IPv6 address or enclosed in [ ].

tlitody

Thanks guys,

I just installed on mod_geoip win2K apache 2.2.11 and it seems to work fine.

Regarding city lookups. In my experience city lookup is a waste of time because the data is unreliable. The ISPs buy blocks of IP numbers and they all get assigned the town/city of the registered address of the ISP. When the ISPs allocate IP numbers to subscribers they don't assign town/city of the subscriber to the IP number. Also many ISPs use dynamically allocated IPs from their range so that one minute a visitor on one IP is in one city and an hour later the same IP may be being used by someone in another city.
The concept of using city lookups is fundamentally flawed due to inaccurate data. At least it is in the UK. Things may be different in different countries but I doubt it.
So I wouldn't waste your time on city and or lat/long modules.

You can check on the maxmind lookup at:

http://www.maxmind.com/app/locate_demo_ip

if I use my dedicated IP number in the UK it tells me I'm in Uxbridge about 180 miles from where I really am.

ricardok1

Hi there,

Anybody have a win32 vc6 compiled version of mod_geoip 1.4.6 or 1.4.7 beta ?

Thanks,
Ricardo

Gregg

Quote from: tlitody on December 08, 2010, 02:33:16 PM
Regarding city lookups. In my experience city lookup is a waste of time because the data is unreliable. The ISPs buy blocks of IP numbers and they all get assigned the town/city of the registered address of the ISP. When the ISPs allocate IP numbers to subscribers they don't assign town/city of the subscriber to the IP number. Also many ISPs use dynamically allocated IPs from their range so that one minute a visitor on one IP is in one city and an hour later the same IP may be being used by someone in another city.
The concept of using city lookups is fundamentally flawed due to inaccurate data. At least it is in the UK. Things may be different in different countries but I doubt it.
So I wouldn't waste your time on city and or lat/long modules.

Well, Lat/Lon is totally useless unless you want the rough estimate of the lat/lon of whatever city.

I do know that although yes,city can be unreliable, I am finding to be getting better, and even when off it is getting close. I've been using this module for years now and just watching how it handled my own IP was telling enough.

Many years ago I was showing as coming from the city my ISP was located.
Then one day ISP switched the netblock pool I was being assigned from and it started showing a city that at least was in my county and 20 miles up the coast. Another IP some time later and it starte showing the city properly.

Then I moved, was assigned a IP from yet another net block and it was showing the correct city again. Then I got pushed to yet a different net block and now it is showing San Diego, which is not 100%, but close enough for nuclear weapons.

Obviously, anything free and mileage may vary, since I see ads targeted for my area on the likes of Youtube and others, I have a feeling the paid for database is rather accurate. Since they need to make a reason for you to pay, making the free database to be less accurate seems logical.


Gregg

OK, along with my diatribe above, I'm going to do this.

I finally remembered what I had to do to build this module in the first place so since that is the case, and this feature has now been asked for, I should have binaries up this week sometime.

Notice, they will be labeled with that horrible 12 letter word "Experimental," since it honestly is. I have no ipv6 outside the perimeter till we move into the 21st century here so cannot test the ipv6 lookups, however is seems to play nice on v4 which is good enough for me.

I have vc6 & vc9 x86 modules built at the moment, will do x64 when I crank up that box to build Apache 2.3.10. I may also be including it in the 2.3.10-alpha should I have time to work it into the build.

mario

Quote from: Gregg on December 13, 2010, 09:52:25 PM
I have no ipv6 outside the perimeter till we move into the 21st century here so cannot test the ipv6 lookups, however is seems to play nice on v4 which is good enough for me.

With IPv6 you should be able to locate your toaster if linux with network is running on it  ;D

Gregg


tlitody

Quote from: Gregg on December 13, 2010, 09:12:33 PM
Well, Lat/Lon is totally useless unless you want the rough estimate of the lat/lon of whatever city.

I do know that although yes,city can be unreliable, I am finding to be getting better, and even when off it is getting close. I've been using this module for years now and just watching how it handled my own IP was telling enough.

My experience is in the UK where it just isn't anywhere near accurate. I think it really does depend on the ISPs updating the city/town of the end user. Or possibly some mechanism in the network to detect where the connection is coming is from but it certainly doesn't work in the UK.

mario

In Germany it depends where the "point of presence" is (Hope that that is correct word german: Einwahlknoten) and the first DNS server in the backbone. Someday it is 150 km away and than only 20 km.
On the one hand it is bad not to locate users. On the other hand I don't want ppl to know exact where I live.

Gregg

Looking at the numbers, your toaster in Algeria Mario might easily show up a couple hundred miles away  from where it should :D

http://www.maxmind.com/app/geolite_city_accuracy

tlitody

Quote from: mario on December 14, 2010, 12:01:35 PM
In Germany it depends where the "point of presence" is (Hope that that is correct word german: Einwahlknoten) and the first DNS server in the backbone. Someday it is 150 km away and than only 20 km.
On the one hand it is bad not to locate users. On the other hand I don't want ppl to know exact where I live.

But even that would depend on the Point of presence having had its city attributed to a specific town/city.

I beleive the level of accuracy is really determined by the fact that the ISPs are huddled around internet backbone hubs which tend to be in big towns and cities with high population densities. Therefore there is a natural tendancy for a large part of the population to be living fairly close to their ISP. But once you get out of the big towns the level of accuracy will will be much lower.
I only raised it because it seems pointless (in the UK) for anyone to be trying to use city data to make any kind of business decisions. It's simply not accurate enough.