The Apache Haus Forum

Forum Topics => Apache Programming and Building => Topic started by: Gregg on January 09, 2011, 02:29:22 AM

Title: Adding a new trick to an old dog
Post by: Gregg on January 09, 2011, 02:29:22 AM
Surfing along the info superhighway one day looking for a way to get a remote users time zone using the GeoIP data I happened to stumble onto GeoIP Extended (https://forums.alliedmods.net/showthread.php?t=95665).

This geoip API is gutted of much over and above what comes from the CityLite database only. There is then code to make a php extension as well. It includes code to enable the API to tap into the time zone data of the City/CityLite databases.

I diffed the GeoIP Extended API against 1.4.6 from Maxmind, followed the yellow brick road modifying a Maxmind original, debuged some issues, and built as a static lib instead of dll.

I then added the functionality in mod_geoip to include GEOIP_TIME_ZONE_NAME into the environment to be picked up by a scripting language of your pleasure. It was some trial and error but to my complete surprise, It works!

Of course, mileage/kilometers will vary according to the errors of the citylite db.
Since there is only one function in the timezone.c file of the API, it looks like time zones need to be chosen by country and region, it may not be able to be retrieved from ipv6, unless a more general time zone works.

So I need to dig around and experiment some more, I see three other things  that might be worth extracting from the db

Title: Re: Adding a new trick to an old dog
Post by: Gregg on January 10, 2011, 10:40:28 PM
Upon further inspection, nothing needs to be changed in the geoip api, it just needs to be built to include everything. The addition of the timezone to mod_geoip is working good, I'm now trying to get the country code3 to it but it's being a challenge. 
Title: Re: Adding a new trick to an old dog
Post by: Sob on January 10, 2011, 11:23:11 PM
I'm now trying to get the country code3 to it but it's being a challenge.
You mean finding lines that contain some form of "country_code", duplicating them and appending "3" to newly added ones? That doesn't sound like a big challenge. ;)
Title: Re: Adding a new trick to an old dog
Post by: Gregg on January 11, 2011, 03:45:59 AM
No, I found those in the API, getting them used in mod_geoip is what my first attempt had problems with, but that was my first attempt. It compiled clean but crashed Apache. I haven't attempted it again yet but will soon.
Title: Re: Adding a new trick to an old dog
Post by: Sob on January 11, 2011, 04:28:46 AM
I did exactly what I wrote before:

http://web.hisoftware.cz/sob/download/mod_geoip2_1.2.5-cc3.patch

And it works fine here (I admit that testing was very limited).
Title: Re: Adding a new trick to an old dog
Post by: Gregg on January 11, 2011, 05:47:03 AM
I have the first part of that which works only with the country database. I tried same in the case for the city & region DBs and it was seizing Apache, but I figured out why that was happening so all is good.

$ENV{'GEOIP_REGION_NAME'} = California
$ENV{'GEOIP_COUNTRY_CODE3'} = USA
$ENV{'GEOIP_LATITUDE'} = 32.797699
$ENV{'GEOIP_COUNTRY_CODE'} = US
$ENV{'GEOIP_COUNTRY_NAME'} = United States
$ENV{'GEOIP_CONTINENT_CODE'} = NA
$ENV{'GEOIP_DMA_CODE'} = 825
$ENV{'GEOIP_CITY'} = San Diego
$ENV{'GEOIP_AREA_CODE'} = 619
$ENV{'GEOIP_REGION'} = CA
$ENV{'GEOIP_LONGITUDE'} = -117.132202
$ENV{'GEOIP_TIME_ZONE_NAME'} = America/Los_Angeles

Monday, Jan 10, 2011 9:00 PM PST
Tuesday, Jan 11, 2011 05:00:44 UTC

Even though the Region db uses a different struct (*giregion), I'm assuming it has the same stuff since it spits out a county code as well. That was where the crashing was coming from. There's no giregion->country_code3 and the way I was trying to obtain it (like in the country db) was problematic. Duh me!

Of course, I never plan to buy a region db so I'll never be able to test it. So I guess the only question left is, does it show the right city info and if so, the right time zone?

http://www.apachehaus.net/

Is it showing the proper time for you there on the left side?
Title: Re: Adding a new trick to an old dog
Post by: Sob on January 11, 2011, 04:03:14 PM
I tried only free country and city databases as I don't have access to paid ones.

Quote
Is it showing the proper time for you there on the left side?
Tried it from two locations and have the same results. It finds the correct location, but on the left under hostname there's "[an error occurred while processing this directive]" and the same is on the right under GMT Time and Server Time.
Title: Re: Adding a new trick to an old dog
Post by: mario on January 12, 2011, 09:25:02 PM
From home it didn't work with the location, well the ISP is somewhere ;D  Form de mirror with lynx it shows the correct location :D
Title: Re: Adding a new trick to an old dog
Post by: Gregg on January 12, 2011, 09:44:01 PM
Yeah, it's not perfect outside the US at all. So I need to rethink it some. That was a quick & dirty script, DateTime module in Perl could use to be a bit more forgiving as well which is what caused the errors you got Sob.
Title: Re: Adding a new trick to an old dog
Post by: DnvrSysEngr on January 13, 2011, 02:04:12 AM
FYI -

When I went to www.apachehaus.net, it populated the following fields for me:

Hello visitor from Arvada, Colorado, United States
IP Address: 97.118.35.118
Hostname: 97-118-35-118.hlrn.qwest.net
Wednesday, Jan 12, 2011 5:58 PM MST

GMT Time:
Thursday, Jan 13, 2011 00:58:11 UTC

Server Time:
Wednesday, Jan 12, 2011 4:58 PM PST

Server Uptime:
1 day 0 hours 59 minutes 45 seconds

Apache Uptime:
1 days, 0 hrs, 50 mins, 542 secs

Is there anything specific I should be looking for?

Title: Re: Adding a new trick to an old dog
Post by: Gregg on January 13, 2011, 05:36:05 AM
Hi DnvrSysEngr,

What I cared about was that it got the timezone correct.

Are you in Colorado?
If so you're in MST which is correct regardless of what city.

MST = PST+1 & GMT-7 so your time looks correct and UTC looks correct.

Thanks for checking and reporting in  :)

Looks like I could use to do some work on Apache Uptime too, 542 seconds lol ::)
Title: Re: Adding a new trick to an old dog
Post by: DnvrSysEngr on January 13, 2011, 06:40:23 AM
Gregg:

545 seconds in a minute?  On another planet maybe? LOL

I should not say anything as coding is not my area of expertise.

Apache Uptime:
1 days, 5 hrs, 29 mins, 545 secs


I do like this script and would not mind having it to put on some of my customers sites.

 - Scott
Title: Re: Adding a new trick to an old dog
Post by: Gregg on January 13, 2011, 10:24:37 AM
Well, let me make it a little less prone to error first. I have a backup plan too, figure it out by country. Of course I can think of many countries that cover multiple time zones. I'm not worried about AU,CA & US as I know those are covered pretty well, although if you're in Adelaide, AU on optusnet.com.au you will show up as Melbourne or Brighton which is a 1/2 hour ahead of Adelaide. Thats cause of how Optus gives out IPs.

The ones that will not fair well I believe are China, Russia and the Ukraine. Each span at least 3 time zones or more. Picking one is going to be wrong in most cases but oh well.
Title: Re: Adding a new trick to an old dog
Post by: Sob on January 14, 2011, 04:53:30 PM
You may try GeoIP 1.4.7 beta, unlike 1.4.6 it has more countries divided into regions in timeZone.c.
Title: Re: Adding a new trick to an old dog
Post by: DnvrSysEngr on January 14, 2011, 06:22:21 PM
Update:

Today, when I went to www.apachehaus.net, it populated the following fields for me:

Hello visitor from (none), (none), United States
IP Address: 65.106.84.254
Hostname: 65.106.84.254.ptr.us.xo.net
[an error occurred while processing this directive]

GMT Time:
[an error occurred while processing this directive]
Server Time:
[an error occurred while processing this directive]Server Uptime:
2 days 17 hours 22 minutes 42 seconds

Apache Uptime:
0 days, 10 hours, 11 minutes, 48 seconds

Not as successful as last time.

 - Scott
 


Title: Re: Adding a new trick to an old dog
Post by: Gregg on January 14, 2011, 07:29:26 PM
@Dev,

Yes, I know why it's happening, the only thing I've done recently is fix the 5xx seconds. I should do, and most likely will do something about the rest of it this weekend.

@Sob, I compiled 1.4.7b9 a couple weeks ago, it was somewhat a nightmare. That first array declaration in geoipcity.c I think really made the compiler scream. For some reason it just refuses to add THAT_CONST to 4, but since that const = 6, temp_array[10] worked fine. The rest was easy.

Hmmm, so there's more there, then the region file should have more. The timezone requires country + region name to work or it spills out the (none), which is what DateTime in perl barfs on. I cannot understand why the author of it chose die if it's not correct, either return nothing or a warning, anything but die.
Title: Re: Adding a new trick to an old dog
Post by: Gregg on January 14, 2011, 08:08:20 PM
Interestingly, this is not the case.

They ripped 13k worth out of regionName but added 32k to timeZone, this might be interesting to try.
Title: Re: Adding a new trick to an old dog
Post by: Sob on January 18, 2011, 08:50:42 PM
That first array declaration in geoipcity.c I think really made the compiler scream.

I'm no expert, but it seems that C does not like const as array size. C++ has no objections against it. But adding /TP parameter to cl.exe (to force C++ mode) produce errors in other files (headers) that probably don't like to be compiled as C++.
I'm not sure how it's with other compilers (GeoIP is clearly not developed primarily using MSVC :) ). This kind of declaration may even be "variable length array", the new feature of C99 standard. And don't look for those in MSVC. :(
Title: Adding a new trick to an old dog
Post by: Seeriolojen on March 04, 2011, 06:29:40 PM
Hey guys

Say I have a number in the database e.g. 5

I have a variable in PHP thats 4.. I want to add that to the 5 in the database and make it 9...

Ratehr than getting the current number in the database, handling it with PHP, then putting the new one in..

Is there a way?

Thanks in advance

crE