Visualize NGiNX log with GoogleMap

1. Visualize NGiNX log with GoogleMap

3. purpose ! NGiNX ! !

5. ! Internet ! ! GoogleMap

6. ! ngx_http_geoip_module ! Google BigQuery ! () GeoFUSE

7. GeoFUSE? ! GPL ! mbasa/GeoFuse GitHub : https://github.com/mbasa/geofuse ! GeoFUSE : http://geofuse.georepublic.net/geofuse/ ! ! (GUI)

10. overwiew GeoFUSE Big QueryNGiNX 1.Logging 2.Transfer 3.Query 4.CSV

11. 1.Logging ! NginxGeoIP ! GeoIP C APIGeoIP Datale ! nginx--with-http_geoip_module ! Nginx.confGeoIP Datale ! GeoIPIP http { geoip_country /usr/local/share/GeoIP/GeoIP.dat; geoip_city /usr/local/share/GeoIP/GeoIPCity.dat; real_ip_header X-Forwarded-For; set_real_ip_from 10.0.0.0/8; log_format ltsv "geoip_country_name:" "$geoip_city_country_name" "t" # "geoip_country_code3:" "$geoip_city_country_code3" "t" # JPN/USA "geoip_city:" "$geoip_city" "t" # "geoip_latitude:" "$geoip_latitude" "t" # "geoip_longitude:" "$geoip_longitude" # ; }

12. 2.Transfer ! NginxBigQuery ! uentd ! in_tail ! uent-plugin-google-cloud ! NGiNXltsv ! BigQueryjson ()

13. 3.Query ! textPayload json ! JSON_EXTRACT JSON_EXTRACT_SCALAR ! (SUM,AVG,MIN,MAX) SELECT ROW_NUMBER() OVER (ORDER BY count DESC) AS ID, COUNT (*) AS count, JSON_EXTRACT_SCALAR(textPayload,'$.geoip_longitude') AS lon, JSON_EXTRACT_SCALAR(textPayload,'$.geoip_latitude') AS lat, AVG(JSON_EXTRACT_SCALAR (textPayload,'$.response_time')) AS avelage_time FROM [HOGE.nginx_access_20150402] GROUP BY lat,lon ORDER BY count DESC

14. 4.csv ! BigQuerycsv ! Geo-FUSE ! Geo-FUSE ! ! ID ! lon.lat() ID,count,avelage_time,lon,lat 1,3135686,0.2498652794178637,139.6900,35.6900 2,1596097,0.2536400474791719,139.7514,35.6850 3,182082,0.23747259653015085,135.5200,34.6864 4,117914,0.22794176617882336,121.5250,25.0392

16. ! EC ( IP) ! (Wi-)