Install analog + report magic
#########################################################
# #
# OS: Fedora Core 6 #
# Date: 2007.5.15 #
# Server: Apache/2.2.3 #
# Homepage: http://www.analog.cx/ (analog) #
# http://www.reportmagic.org/ (Report magic) #
#########################################################
wget http://www.archive.hk/analog/analog-6.0.tar.gz
tar xvfzp analog-6.0.tar.gz
cd analog-6.0
vi analog.cfg
# Configuration file for analog 6.0
# See http://www.analog.cx/
#
# Here are a couple of configuration commands to get you started. Add any more
# you like from the Readme.
#
# Lines starting with # are comments.
#
# There is a much more extensive configuration file in examples/big.cfg
#
# If you need a LOGFORMAT command (most people don't -- try it without first!),
# it must go here, above the LOGFILE commands.
LOGFILE /var/log/httpd/access_log.*
#LOGFILE logfile.log
# LOGFILE /old/logs/access_log.*
OUTFILE Report.html
HOSTNAME "[Analog test for gpse001.gopetslive.com]"
#
# You probably don't need to change anything below this until you are
# experienced.
#
# ERRFILE errors.txt
# REQINCLUDE pages
REQLINKINCLUDE pages
REFLINKINCLUDE *
REDIRREFLINKINCLUDE *
FAILREFLINKINCLUDE *
SUBBROW */*
SUBTYPE *.gz,*.Z
# OSCHARTEXPAND Windows
# Add whichever of these types of pages you have on your server, or others.
# PAGEINCLUDE *.shtml
# PAGEINCLUDE *.asp
# PAGEINCLUDE *.jsp
# PAGEINCLUDE *.cfm
# PAGEINCLUDE *.pl
# PAGEINCLUDE *.php
# More SEARCHENGINE commands can be found at
# http://www.analog.cx/helpers/#conffiles
SEARCHENGINE http://*google.*/* q,as_q,as_epq,as_oq
SEARCHENGINE http://*altavista.*/* q
SEARCHENGINE http://*yahoo.*/* p
SEARCHENGINE http://*lycos.*/* query,wfq
SEARCHENGINE http://*aol.*/* query
SEARCHENGINE http://*excite.*/* search
SEARCHENGINE http://*go2net.*/* general
SEARCHENGINE http://*metacrawler.*/* general
SEARCHENGINE http://*msn.*/* q,MT
SEARCHENGINE http://*netscape.*/* search
SEARCHENGINE http://*looksmart.*/* key
SEARCHENGINE http://*webcrawler.*/* qkw,search,searchText
SEARCHENGINE http://*overture.*/* Keywords
SEARCHENGINE http://*teoma.*/* q
SEARCHENGINE http://*infospace.*/* qkw
SEARCHENGINE http://*alltheweb.*/* q
SEARCHENGINE http://*dogpile.*/* q
SEARCHENGINE http://*ask.*/* q,ask
SEARCHENGINE http://*alltheweb.*/* query
SEARCHENGINE http://*northernlight.*/* qr
SEARCHENGINE http://*nlsearch.*/* qr
SEARCHENGINE http://*dmoz.*/* search
SEARCHENGINE http://*/netfind* query
SEARCHENGINE http://*/pursuit query
ROBOTINCLUDE REGEXPI:robot
ROBOTINCLUDE REGEXPI:spider
ROBOTINCLUDE REGEXPI:crawler
ROBOTINCLUDE Googlebot*
ROBOTINCLUDE msnbot*
ROBOTINCLUDE Infoseek*
ROBOTINCLUDE Scooter*
ROBOTINCLUDE *Slurp*
ROBOTINCLUDE *Validator*
ROBOTINCLUDE Ultraseek*
TYPEALIAS .html ".html [Hypertext Markup Language]"
TYPEALIAS .htm ".htm [Hypertext Markup Language]"
TYPEALIAS .shtml ".shtml [Server-parsed HTML]"
TYPEALIAS .ps ".ps [PostScript]"
TYPEALIAS .gz ".gz [Gzip compressed files]"
TYPEALIAS .tar.gz ".tar.gz [Compressed archives]"
TYPEALIAS .jpg ".jpg [JPEG graphics]"
TYPEALIAS .jpeg ".jpeg [JPEG graphics]"
TYPEALIAS .gif ".gif [GIF graphics]"
TYPEALIAS .png ".png [PNG graphics]"
TYPEALIAS .txt ".txt [Plain text]"
TYPEALIAS .cgi ".cgi [CGI scripts]"
TYPEALIAS .pl ".pl [Perl scripts]"
TYPEALIAS .css ".css [Cascading Style Sheets]"
TYPEALIAS .class ".class [Java class files]"
TYPEALIAS .pdf ".pdf [Adobe Portable Document Format]"
TYPEALIAS .zip ".zip [Zip archives]"
TYPEALIAS .hqx ".hqx [Macintosh BinHex files]"
TYPEALIAS .exe ".exe [Executables]"
TYPEALIAS .wav ".wav [WAV sound files]"
TYPEALIAS .avi ".avi [AVI movies]"
TYPEALIAS .arc ".arc [Compressed archives]"
TYPEALIAS .mid ".mid [MIDI sound files]"
TYPEALIAS .mp3 ".mp3 [MP3 sound files]"
TYPEALIAS .doc ".doc [Microsoft Word document]"
TYPEALIAS .rtf ".rtf [Rich Text Format]"
TYPEALIAS .mov ".mov [Quick Time movie]"
TYPEALIAS .mpg ".mpg [MPEG movie]"
TYPEALIAS .mpeg ".mpeg [MPEG movie]"
TYPEALIAS .asp ".asp [Active Server Pages]"
TYPEALIAS .jsp ".jsp [Java Server Pages]"
TYPEALIAS .cfm ".cfm [Cold Fusion]"
TYPEALIAS .php ".php [PHP]"
TYPEALIAS .js ".js [JavaScript code]"
TYPEALIAS .swf ".swf [Flash files]"
cd src
vi anlghead.h
/*** analog 6.0 http://www.analog.cx/ ***/
/*** This program is copyright (c) Stephen R. E. Turner 1995 - 2004 except as
*** stated otherwise.
***
*** This program is free software. You can redistribute it and/or modify it
*** under the terms of version 2 of the GNU General Public License, which you
*** should have received with it.
***
*** This program is distributed in the hope that it will be useful, but
*** without any warranty, expressed or implied. ***/
/*** anlghead.h: first header file ***/
/*** This file contains the user-definable options which you are most likely to
*** want to change. Edit any of the #define lines according to your desires.
*** However, all of them except MANDATORYCONFIGFILE can be changed at run
*** time as well. ***/
/*** These settings are overridden by any in your configuration file. ***/
/*** DON'T REMOVE THE #'S -- THEY'RE NOT COMMENTS! ***/
#ifndef ANLGHEAD_H
#define ANLGHEAD_H
#ifndef HOSTNAME
#define HOSTNAME "[Web Report for gpse001.gopetslive.com]"
/*#define HOSTNAME "[my organisation]"*/
/* The name of your organisation or web server. This is used for printing
at the top of the output. */
#endif
#ifndef HOSTURL
#define HOSTURL "none"
/* The URL of your host's home page, for linking to at the top of the
output; use "none" for no linking. */
#endif
#ifndef LOGFILE
#define LOGFILE "/var/log/httpd/access_log.*"
/*#define LOGFILE "logfile.log"*/
/* The name of the default logfile. Can be a list, separated by commas
(but no spaces), and can include wild cards. */
#endif
#ifndef OUTFILE
#define OUTFILE "stdout"
/* The default file for output. Use "stdout" for stdout. */
#endif
#ifndef IMAGEDIR
#define IMAGEDIR "images/"
/* URL of the directory where the images for the graphical reports live.
The URL can be absolute, or relative to the output page: e.g., just the
empty string "" for the same directory as the output page. */
#endif
#ifndef LOGO
#define LOGO "analogo"
/* the file where the analog logo lives: within the IMAGEDIR unless it
starts with / or contains :// . You can put another logo in here (your
organisation's logo, for example -- but if you do, change the LOGOURL
below too). Use "none" for no logo. */
#endif
#ifndef LOGOURL
#define LOGOURL "http://www.analog.cx/"
/* Where the logo links to, in XHTML output. */
#endif
#ifndef STYLESHEET
#define STYLESHEET "none"
/* The URL of your style sheet. Use "none" for no style sheet. */
#endif
#ifndef DEFAULTCONFIGFILE
#define DEFAULTCONFIGFILE "analog.cfg"
/* The name of the default configuration file. */
#endif
#ifndef MANDATORYCONFIGFILE
#define MANDATORYCONFIGFILE "none"
/* A mandatory configuration file; read after all other arguments so
overrides everything. Program exits if it is not found. Use "none"
for none. */
#endif
/* Finally, some directories where particularly files are looked for (you
MUST include the trailing slash in the directory name!). You can also use
"" to mean "the current working directory at the time you run the program".
*
If any of these directories is defined to be NULL, analog will attempt to
deduce the directory of the analog binary from the command name you used,
and use that directory; or the current directory if it couldn't deduce a
directory. This may or may not work, so it's better to specify a location
explicitly here if you know where the files will be kept. */
#ifndef LANGDIR
#define LANGDIR NULL
#endif
/* Directory where the language files live. Actually, if this one is defined
to be NULL, they will be looked for inside the "lang" subdirectory of the
directory containing the analog binary. */
#ifndef CONFIGDIR
#define CONFIGDIR NULL
#endif
/* Directory containing configuration files. */
#ifndef LOGSDIR
#define LOGSDIR NULL
#endif
/* Directory containing logfiles. */
#ifndef CACHEDIR
#define CACHEDIR NULL
#endif
/* Directory containing cache files. */
#ifndef OUTDIR
#define OUTDIR NULL
/*#define OUTDIR NULL*/
#endif
/* Directory for writing output files: must already exist. */
#ifndef HEADERDIR
#define HEADERDIR NULL
#endif
/* Directory containing header and footer files. */
#ifndef DNSDIR
#define DNSDIR NULL
#endif
/* Directory containing DNS files. */
#ifndef LOCKDIR
#define LOCKDIR NULL
#endif
/* Directory containing the DNS lock file. */
#ifndef ERRDIR
#define ERRDIR NULL
#endif
/* Directory containing the ERRFILE. */
#include "anlghea2.h"
/* You can edit anlghea2.h as well. But the options in there are ones which
you're unlikely to want to change now. */
#endif /* ANLGHEAD_H */
make
cd ..
./analog
wget http://www.reportmagic.org/rmagic-2.21.tar.gz
tar xvfzp rmagic-2.21.tar.gz
mv rmagic-2.21 rmagic
cd rmagic-2.21
perl Install.PL
------------------------------------------------------
Report Magic 2.21 installation script
------------------------------------------------------
This script will install or upgrade the following modules:
GD
Config::IniFiles
File::Spec
HTML::Tagset
HTML::Parser
GD::Text
GD::Graph
GD::Graph3d
File::Temp
The CPAN interface will now attempt to locate your nearest archive
and install modules from there. If this is the first time you have
run the CPAN shell, you will need to provide some configuration
information. If, when asked for your favorite CPAN site, you are not
given a list to choose from you can type in 'ftp://ftp.cpan.org/CPAN'.
Press [ENTER] to continue.
:
:
:
Installation complete.
For help on using Report Magic see /usr/local/bin/rmagic-2.21/docs/index.html.
You must agree to the license agreement at /usr/local/bin/rmagic-2.21/docs/license.html before using Report Magic.
vi analog.cfg
# Configuration file for analog 6.0
# See http://www.analog.cx/
#
# Here are a couple of configuration commands to get you started. Add any more
# you like from the Readme.
#
# Lines starting with # are comments.
#
# There is a much more extensive configuration file in examples/big.cfg
#
# If you need a LOGFORMAT command (most people don't -- try it without first!),
# it must go here, above the LOGFILE commands.
LOGFILE /var/log/httpd/access_log.*
#LOGFILE logfile.log
# LOGFILE /old/logs/access_log.*
#OUTFILE /var/www/html/Report/Report.html
OUTPUT COMPUTER
OUTFILE /usr/local/src/report.dat
HOSTNAME "[Analog test for gpse001.gopetslive.com]"
#
# You probably don't need to change anything below this until you are
# experienced.
#
cp rmagic.sample rmagic.ini
vi rmagic.ini
#
# Sample Report Magic settings file.
# For details on commands see the documentations, starting at docs/index.html
# or see the comments in the rmagic.ini file in each language directory
# (e.g lang/en/rmagic.ini)
#
[statistics]
File_In = report.dat ====
Frame_File_Out = index.html
Format= XHTML 1.0
[reports]
Active_Column=R
File_Out = /var/www/html/Report/ ====
Title_Font = Helvetica,Arial,sans-serif
Title_BG_Color = #000000
Title_Font_Color = #FFCC00
Data_Font = Helvetica,Arial,sans-serif
Data_Color_1 = #D0FFD0
Data_Color_2 = #F0FFF0
Data_Total_Color = #FFCC66
Data_Header_Color = #9999CC
[QUICK]
Rows = ALL
[graphs]
BG_Color = #FFFFCC
[navigation]
File_Out = nav.html
[website]
Title = Web Statistics for Za.com ====
Webmaster = webmaster@za.com ====
Base_URL = http://www.za.com ====
analog access_log.1
analog: analog version 6.0/Unix
analog: Warning R: Turning off empty Search Word Report
(For help on all errors and warnings, see docs/errors.html)
./rmagic.pl
Report Magic 2.21
Copyright (C) 1999-2003 Wadsack-Allen. All rights reserved.
rmagic: Beginning report creation (Wed May 16 18:09:57 2007).
rmagic: --> Settings file: ./rmagic.ini.
rmagic: --> Input data file: /usr/local/src/report.dat.
rmagic: --> Output path: /var/www/html/Report/.
rmagic: --> Output frameset file: /var/www/html/Report/index.html.
rmagic: --> Output navigation file: /var/www/html/Report/nav.html.
rmagic: Creating report file in /var/www/html/Report/GENERAL.html.
rmagic: Creating report file in /var/www/html/Report/MONTHLY.html.
rmagic: ... Creating graph file in /var/www/html/Report/MONTHLY_line.png.
rmagic: Creating report file in /var/www/html/Report/DAILYSUM.html.
rmagic: ... Creating graph file in /var/www/html/Report/DAILYSUM_bar.png.
rmagic: Creating report file in /var/www/html/Report/HOURLYSUM.html.
rmagic: ... Creating graph file in /var/www/html/Report/HOURLYSUM_bar.png.
rmagic: Creating report file in /var/www/html/Report/DOMAIN.html.
rmagic: Creating report file in /var/www/html/Report/ORGANISATION.html.
rmagic: Creating report file in /var/www/html/Report/OSREP.html.
rmagic: Creating report file in /var/www/html/Report/STATUS.html.
rmagic: Creating report file in /var/www/html/Report/SIZE.html.
rmagic: ... Creating graph file in /var/www/html/Report/SIZE_bar.png.
rmagic: Creating report file in /var/www/html/Report/FILETYPE.html.
rmagic: ... Creating graph file in /var/www/html/Report/FILETYPE_bar.png.
rmagic: Creating report file in /var/www/html/Report/DIRECTORY.html.
rmagic: ... Creating graph file in /var/www/html/Report/DIRECTORY_bar.png.
rmagic: Creating report file in /var/www/html/Report/REQUEST.html.
rmagic: Creating report file in /var/www/html/Report/QUICK.html.
rmagic: Report creation complete (Wed May 16 18:09:58 2007).
analog의 본능에 충실한 모습.
Report Magic라는 옷을 입힌후의 변화된 모습..^0^
analog help page.
http://www.analog.cx/helpers/#sched (auto scripts)
http://www.analog.cx/helpers/#dns (DNS lookups)
아쉽게도 DNS lookup이 설치시 자동으로 off되어있는것 같은데 on시키는 방법을 못 찾겠다.-_-;
이곳저곳 돌아다녀보니 이를 도와주는 툴이 있어 도전..^^*
Site: http://www.summary.net/soft/dnstran.html
DNSTran I was able to process a 23.6Meg log file in 18 minutes (DNSTran and Analog) as compared to a time of about 15 hours using Analog alone.
빠르다고 자랑이 이만저만이 아닌..ㅎㅎ
wget http://www.summary.net/soft/dns-x86.tar.gz
tar xvfzp dns-x86.tar.gz
mv dnstran1.5.2/ dnstran (개인적으로 버전붙어 있는거 이상하게 싫다.-_-;;;)
cd dnstran
vi dnstran.cfg
# dnstran.cfg - DNSTran configuration file
# For use with version 1.5.x of DNSTran
#
# Translate the log file? (yes or no)
translate yes =====
#
# Compress the translated log file? (no, gzip, or bzip2)
compress gzip
#
# Level of compression to use 0-9, (significant speed and size differences)
# 0 means very little compression and fast run times, 9 means lots of
# compression and slow run times. 6 is a reasonable balance of both.
level 9
#
# Verbose progress messages? (yes or no)
verbose yes =====
#
# DNSTran will read and convert Analog dns cache files,
# even if "private" is set to "yes" below.
#
# Private (smaller and extra info) dns cache file format? (yes or no)
private no
#
# Time offset
# Analog 2.1.1d or older use -89327520
# Analog 2.1.1e or newer but before 2.90 use 36816480
# Analog 2.90 or newer use 0
# Some versions of Analog (mostly 3.x) use localtime and others use universal
# time. DNSTran always uses universal time, but the difference will not
# be significant as it only affects the expire time by a partial day.
# Some versions of Analog are additionally off by one day, using Dec 31,
# 1969 instead of Jan 1, 1970 as their base time. Again this is not that
# important since even a whole day difference in the expire times is not
# important.
offset 0
#
# Time divisor
# Analog 2.1.1x or older use 1
# Analog 2.90 or newer use 60
divisor 60
#
# File name for the dns cache
cache dnscache
#
# Number of 1/60th secs to wait between lookups (0 - 120)
# This setting allows you to slow down DNSTran if it is overloading your DNS
# server. The setting is approximate, DNSTran may run a little faster or
# slower. 0-full speed 1-limit to 60 lookups per second
# 2-limit to 30 lookups per second 3-limit to 20 lookups per second
# 4-limit to 15 lookups per second 5-limit to 12 lookups per second
# 6-limit to 10 lookups per second etc.
#
wait-ticks 0
#
# Expire sucsessful lookups after N days not used for translations,
# default 9 weeks (1-365)
expire-good 63
#
# Expire failed lookups after N days since lookup, default 18 weeks (1-365)
expire-failed 126
#
# Failed lookups are retried three times, each N days since last attempt,
# default 4 days (1-35) This feature is only enabled if private dns cache
# file format is turned on (set to yes above).
retry-failed 4
#
# Newline format for translated output files: mac, unix, or dos (dos includes
# Windows)
newlines unix
#
# Replace the original log with the translated result? (yes or no)
exchange no
#
# Delete input file after translation? (yes or no)
# Ignored when replacing the input file with the translated file
delete-input no
#
# end of dnstran.cfg
./dnstran -help
DNSTran version 1.5.2
Copyright (c) 1997-2002 by Jason T. Linhart,
Portions are Copyright 1998 by the Massachusetts Institute of Technology.
Usage 'DNSTran <options> <log-file-list>'
where <options> may include:
-help - display this message
-settings <config-file> - use the specified file as the config file.
This must be the first argument if used.
-translate - translate the log file
-verbose - display progress messages
-cache - <cache-file> specify where to save the DNS name cache file
-level <digit> - specify the compression level
-offset <amount> - offset times in the dnscache file by this amount
-divisor <amount> - divide times by this in the dnscache file
-good <days> - expire good lookups after N days
-failed <days> - expire failed lookups after N days
-retry <days> - retry failed lookups after N days
-normal - don't compress translated log files (default is GZip)
-bzip2 - compress translated log files using BZip2
-private - use a compressed dns cache file with extra information
-exchange - replace original log with translated result
-z <ticks> - Number of 1/60th secs between lookups
-mac newlines
-unix newlines
-windows/dos newlines
-kill - delete input file after translation
and <log-file-list> is a set of text files to lookup IP addresses from.
Complete.
./dnstran ../access_log.1
'dnscache' not found, creating it.
Looking up '../access_log.1'
Read 10,496 lines (50%), 249 lookups, 1049 lines/sec, 24.9 lookups/sec.
Read 10,752 lines (51%), 250 lookups, 537 lines/sec, 12.5 lookups/sec.
Read 16,896 lines (77%), 304 lookups, 563 lines/sec, 10.1 lookups/sec.
Read 19,200 lines (87%), 353 lookups, 480 lines/sec, 8.8 lookups/sec.
Read 21,760 lines (99%), 393 lookups, 435 lines/sec, 7.9 lookups/sec.
Looked up 394 IPs, used 0 from cache, total of 394 in cache,
28,963 duplicates, 21,953 lines in 0:51!
Waiting for last few lookups to complete.
Wrote 394 cache entries, 0 expired, 124 (31%) were unresolved.
Translating '../access_log.1'
Translated 5,559 IPs, 23,798 unresolved, 21,953 lines in 0:01!
Complete.
more dnscache
19656161 222.166.160.28 cm222-166-160-28.hkcable.com.hk
19656161 64.127.120.131 *
19656161 24.21.0.45 c-24-21-0-45.hsd1.mn.comcast.net
19656161 198.54.202.195 ctb-cache5-vif1.saix.net
19656161 61.18.170.214 cm61-18-170-214.hkcable.com.hk
19656161 24.20.78.127 *
19656161 124.6.173.227 *
19656161 68.23.8.197 adsl-68-23-8-197.dsl.ipltin.ameritech.net
19656161 61.18.170.205 cm61-18-170-205.hkcable.com.hk
19656161 222.166.160.196 cm222-166-160-196.hkcable.com.hk
19656161 61.18.170.19 cm61-18-170-19.hkcable.com.hk
19656161 84.177.56.76 p54b1384c.dip0.t-ipconnect.de
19656161 202.57.111.253 *
19656161 62.171.194.41 proxy-21.swgfl.ifl.net
19656161 61.250.88.70 *
19656161 165.30.11.1 *
19656161 207.200.116.199 cache-ntc-ad05.proxy.aol.com
19656161 142.76.1.62 *
19656161 207.200.116.13 cache-ntc-aa09.proxy.aol.com
19656161 210.5.92.196 210.5.92.196.pldt.net
19656161 222.166.160.60 cm222-166-160-60.hkcable.com.hk
vi analog.cfg
# Configuration file for analog 6.0
# See http://www.analog.cx/
#
# Here are a couple of configuration commands to get you started. Add any more
# you like from the Readme.
#
# Lines starting with # are comments.
#
# There is a much more extensive configuration file in examples/big.cfg
#
# If you need a LOGFORMAT command (most people don't -- try it without first!),
# it must go here, above the LOGFILE commands.
LOGFILE access_log
# LOGFILE /old/logs/access_log.*
#OUTFILE /var/www/html/Report/Report.html
OUTPUT COMPUTER
OUTFILE /usr/local/src/report.dat
DNS READ ===== 이거 찾느라구..-_-;;
DNSFILE /usr/local/src/dnstran/dnscache =====
# HOSTNAME "[my organisation]"
근데 결과는 기대 이하다..ㅠㅠ
볼품 없이 이랬던 놈이..
이렇게 변했네요.
제가 기대했던 결과는 Domain이 다 나올꺼라 생각했었는데..
하여간 성공..-_-v
analog + report magic 은 interface는 이쁘게 나오지만 이렇게 하기 위해서는 최소 3개의 process가 필요하게 된다.
1. ip lookup
2. analog
3. report magic
이다.
이거 원 번거로와서.....