My boss has asked me to generate a simple report on list of IP addresses which access to our website yesterday with a “ref=1” keyword. What I need most is the Apache access logs which located under /etc/httpd/domlogs/mydomain.net (default Apache logs for cPanel servers).
From the log files, I will need to extract yesterday’s log which is 22 November 2011 and output them into another file called 20111122.txt. Command as below:
$ cat /etc/httpd/logs/domlogs/mydomain.net | grep "22/11/2011" > 20111122.txt |
Next, I extract the logs to match the keyword (ref=1) and output to another file:
$ cat /etc/httpd/logs/domlogs/20111122.txt | grep '\bref=1\b' > 20111122_keyword.txt |
Once the keyword extract, I will do the counting and generate a report in http root folder so I can view it via web browser:
$ cat /etc/httpd/logs/domlogs/20111122_keyword.txt | awk '{print $1}' | cut -d: -f1 | sort | uniq -c | sort -n > /home/mydomain/public_html/ip_report.txt |
After that, I view the generate report via web browser at http://mydomain.net/ip_report.txt and following output should appear:
---------------- 818 118.100.150.10 821 175.142.245.213 824 124.43.99.74 829 137.186.89.211 835 92.101.85.191 855 41.78.17.172 855 85.65.48.168 855 87.69.176.55 858 201.241.218.180 880 14.99.3.152 889 190.197.28.142 889 41.72.10.60 902 201.248.128.162 905 123.176.15.119 911 115.132.145.225 911 115.66.95.153 918 151.27.94.55 939 189.115.158.192 947 65.49.71.172 949 91.185.109.175 952 186.146.220.235 980 85.230.95.192 1019 41.236.216.132 1030 38.111.147.83 1040 99.66.113.53 1062 41.70.178.190 1071 175.144.133.164 1107 41.78.17.186 ---------------- |
The left column is number of IP counted while the right column is the IP address respectively. This simple report should be enough to suit what my boss wants!