Analyzing NGINX Logs with Linux Commands 📝🔍🐧

Dive into the power of NGINX log analysis! Uncover critical insights about your server's traffic, spot top-performing URLs, track unique visitors, and detect potential threats—all with straightforward Linux commands. Boost your server’s performance, enhance security, and gain a clear view of user behavior in just a few steps!

1. tail - Real-Time Monitoring 📡

Displays the last lines of a file and continuously updates in real time.

tail -f access.log

2. Count Total Hits (Requests) 📊

Calculate the total number of hits (requests) across your NGINX log file:

wc -l access.log

cat access.log | wc -l

3. Count Total Hits (Requests) per Day

Objective: Calculate the total number of requests for each day.

awk '{print $4}' access.log | cut -d: -f1 | tr -d '[' | sort | uniq -c | awk '{printf "%s %s\n", $2, $1}'

4. Get Requests Per Hour

awk -F'[:[]' '{print $3}' access.log | cut -d':' -f1 | sort | uniq -c

This command counts the number of requests made to your server per hour. It extracts the hour portion from the timestamps in the access log entries, sorts them, and then counts how many requests occurred for each hour. This can help you understand traffic patterns and identify peak usage times on your server.

5. Top 10 Frequent IPs 🚨

Find the top 10 IP addresses accessing your server:

awk '{print $1}' access.log | sort | uniq -c | sort -nr | head -10

This command lists the 10 IP addresses with the most requests, helping you identify high-traffic sources or potential issues.

Show the results in a table format with headers:

echo -e "Total Hits\tIP Address" && awk '{print $1}' access.log | sort | uniq -c | sort -nr | head -10 | awk '{printf "%-10s\t%s\n", $1, $2}'

This version displays the total hits and IP addresses in a clear table format for easy reading.

6. Count the Number of Unique IP Addresses 🚨

awk '{print $1}' access.log | sort | uniq | wc -l

This command counts the number of unique IP addresses that have accessed your server, providing insight into the diversity of traffic sources.

To display the total number of unique IP addresses with a message:

echo -n "Total unique IPs: " && awk '{print $1}' access.log | sort | uniq | wc -l

This version prints "Total unique IPs:" followed by the count, offering a more informative output.

7. Get a List of Unique IP Addresses 🚨

awk '{print $1}' access.log | sort | uniq

This extracts and displays all unique IP addresses that have accessed your server. Useful for identifying the sources of traffic without counting them.

8. Find the Top 10 Requested URLs 🚨

awk -F'"' '{print $2}' access.log | awk '{print $2}' | sort | uniq -c | sort -nr | head -10

This command extracts and counts the most frequently requested URLs from the access log. It sorts the URLs by the number of requests in descending order and displays the top 10, helping you understand which resources are most popular on your server.

9. Calculate Total Bytes 🚨

awk '{sum += $10} END {print "Total Bytes: " sum}' access.log

This command sums up the response sizes in bytes and displays the total.

Convert to Megabytes You can directly convert the total to MB by dividing it by 1048576 (1 MB = 1024 * 1024 bytes).

awk '{sum += $10} END {print "Total Bytes: " sum, "Bytes\nTotal MB: " sum/1048576 " MB"}' access.log