AWStats Version which this Document is based on: 6.4

Prefix: AWStats “include” statement does not work as I would an include statement expect to work so I do not use it. I do not know how it works…

What I have:

• /etc/awstats/awstats.conf → A default config file with default settings
• /etc/awstats/awstats.conf.local → A local config file containing common local settings for every domain

What I want:

• One config per domain generated automatically using default- and common settings and and a template

for per domain settings

  - 
  -  domain specific settings
  - 

SiteDomain="%DOMAIN%"
HostAliases="%ALIASES%"
LogFile="%LOGFILE%"
AllowAccessFromWebToFollowingAuthenticatedUsers="%USERS%"

Example: links2linux.de File: /etc/awstats/domain.conf.d/links2linux.de.conf

  -  awstats config for links2linux.de
AW_DOMAIN="links2linux.de"
AW_ALIASES="wiki.links2linux.de"
AW_ACCESS_LOG="/var/log/apache/access_links2linux.de.log"
AW_USER="aw-user $AW_SUPERUSER"

Whenever you change the template, the common or default settings you only have to run this script again to have fresh configs.

  - !/bin/sh
  - 
  -  create awstats configs
  - 

AW_SUPERUSER="awstats-root"

find domain.conf.d/*.conf | while read CONFIG; do
<code>
  . $CONFIG
  echo -n "Creating awstats.$AW_DOMAIN.conf ... "
  echo -n "" > awstats.$AW_DOMAIN.conf
  for f in awstats.conf awstats.conf.local; do
    echo "#" >> awstats.$AW_DOMAIN.conf
    echo "# Include $f" >> awstats.$AW_DOMAIN.conf
    echo "#" >> awstats.$AW_DOMAIN.conf
    sed 's/#.*//;s/^[[|\t]]*//;s/[[|\t]]*$//' $f | sort -u >> awstats.$AW_DOMAIN.conf
  done
  cat template \
    | sed "s|%DOMAIN%|$AW_DOMAIN|" \
    | sed "s|%ALIASES%|$AW_ALIASES|" \
    | sed "s|%LOGFILE%|$AW_ACCESS_LOG|" \
    | sed "s|%USERS%|$AW_USER|" \
    >> awstats.$AW_DOMAIN.conf
    echo "OK"

done </code>

Imagine you changed some common stats parameters like enabling a new plugin and now you want to recreate as much stats as possible (all stats where logfiles still exist from). This Script does this automatically:

  - !/bin/sh
  - 
  -  recreate awstats for a specific domain
  - 
  -  what does it do?
  - 
  -  what we have:
  -    - awstats for a sepcific timeframe, maybe much greater than 1 year
  -    - max last ~52 logfiles ~= 1 year
  - 
  -  what we can do:
  -    1) check for which timeframe we have logs now
  -    2) remove awstats for that timeframe we detected in 1)
  -    3) let awstats create stats from all logfiles

DOMAIN="$1"
LOG="$2"

  -  try to find LOG automatically if not defined
test "$LOG" || {
<code>
  echo "Logfile not specified, trying to find it automatically (in domain.conf.d)"
  if [[|-f domain.conf.d/$DOMAIN.conf ]]; then
    . domain.conf.d/$DOMAIN.conf
    if [[|"$AW_ACCESS_LOG" ]]; then
      echo "Logfile found: $AW_ACCESS_LOG"
      LOG="$AW_ACCESS_LOG"
    else
      echo "Nothing found"
    fi
  fi

}

test “$DOMAIN” || { echo “Syntax: ./me /path/to/access_log example.com ”; exit 1; } test “$LOG” || { echo “Syntax: ./me /path/to/access_log example.com ”; exit 1; } test -f “$LOG” || { echo “$LOG not found”; exit 1; }

echo “Analyzing existing logfiles…” DATA_DIR=”/var/lib/awstats”

1. 1) check which is the oldest monthe we have complete logs for

OLDEST_LOG=“$(ls -1 “$LOG”-* | head -1)”

1. we do not know when a domain started to exist
2. so we assume that the month from the oldest logline +1 is
3. the oldest month we have complete
5. (we take 'zless' here because it reads gzipped and unzipped files)

OLDEST_LOGLINE=“$(zless $OLDEST_LOG | head -n1)” OLDEST_DATE_STRING=“$(echo “$OLDEST_LOGLINE” | sed 's|.*\\(.*\)\.*|\1|;s|/| |g;s/:/ /')” OLDEST_MONTH=“$(date +%m -d “$OLDEST_DATE_STRING”)” OLDEST_YEAR=“$(date +%Y -d “$OLDEST_DATE_STRING”)”

echo “Oldest logline is from $OLDEST_YEAR/$OLDEST_MONTH ($OLDEST_DATE_STRING)” if "$OLDEST_MONTH" == "12" ; then

  # set jan in the following year
  OLDEST_COMPLETE_MONTH="01"
  OLDEST_YEAR="$((OLDEST_YEAR+1))"

else

  if [[|"${OLDEST_MONTH:0:1}" == "0" ]]; then
    # remove leading zero
    OLDEST_MONTH=${OLDEST_MONTH:1:1}
  fi
  OLDEST_COMPLETE_MONTH="$((OLDEST_MONTH+1))"
  if [[|${#OLDEST_COMPLETE_MONTH} -eq 1 ]]; then
    # add leading zero
    OLDEST_COMPLETE_MONTH="0$OLDEST_COMPLETE_MONTH"
  fi

fi echo “So I assume oldest complete month is $OLDEST_YEAR/$OLDEST_COMPLETE_MONTH” echo “It may be safe to remove all awstats files awstats$OLDEST_COMPLETE_MONTH$OLDEST_YEAR.$DOMAIN.txt and newer” echo “Hardcoded DATA_DIR is $DATA_DIR where I will look for awstats files” echo “Preas a key to continue or ctrl-C to abort”; read

BACKUP=”./awstats_$DOMAIN.$(date +%Y%m%d%H%M%S).tgz” echo “Backing up awstat files to $BACKUP” tar czvf $BACKUP $DATA_DIR/awstats*.$DOMAIN.txt

ls -1 $DATA_DIR/awstats0-9[0-9]0-9[0-9]0-9[0-9].$DOMAIN.txt | while read AWSTAT; do

  AWSTAT_YEAR="$(echo "$(basename $AWSTAT)" | sed 's/awstats[[0-9]]\{2\}\([[0-9]]\{,4\}\).*/\1/')"
  AWSTAT_MONTH="$(echo "$(basename $AWSTAT)" | sed 's/awstats\([[0-9]]\{2\}\).*/\1/')"
  #echo "Found $AWSTAT (stats for $AWSTAT_YEAR/$AWSTAT_MONTH)"
  if [[|$AWSTAT_YEAR -eq $OLDEST_YEAR ]]; then
    # same year
    if [[|$AWSTAT_MONTH -ge $OLDEST_COMPLETE_MONTH ]]; then
      # same month or newer
      echo "Deleting $AWSTAT"
      rm -f "$AWSTAT"
    fi
  fi
  if [[|$AWSTAT_YEAR -gt $OLDEST_YEAR ]]; then
    # newer year
    echo "Deleting $AWSTAT"
    rm -f "$AWSTAT"
  fi

done

1. now recreate logs
2. we must feed all logs to awstats in chronological order

ls -1 “$LOG”-* | while read old_log; do

  echo $old_log | egrep -q '\.gz$' && {
    zcat $old_log
  } || {
    cat $old_log
  }

done | nice /usr/lib/cgi-bin/awstats.pl -config=$DOMAIN -update -showsteps -showcorrupted -showdropped -LogFile=”-”

1. and current log

nice /usr/lib/cgi-bin/awstats.pl -config=$DOMAIN -update -showsteps -showcorrupted -showdropped

echo “Finished” </code>

Calls the previous script for all domains

  - !/bin/sh
  - 
  -  recreate awstats for all known domains
  - 

find domain.conf.d/*.conf | while read CONFIG; do
<code>
  . $CONFIG
  ./recreate_stats.sh $AW_DOMAIN $AW_ACCESS_LOG

done </code>

–mschiff 17:02, 10 Jun 2005 (CEST)