Using native APEL Client to report usage statistics from ARC CE


BLAH logger distributed along with nordugrid-arc-arex package since version 4.1.0, so with a latest version you already have logger installed.

For previous ARC versions you can install standalone package from Blackjack repos.

Install APEL parser and client from EMI repos:

yum -y localinstall
yum -y install apel-client apel-parsers

Install MySQL backend for APEL clients. MySQL can be installed on any host, running it on ARC CE host is not mandatory.

yum -y install mysql-server mysql
chkconfig mysqld on
service mysqld start
mysql -u root -p
mysql> CREATE DATABASE apelclient;
mysql> GRANT ALL PRIVILEGES ON apelclient.* TO apel@localhost IDENTIFIED BY 'myverysecureapelpassword';
mysql> quit

Provision APEL database with the required structures using the SQL file supplied with APEL packages:

mysql -u apel -p apelclient < /usr/share/apel/client.sql


1. Configure ARC CE to call installed BLAH logger. In the /etc/arc.conf add the following configuration and restart A-REX:

authplugin="FINISHED timeout=10,onfailure=pass /usr/libexec/arc/arc-blahp-logger -I %I -U %u -L %C/job.%I.local -P %C/job.%I.proxy"

By default BLAH log is written to /var/log/arc/accounting/blahp.log-YYYYMMDD. The log prefix (without -YYYYMMDD) can be redefined with optional -p option.

CE ID is generated automatically and has a format host.fqdn:2811/nordugrid-torque-queue in accordance to GLUE1.3 publishing. Queue is added in runtime anyway, but prefix can be redefined with optional -c option.

To debug logger execution you can add -d 5 option and see all ARC logger output from all subsystems used.

2. Edit APEL parser configuration file (/etc/apel/parser.cfg) and specify DB access settings, batch system accounting logs path, and ARC CE BLAH plugin logs path:

hostname = localhost
port = 3306
name = apelclient
username = apel
password = myverysecureapelpassword

site_name = GOCBD-SITE-NAME
lrms_server =

enabled = true
dir = /var/log/arc/accounting
filename_prefix = blahp.log
subdirs = false

enabled = true
reparse = false
type = PBS
dir = /path/to/torque/server_priv/accounting/
filename_prefix = 201
subdirs = false

logfile = /var/log/apel/parser.log
level = INFO
console = false

Ensure that LRMS accounting logs are available on APEL node: configure file sharing (NFS?) accordingly is LRMS is not local to APEL node.

Run apelparser and make sure records are parsed successfully looking into parser.log and of course the database content :-)

If everything is OK, it's time to configure APEL client and broker network to use.

3. Configure APEL broker network client. Edit the /etc/apel/client.cfg:

hostname = localhost
port = 3306
name = apelclient
username = apel
password = myverysecureapelpassword

enabled = true
site_name = GOCBD-SITE-NAME
ldap_host =
ldap_port = 2170

enabled = true
local_jobs = false

enabled = true
dir_location = /var/spool/apel/
send_summaries = true
withhold_dns = false
interval = latest
send_ur = false

enabled = true

logfile = /var/log/apel/client.log
level = INFO
console = false

And finally SSM configuration should be specified in /etc/apel/sender.cfg:

# production
bdii: ldap://
network: PROD
use_ssl: true
# testing
#port: 6163
#use_ssl: false

certificate: /etc/grid-security/hostcert.pem
key: /etc/grid-security/hostkey.pem
capath: /etc/grid-security/certificates

# production
destination: /queue/global.accounting.cpu.central
# testing
#destination: /queue/global.accounting.cputest.CENTRAL
path: /var/spool/apel/outgoing

logfile: /var/log/apel/ssmsend.log
level: INFO
console: false

4. Finally you need to create a CRON job to parse logs and sent records regularly. It can be simply /etc/cron.daily/apel that contains:



Operational part to publish data to PROD Network

To speed-up the process you can declare "APEL" and "gLite-APEL" endpoints simultaneously.

Running ARC CE along with CREAM

Not a problem, on your ARC CE you just does not need to parse LRMS logs (once they are already parsed on CREAM box) and does not need to run apelclient for the same reason. All you need is BLAH logger and APEL parser to parse only one more BLAH logs.

enabled = false

Several ARC CEs within the same grid site can be configured the same way.

Configuration for SLURM LRMS

  1. Please refer to the APEL Parsers System Administrator Guide (section 4.3) for instructions how to configure SLURM to write an accounting records for APEL parser.
  1. In APEL parser configuration specify SLURM type and corresponding accounting records path.
  1. Add and extra argument to ARC BLAH logger:
    authplugin="FINISHED timeout=10,onfailure=pass /usr/libexec/arc/arc-blahp-logger -I %I -U %u -L %C/job.%I.local -P %C/job.%I.proxy -c host.fqdn:2811/nordugrid-SLURM"

BLAH logger for ARC < 4.1.0

For ARC 4.0.x install standalone package from Blackjack repo (EL6 only):

yum -y localinstall
yum -y install nordugrid-arc-blahp-logger

For ARC 3.x you need to specify version 3.0.0 directly (EL6 only):

yum -y install nordugrid-arc-blahp-logger-3.0.0

For previous ARC versions you can use compat BLAH logger.

BLAH logger also available for EL5 in Blackjack repo, for other OSes you can use sources.

Compat version of logger for older ARC CE versions

Compat version of logger has been written in BASH, depend on arcproxy tool and also available in Blackjack repo:

yum -y localinstall
yum -y install nordugrid-arc-blahp-logger-compat

The syntax of compat arc-blahp-logger invocation from A-REX is slightly differ from binary version:

authplugin="FINISHED timeout=10,onfailure=pass /usr/libexec/arc/arc-blahp-logger /var/log/arc/accounting/blahp.log- %W %C %I %u"

The last parameter is optional like -c in binary version.

Last modified 8 years ago Last modified on May 1, 2014 4:24:31 PM