Using native APEL Client to report usage statistics from ARC CE

Installation (EL6)

Install ARC CE plugin to write logs for APEL BLAH parser from Blackjack repo:

yum -y localinstall http://blackjack.grid.org.ua/pub/linux/rhel/6x/base/x86_64/blackjack-release-6-2.noarch.rpm
yum -y install nordugrid-arc-blahp-logger

Install APEL parser and client from EMI repos:

yum -y localinstall http://emisoft.web.cern.ch/emisoft/dist/EMI/3/sl6/x86_64/base/emi-release-3.0.0-2.el6.noarch.rpm
yum -y install apel-client apel-parsers

Install MySQL backend for APEL clients. MySQL can be installed on any host, running it on ARC CE host is not mandatory.

yum -y install mysql-server mysql
chkconfig mysqld on
service mysqld start
mysql_secure_installation
mysql -u root -p
mysql> CREATE DATABASE apelclient;
mysql> GRANT ALL PRIVILEGES ON apelclient.* TO apel@localhost IDENTIFIED BY 'myverysecureapelpassword';
mysql> FLUSH PRIVILEGES;
mysql> quit

Provision APEL database with the required structures using the SQL file supplied with APEL packages:

mysql -u apel -p apelclient < /usr/share/apel/client.sql

Configuration

1. Configure ARC CE to call installed BLAH logger. In the /etc/arc.conf add the following configuration and restart A-REX:

authplugin="FINISHED timeout=10,onfailure=pass /usr/libexec/arc/arc-blahp-logger -I %I -U %u -L %C/job.%I.local -P %C/job.%I.proxy"

By default BLAH log is written to /var/log/arc/accounting/blahp.log-YYYYMMDD. The log prefix (without -YYYYMMDD) can be redefined with optional -p option.

CE ID is generated automatically and has a format host.fqdn:2811/nordugrid-torque-queue in accordance to GLUE1.3 publishing. Queue is added in runtime anyway, but prefix can be redefined with optional -c option.

To debug logger execution you can add -d 5 option and see all ARC logger output from all subsystems used.

2. Edit APEL parser configuration file (/etc/apel/parser.cfg) and specify DB access settings, batch system accounting logs path, and ARC CE BLAH plugin logs path:

[db]
hostname = localhost
port = 3306
name = apelclient
username = apel
password = myverysecureapelpassword

[site_info]
site_name = GOCBD-SITE-NAME
lrms_server = some.fqdn.lrms.id

[blah]
enabled = true
dir = /var/log/arc/accounting
filename_prefix = blahp.log
subdirs = false

[batch]
enabled = true
reparse = false
type = PBS
dir = /path/to/torque/server_priv/accounting/
filename_prefix = 201
subdirs = false

[logging]
logfile = /var/log/apel/parser.log
level = INFO
console = false

Ensure that LRMS accounting logs are available on APEL node: configure file sharing (NFS?) accordingly is LRMS is not local to APEL node.

Run apelparser and make sure records are parsed successfully looking into parser.log and of course the database content :-)

If everything is OK, it's time to configure APEL client and broker network to use.

3. Configure APEL broker network client. Edit the /etc/apel/client.cfg:

[db]
hostname = localhost
port = 3306
name = apelclient
username = apel
password = myverysecureapelpassword

[spec_updater]
enabled = true
site_name = GOCBD-SITE-NAME
ldap_host = bdii.ha.grid.org.ua
ldap_port = 2170

[joiner]
enabled = true
local_jobs = false

[unloader]
enabled = true
dir_location = /var/spool/apel/
send_summaries = true
withhold_dns = false
interval = latest
send_ur = false

[ssm]
enabled = true

[logging]
logfile = /var/log/apel/client.log
level = INFO
console = false

And finally SSM configuration should be specified in /etc/apel/sender.cfg:

[broker]
# production
bdii: ldap://bdii.ha.grid.org.ua:2170
network: PROD
use_ssl: true
# testing
#host: test-msg02.afroditi.hellasgrid.gr
#port: 6163
#use_ssl: false

[certificates]
certificate: /etc/grid-security/hostcert.pem
key: /etc/grid-security/hostkey.pem
capath: /etc/grid-security/certificates

[messaging]
# production
destination: /queue/global.accounting.cpu.central
# testing
#destination: /queue/global.accounting.cputest.CENTRAL
path: /var/spool/apel/outgoing

[logging]
logfile: /var/log/apel/ssmsend.log
level: INFO
console: false

You can start with "testing" APEL server and open a GGUS ticket to verify if everything is OK. But once the solution will be verified for one site, another sites can go to production using the standard operations procedure:

• declare "gLite-APEL" endpoing in GOCDB for grid-site
• use PROD brockers from BDII
• check ​​http://goc-accounting.grid-support.ac.uk/rss/GOCBD-SITE-NAME_Sync.html and ​​http://goc-accounting.grid-support.ac.uk/rss/GOCBD-SITE-NAME_Pub.html in a few days.

4. Finally you need to create a CRON job to parse logs and sent records regularly. It can be simply /etc/cron.daily/apel that contains:

#!/bin/sh

/usr/bin/apelparser
/usr/bin/apelclient

Compat version of logger for older ARC CE versions

Compat version of logger has been written in BASH, depend on arcproxy tool and also available in Blackjack repo:

yum -y localinstall http://blackjack.grid.org.ua/pub/linux/rhel/6x/base/x86_64/blackjack-release-6-2.noarch.rpm
yum -y install nordugrid-arc-blahp-logger-compat

The syntax of compat arc-blahp-logger invocation from A-REX is slightly differ from binary version:

authplugin="FINISHED timeout=10,onfailure=pass /usr/libexec/arc/arc-blahp-logger /var/log/arc/accounting/blahp.log- %W %C %I %u arc.univ.kiev.ua:2811/nordugrid-torque"

The last parameter is optional like -c in binary version.

Running ARC CE along with CREAM

Not a problem, on your ARC CE you just does not need to parse LRMS logs (once they are already parsed on CREAM box) and does not need to run apelclient for the same reason. All you need is BLAH logger and APEL parser to parse only one more BLAH logs.

[batch]
enabled = false