= Using native APEL Client to report usage statistics from ARC CE = == Installation (EL6) == Install ARC CE plugin to write logs for APEL BLAH parser from Blackjack repo: {{{ yum -y localinstall http://blackjack.grid.org.ua/pub/linux/rhel/6x/base/x86_64/blackjack-release-6-2.noarch.rpm yum -y install nordugrid-arc-blahp-logger }}} Install APEL parser and client from EMI repos: {{{ yum -y localinstall http://emisoft.web.cern.ch/emisoft/dist/EMI/3/sl6/x86_64/base/emi-release-3.0.0-2.el6.noarch.rpm yum -y install apel-client apel-parsers }}} Install MySQL backend for APEL clients. MySQL can be installed on any host, running it on ARC CE host is not mandatory. {{{ yum -y install mysql-server mysql chkconfig mysqld on service mysqld start mysql_secure_installation mysql -u root -p mysql> CREATE DATABASE apelclient; mysql> GRANT ALL PRIVILEGES ON apelclient.* TO apel@localhost IDENTIFIED BY 'myverysecureapelpassword'; mysql> FLUSH PRIVILEGES; mysql> quit }}} Provision APEL database with the required structures using the SQL file supplied with APEL packages: {{{ mysql -u apel -p apelclient < /usr/share/apel/client.sql }}} == Configuration == ''1.'' Configure ARC CE to call installed BLAH logger. In the {{{/etc/arc.conf}}} add the following configuration and restart A-REX: {{{ authplugin="FINISHED timeout=10,onfailure=pass /usr/libexec/arc/arc-blahp-logger -I %I -U %u -L %C/job.%I.local -P %C/job.%I.proxy" }}} By default BLAH log is written to {{{/var/log/arc/accounting/blahp.log-YYYYMMDD}}}. The log prefix (without {{{-YYYYMMDD}}}) can be redefined with optional {{{-p}}} option. CE ID is generated automatically and has a format {{{host.fqdn:2811/nordugrid-torque-queue}}} in accordance to GLUE1.3 publishing. Queue is added in runtime anyway, but prefix can be redefined with optional {{{-c}}} option. To debug logger execution you can add {{{-d 5}}} option and see all ARC logger output from all subsystems used. ''2.'' Edit APEL parser configuration file ({{{/etc/apel/parser.cfg}}}) and specify DB access settings, batch system ''accounting'' logs path, and ARC CE BLAH plugin logs path: {{{ [db] hostname = localhost port = 3306 name = apelclient username = apel password = myverysecureapelpassword [site_info] site_name = GOCBD-SITE-NAME lrms_server = some.fqdn.lrms.id [blah] enabled = true dir = /var/log/arc/accounting filename_prefix = blahp.log subdirs = false [batch] enabled = true reparse = false type = PBS dir = /path/to/torque/server_priv/accounting/ filename_prefix = 201 subdirs = false [logging] logfile = /var/log/apel/parser.log level = INFO console = false }}} Ensure that LRMS accounting logs are available on APEL node: configure file sharing (NFS?) accordingly is LRMS is not local to APEL node. Run {{{apelparser}}} and make sure records are parsed successfully looking into {{{parser.log}}} and of course the database content :-) If everything is OK, it's time to configure APEL client and broker network to use. ''3.'' Configure APEL broker network client. Edit the {{{/etc/apel/client.cfg}}}: {{{ [db] hostname = localhost port = 3306 name = apelclient username = apel password = myverysecureapelpassword [spec_updater] enabled = true site_name = GOCBD-SITE-NAME ldap_host = bdii.ha.grid.org.ua ldap_port = 2170 [joiner] enabled = true local_jobs = false [unloader] enabled = true dir_location = /var/spool/apel/ send_summaries = true withhold_dns = false interval = latest send_ur = false [ssm] enabled = true [logging] logfile = /var/log/apel/client.log level = INFO console = false }}} And finally SSM configuration should be specified in {{{/etc/apel/sender.cfg}}}: {{{ [broker] # production bdii: ldap://bdii.ha.grid.org.ua:2170 network: PROD use_ssl: true # testing #host: test-msg02.afroditi.hellasgrid.gr #port: 6163 #use_ssl: false [certificates] certificate: /etc/grid-security/hostcert.pem key: /etc/grid-security/hostkey.pem capath: /etc/grid-security/certificates [messaging] # production destination: /queue/global.accounting.cpu.central # testing #destination: /queue/global.accounting.cputest.CENTRAL path: /var/spool/apel/outgoing [logging] logfile: /var/log/apel/ssmsend.log level: INFO console: false }}} You can start with "testing" APEL server and open a GGUS ticket to verify if everything is OK. But once the solution will be verified for one site, another sites can go to production using the standard operations procedure: - declare "gLite-APEL" endpoing in GOCDB for grid-site - use PROD brockers from BDII - check ​http://goc-accounting.grid-support.ac.uk/rss/GOCBD-SITE-NAME_Sync.html and ​http://goc-accounting.grid-support.ac.uk/rss/GOCBD-SITE-NAME_Pub.html in a few days. ''4.'' Finally you need to create a CRON job to parse logs and sent records regularly. It can be simply {{{/etc/cron.daily/apel}}} that contains: {{{ #!/bin/sh /usr/bin/apelparser /usr/bin/apelclient }}} == Running ARC CE along with CREAM == Not a problem, on your ARC CE you just does not need to parse LRMS logs (once they are already parsed on CREAM box) and does not need to run {{{apelclient}}} for the same reason. All you need is BLAH logger and APEL parser to parse only one more BLAH logs. {{{ [batch] enabled = false }}}