PICARD 59g

Summary

* Sending jobReport.json to the panda server instead of metadata xml along

with the job report at the end of a job, if the json report version is >=

1.0.0. For these jobs, any trf error info will be extracted and displayed

by the panda monitor.

* Fixed a bug seen in HC test jobs due to problems with locating the

jobReport.json in ANALY jobs; reported by Johannes Elmsheuser.

* New version of aria2c site mover from Vincenzo Lavorini, e.g. updating

the auth cluster name.

* Now using env variable ATLAS_CGROUPS_BASE for identifying cgroups sites.

If defined and set to a non-empty string, the pilot will not issue the

resource.setrlimit (which in turn is normally used if maxmemory is

defined). The env variable should point to the memory info (e.g.

/var/cgroups/memory). Further support for cgroups sites will be

implemented in later pilot versions.

* Bug fix related to an undefined variable in some site movers

(local/xrdcp/xrootdObjectstore/FAX/GFAL2); previously this would lead to

an exception for transfers that were timed-out, i.e. leading to the wrong

error message.

* glExec update from Edward Karavakis: I have discovered that in some WNs,

the GLEXEC env var is set but contains an empty string instead of the

actual location of glexec. I've now added an additional check when this

env var is set pointing to nowhere by forcing the use of the default

location of glexec -> /usr/sbin/glexec.

Version info

Using default metadata XML file if reportVersion is not found in jobReport.json, in getPayloadMetadataFilename() (PandaServerClient)

Now sending jobReport.json if report version >= 1.0.0 in getPayloadMetadataFilename() (PandaServerClient)

Created getMaxMemoryUsageFromCGroups() [to be used by Monitor.py] (processes)

Removed misleading log messages related to process killing ('Going to kill pid ..') (Monitor)

Added curtime_mem in monitor_job() (Monitor)

Created __verify_memory_limits(), called from monitor_job() (Monitor)

Created isCGROUPSSite(), used by __getsetWNMem() and its duplicate getsetWNMem() (Monitor, pilot)

New version of aria2c Site Mover from Vincenzo Lavorini, e.g. updating the auth cluster (to rucio-auth-prod.cern.ch)

__getsetWNMem() now using getMaxMemoryUsageFromCGroups() for testing purposes (Monitor)

Corrected error message 'cmtside command was timed out' -> 'cmtsite command was timed out' in verifyCmtsiteCmd() (ATLASExperiment, TaiwanExperiment)

Now using ATLAS_CGROUPS_BASE in isCGROUPSSite() (processes)

Replaced globalSite with thisSite, and added a try-statement, in sig2exc() (pUtil)

Added protection for non-allowed server states in verifyJobState() (pUtil)

Updates from Edward Karavakis for glexec (pilot, glexec_utils)

Corrected undefined variable, t -> timeUsed, in errorToReport() (LocalSiteMover, FAXSiteMover, GFAL2SiteMover, xrdcpSiteMover, xrootdObjectstoreSiteMover)