PICARD 59c

Summary

A new pilot has been released. The new features and fixes are:

* New HPC classes for NERSC sites; RunJobHopper and RunJobEdison, by

Danila Oleynik.

* Activation of checksum verifications (supported on both BNL and CERN

object stores) and minor bug fixes in object store site movers, by Wen

Guan.

* New version of aria2c site mover now using env variable X509_CERT_DIR

instead of a hardcoded path (fixing a problem seen at UKI-LT2-IC-HEP_SL6,

reported by Duncan Rand); code contribution from Vincenzo Lavorini.

* Support for AthAnalysisBase (minor setup change), requested by Rod

Walker and Will Buttinger.

* Fix for jobs writing to group disk in case the job originated from a

different cloud. Previously path would be generated in the wrong way. Now

the pilot will reset the group disk space token and write the files to the

default space token. The files will be moved to group disk by DDM.

* glexec update, by Edward Karavakis: Updated the logic of gLExec. If the

schedconfig flag is set to 'test', the pilot will hard-fail so that we can

easily debug the error and identify sites with problematic glexec

installations. If the flag is set to 'True', just before starting the

glexec mode, we ping the glexec infrastructure to test if it is OK and

properly configured at the site. If it is OK, we go ahead with glexec. If

not, it switches back to the normal running mode without using glexec.

* Improved error handling in event service jobs in case of transfer

failures.

Version info

Bug fixes in S3OjectstoreSiteMover

Skipping TURL based PFC in shouldPFC4TURLsBeCreated() if copyprefixin does not contain oldPrefix (Mover)

Added eventService argument to doSpecialLogFileTransfer() (Experiment, ATLASExperiment)

Sending job.eventService to doSpecialLogFileTransfer() from transferLogFile() (JobLog)

Updated RunJobFactory for Hopper and Edison (RunJobFactory)

Cleaned up some debugging log messages from PandaServerClient related to PandaID (PandaServerClient)

Added stage-out error handling to RunJobEvent

Created get/setErrorCode() used by stageOut() and main function (RunJobEvent)

Printing the job definition backup filename instead of the full dispatcher response in backupDispatcherResponse() (pilot)

Avoiding duplicated of "Put error: " at the beginning of pilotErrorDiag in stageOut() (RunJob, RunJobEvent)

Updated run command for special homePackage (AthAnalysisBase) in getJobExecutionCommand() (ATLASExperiment)

Created getTiersOfATLASSE() used by getGroupDiskPath() (SiteMover)

Created getDefaultDQ2SiteName() used by getGroupDiskPath() (SiteMover)

Updated getPreDestination() for resetting groupdisk space token (SiteMover)

Code from Danila Oleynik:

RunJobHopper and RunJobEdison (added to SVN)

Updated to RunJobTitan

Code from Edward Karavakis:

glExec bug fixes (pilot, pUtil)

Code from Vincenzo Lavorini

Bug fix related to missing X509_CERT_DIR (aria2SiteMover)