68.2

Summary

* The payload setup has been updated (meaning the asetup options handling). The pilot now allows for more general release information in the 'release' and 'homepackage' fields in the job definition; release = Atlas-[some release string/numbers/time stamps], homePackage = AnalysisTransforms-[some project string] or [some project string]/[some number or string]. The cmtconfig is now using the more modern --platform option instead of --cmtconfig.

* Benchmarks; Pilot is now sending an optimized and enhanced benchmark dictionary to an intermediary server (which in turn populates ES) provided by Ilija Vukotic (UChicago).

* Event service; Fixed problem with special athena jobOptions in release 21.0.15. Problem fixed in pilot version 68.1 for AES on the grid, now also fixed in the HPC version of the code.

* Memory monitor setup; now using the same release as the payload if >= 21.0.18 (otherwise using 21.0.18 for older releases).

Contributors: P. Nilsson, W. Guan, D. Drizhuk, M. Lassnig

Version info

General changes:

- Removed deprecated code form repo (PilotUtils, dataPilot, globusPilot, trivialPilot)

Payload setup:

- Added 'master' and 'multithreading' to special cache dir list in getCacheInfo() (ATLASExperiment)

- Added 'Athena' to special cache dir list in getProperASetup() (ATLASExperiment)

- Changed all occurrances of --cmtconfig to --platform, requested by Graeme Stewart (ATLASExperiment)

- Removed deprecated methods getVerifiedAtlasSetupPath(), installPacman(), installPyJobTransforms(), getCmtsiteCmd(), getTag(), isCMTCONFIGOutcommented(),

  checkCMTCONFIG(), verifyCMTCONFIG(), verifyCmtsiteCmd(), verifyReleaseInTagsFile(), verifySetupCommand(), getProperSiterootAndCmtconfig(),

  getHighestVersionDir(), isForceConfigCompatible(), getLCGSwbase(), findPythonInRelease(), getSiterootWithHomepackage(), getProperASetup(),

  getInstallDir(), getInstallDir2(), getProdCmd2(), getSpecialSetupCommand(), getAnalyCmd2(), updateAnalyCmd2(), useAtlasSetup(), getSplitHomePackage() (ATLASExperiment)

- Added getASetup() and getASetupOptions() (ATLASExperiment)

- Removed unnecessary usage of siteroot in several places (ATLASExperiment)

Memory monitoring:

- getUtilityCommand() now using getModernASetup() instead of deprecated getProperASetup() (ATLASExperiment)

Event Service:

- Improved support for Event Streaming Service (RunJobEvent)

Benchmarking:

- Now sending benchmark dictionary to intermediary ES service in postJobTask() (JobLog)

Updates from D. Drizhuk:

  https://github.com/PanDAWMS/pilot/pull/125

Commit Summary

    Memory monitor version checker

File Changes

    M ATLASExperiment.py (8)

    M pUtil.py (18)

Updates from W. Guan:

  https://github.com/PanDAWMS/pilot/pull/126

Commit Summary

The problem in RunJobEvent.py is already fixed. Last time I forgot to fix the same problem in RunJobHpcEvent.py

fix jps problem in RunJobHpcEvent

File Changes

M RunJobHpcEvent.py (8)

Updates from M. Lassnig:

  https://github.com/PanDAWMS/pilot/pull/128

Commit Summary

fix an easy way to break elasticsearch (sending filesize as long instead of string)

File Changes

M Mover.py (4)