63.4

Summary

Changes related to spill-over jobs

- Support for server instruction allowNoOutput (i.e. which output files are allowed to be empty, if any). This means pilot can now use jobReport produced by user jobs for output file name extractions

- Solution for problem reported by Andrej regarding spill-over jobs on ND. Soft linked output files located inside sub directory are now moved properly

Event Service updates

- Now storing the correct guid for the TAG file in the input list to the Token Extractor

- Now using a random name for the yampl communication channel (pilot-AthenaMP) to prevent problems with several jobs running on the same WN

- Support for new releases (>=20.3.3.2) that have the new mechanism for using positional event numbers instead of using tokens

Added support for AthSimulation* releases

- Previously the pilot could not setup these correctly since the release number is 1.0.3, i.e. the pilot thinks this is a very old release

- In relation to this; the old cmt setup has been removed for AthSimulation* since it is not needed for modern asetups. Will be tested further in later pilot versions with other releases, as suggested by Asoka De Silva

Improved identification of CGROUPS sites

- Now extracting base cgroups path from /proc/mounts instead of using env variable ATLAS_CGROUPS_BASE [no longer needed], as suggested by Alessandra Forti

Version info

Now verifying that nentries is actually an integer (can aparently be 'null') in extractOutputFilesFromJSON() (FileHandling)

Now using allowNoOutput list in extractOutputFilesFromJSON() to know when an empty file can be ignored (FileHandling)

Added allowNoOutput to Job class, including handling (Job)

Now sending job.allowNoOutput list to extractOutputFilesFromJSON() (RunJob)

Overriding the input file guid for the TE input file list in __main__() (RunJobEvent)

Adding a random string to the yampl channel name in __init__() (RunJobEvent)

Telling AthenaMP the name of the yampl channel in __main__() (RunJobEvent)

Updated getMaxMemoryUsageFromCGroups() and removed the need for ATLAS_CGROUPS_BASE (processes)

Created getCGROUPSBasePath() used by getMaxMemoryUsageFromCGroups() (processes)

Updated isCGROUPSSite() to use getCGROUPSBasePath() instead of using ATLAS_CGROUPS_BASE (processes)

Added support for AthSimulation* releases (e.g. for jobs with homePackage=AthSimulationBase/1.0.3) in getProperASetup() and getProperSiterootA\

ndCmtconfig() (ATLASExperiment)

Created getSiterootWithHomepackage(), used by getProperSiterootAndCmtconfig() and getProperASetup() (ATLASExperiment)

Created getSplitHomePackage() used by getSiterootWithHomepackage(), getProperASetup() (ATLASExperiment)

Now extracting output files from jobReport for user jobs as well in __main__() (RunJob)

Avoiding adding cmt setup (getProdCmd2() call) for AthSimulation* releases in getJobExecutionCommand() (ATLASExperiment)

Updated xml source string from "NG Panda server" to "aRC" in getPoolFileCatalogNG() (Mover)

Renamed function getPoolFileCatalogNG() to getPoolFileCatalogND() (Mover)

Updated getFiletypeFromDictionary() to handle ND properly (filetype DISK is implied) (Mover)

Now setting copytool for ND in extractCopytoolForPFN() (Mover)

Using readlink to unfold any soft linked file paths in put_data() (mvSiteMover)