SULU 57d

Summary

* Additional refactoring needed for CAF (e.g. methods related to metadata handling are not experiment specific). Some CMS specific info is now forwarded to the site movers (requested by Valentina Mancinelli)

* Added verification of se[prod]path (properly formatted for rucio paths). Rucio style paths are generated if se[prod]path ends with /rucio (now making sure that it does not end with rucio/, if so the pilot corrects the path)

* Updated handling of complex se[prod]path's. Previsouly a path like

/something/[path1,path2]/rucio was not properly unfolded. In relation to

this, the --overwriteQueuedata handling has also been updated for handling of similar complex paths that are dynamically updated via job parameters.

Version info


Now sending fileDestinationSE, nFiles to sitemover_put_data() from mover_put_data() (two places) (Mover)
Added fileDestinationSE, nFiles arguments to sitemover_put_data() (Mover)
Sending fileDestinationSE, nFiles to sitemover from sitemover_put_data() (Mover)
Defined nFiles, used by getSpaceTokenList(), getSpaceTokenForFile() and sent to sitemover_put_data() (Mover)
Moved rmRedundants to Experiment to JobLog (Experiment, JobLog)
Added removeRedundantFiles() to ATLASExperiment
Renamed rmRedundants() to removeRedundantFiles() (Experiment)
Now getting experiment object in postJobTask() and using it to access the rmRedundants() method (JobLog)
Created minimal list of files and directories to remove in default removeRedundantFiles() (Experiment)
Copied postProcessQueuedata() to ATLASSiteInformation, and cleaned up the prototype method in SiteInformation (SiteInformation, ATLASSiteInformation)
Created verifyRucioPath() called from postProcessQueuedata(), which verifies that any se[prod]path has correctly formatted rucio path (ATLASSiteInformation)
Updated call from rmRedundants() to removeRedundantFiles() in postJobTask() (JobLog)
Replaced faulty getDirList() with correct method updated for rucio (SiteMover)
Patched extractQueuedataOverwrite() to handle double backslashes if present in jobParameters overwriteQueuedata command (SiteInformation)
Added ls to exception handler for BU debugging (runJob)
Updated getDirList() to handle cases like _dirlist = "/[atlasscratchdisk/rucio,atlaslocalgroupdisk/rucio]" (SiteMover)