UHURA 41a

Summary

* Further modifications for SLC5/gcc43/i686-slc5-gcc43-opt jobs for installations with shared compiler and automatic selection of gcc43. AtlasLogin/AtlasSettings verification is now looking in $SITEROOT if the corresponding dirs exist there, if not, it will fall back to swdir (constructed from appdir/$VO_ATLAS_SW_DIR) as before. If it is not needed, the pilot will not set up the compiler as it did/does at BNL/CERN. Successfully tested at INFN-ROMA1 using release 15.3.1 + 15.3.1.4. Changes are compatible with current tests at BNL.

* User analysis jobs are now setup with forceConfig where available, as well as explicitly setting non-empty AtlasVersion/AtlasProject.

* File stager corrections including release check (file stager not compatible with release < 15.4.0). Direct access is now switched off if copysetup does not contain proper setup.

* Protection against down site in user analysis trf download, including re-trials and fallback to optional download site.

* HOTDISK correction. Replica randomization now properly handles hotdisk exception (should not be randomized).

* Storage paths for mover log files now more detailed (fully date based, previous version only created monthly subdirs).

Version info
Changed getSwbase(), added cmtconfig (pilot, RunJobUtilities, pUtil)
Corrected setup in findPythonInRelease() for SLC5 jobs on sites with appdir="" (pUtil)
Added updateCopysetup() after accessmode has been processed (RunJobUtilities)
Added cacheDir and cacheVer to getTag() (RunJobUtilities)
cmd2 is now defined before cmd1 for analysis jobs (RunJobUtilities)
Corrected harmless string formatting error in error string in getHighestVersionDir() (pUtil)
Created getTrfBaseURLs() (RunJobUtilities)
Added protection for down site in getAnalysisTrf(), switching to CERN if necessary (RunJobUtilities)
Storage paths for mover log files now more detailed (runJob, Mover, lcgcp2SiteMover, lcgcpSiteMover, LocalSiteMover)
Removed verifyTrfBaseURL() since it is now integrated with getAnalysisTrf() (RunJobUtilities)
Renamed verifyCMTCONFIG2() to verifyCMTCONFIG() and removed old unused verifyCMTCONFIG() (RunJobUtilities, pUtil)
pilot_version_tag can now also use value 'DDM' for panda mover tests (pilot)
forceConfig now implemented for user analysis jobs as well (RunJobUtilities)
getCmtsiteCmd() modified to handle both production and user analysis jobs (RunJobUtilities)
Code-cleanup of unnecessary code (RunJobUtilities)
Switching off filestage more if release < 15.4.0 (RunJobUtilities)
Switching off direct access if copysetup does not contain proper setup (RunJobUtilities)
Moved grabbing of prodSourceLabel from pilot::getNewJob() to Job::setJobDef() (pilot, job)
prodSourceLabel now sent to put_data (pilot, runJob, Mover, lcgcp2SiteMover)
randomizeReplicas() updated for hotdisk (Mover)
HOTDISK patch added to castorSvcClassSiteMover.py by Graeme
Corrected setup for older releases when forceConfig is not available, needed for finding python in release area (pUtil)