SULU 56b

Summary

* Pilot will use asetup "fast" option if the special env variable
ATLAS_FAST_ASETUP exists

* Added FAX support for direct i/o, in the case that FAX is used as a
primary site mover. Requested by Johannes Elmsheuser et al

* Skipping dq2-list-files command since path can be constructed with DQ2
function to_native_lfn(). This change means that the pilot now can handle
FAX transfers without the need for the new DQ2Client. Suggested by
Hironori Ito

* Now importing pilot module ErrorDiagnosis from the top in runJob
attempting to avoid python bug (?) leading to missing module, seen at BU

* Added new pilot error 1216: "Pilot TCP server has died", as well as code
for verifying that the pilot TCP server is still alive before asking for a
new job. Also, added a socket connection time-out of 120s while sending
TCP message. This error is not sent to the panda monitor so no job will
fail with this code (it might be more widely used at a later time). It
would currently only be visible in a batch log

* Added tmp.* to list of files to be removed before log creation.
Requested by Graeme Stewart

* Corrected a problem with some trf errors not being reported properly
(for the case that both the trf error and exe error codes were set); in
relation to this, the pilot is now grabbing the trf error code and its
diagnostics for the new transforms from jobReport. Requested by Graeme
Stewart

* Added --srm-timeout 3600 for lcg-cp/cr. Requested by Ikuo Ueda

* Added a new version of aria2c site mover, including a fix to Mover. Code
by Rodney Walker

Version info

Removed useless si object from findGlobalFilePath() and its call in get_data() (FAXSiteMover)
Added harmless error code 1009 used to inform Mover that the site mover has skipped stage-in in favor of direct i/o (PilotErrors)
sitemover_get_data() now returns will_use_direct_io boolean (Mover)
Receiving will_use_direct_io in mover_get_data() (Mover)
Now returning error.ERR_DIRECTIOFILE for direct i/o file in get_data() (FAXSiteMover, xrdcpSiteMover, lcgcpSiteMover, lcgcp2SiteMover, LocalSiteMover,
curlSiteMover, dCacheSiteMover, CastorSiteMover, BNLdCacheSiteMover, castorSvcClassSiteMover, rfcpLFCSiteMover, xrootdSiteMover)
Now using fast asetup option if special env variable is defined, in getProperASetup() (pUtil)
Added new boolean usedFAXandDirectIO in stageIn() and it's call in main() (runJob)
get_data() now returns usedFAXandDirectIO to stageIn() (Mover, runJob)
mover_get_data() now returns usedFAXandDirectIO to get_data() (Mover)
Sending usedFAXandDirectIO to updateRunCommandList() from main() (runJob)
Added usedFAXandDirectIO argument to updateRunCommandList() (RunJobUtilities)
Created removePattern() (pUtil)
Now removing --lfcHost, --oldPrefix, --newPrefix and adding --usePFCTurl in the case that FAX was used
as a primary site mover in combination with direct I/O, in updateRunCommandList() (RunJobUtilities)
Now importing ErrorDiagnosis from the top in runJob attempting to avoid python bug seen at BU (runJob)
Changed class inheritance SiteMover -> SiteMover.SiteMover (xrdcpSiteMover, xrootdSiteMover)
Updated SiteMover import due to previous change (xrdcpSiteMover, xrootdSiteMover)
Changed all SiteMover. -> self. (xrootdSiteMover, xrdcpSiteMover)
Added some debug info to updateJobInfo() before and after TCP message (RunJobUtilities)
Added ERR_NOPILOTTCPSERVER used when pilot TCP server has died, not relevant for job info (PilotErrors)
Created isPilotTCPServerAlive used by runMain() (pUtil, pilot) [Alert Fernando/Ramon]
Added socket connection time-out (120s) to updateJobInfo() (pUtil)
Added tmp.* to list of files to be removed before log creation. Requested by Graeme Stewart (JobLog)
Renamed res tuple to res_tuple in executePayload() (runJob)
Added exitMsg to res tuple after trf parsing in executePayload() (runJob)
Removed outdated jobReportError from makeJobReport() and from its call in postJobTask() (pUtil, JobLog)
Now showing exeErrorDiag in job error report, in makeJobReport() (pUtil)
Now setting exeErrorDiag in interpretPayloadStdout() (ErrorDiagnosis)
Corrected wrong variable name pilotErrorCode -> pilotErrorDiag in verifyCmtsiteCmd() (pUtil)
Created filterTCPString() used by updateJobInfo() (RunJobUtilities)
Added exeErrorDiag to handle() (pilot) [Alert Fernando/Ramon]
Created updateRedirector() (FAXSiteMover)
Created encode_string(), decode_string() used by RunJobUtilities::updateJobInfo() and pilot::handle() (pUtil)
Importing encode_string (RunJobUtilities)
Now using the LFN from the global path instead of initial LFN (problem seen with filenames containing __DQ2-bit) in get_data() (FAXSiteMover)
Skipping old style trf error XML file (jobInfo.xml) if exeErrorCode/exeErrorDiag are already set, in postJobTask() (JobLog)
Added --srm-timeout to put_data() and [core_]get_data() (lcgcpSiteMover, lcgcp2SiteMover)
Overwriting pilotErrorDiag with exeErrorDiag if set in postJobTask() (JobLog)
Added new version of aria2c site mover, and modified a call to getID() in getFileInfo(), requested by Rodney Walker (aria2cSiteMover, Mover)