SULU 48a

Summary

* Now setting CPU model on every server update. Before, the CPU info was
only known (and displayed on the panda job page) at the end of a job.
Requested by Alexandre Vaniachine.

* Corrections to the job_script.sh (missing source command on some sites,
\n). The job_script.sh is a file created by the pilot which explicitly
shows all commands required to run the job (stage-ins, setup scripts and
trf execution command), i.e. it can be used to re-create the job on the
site afterward. Requested by Rod Walker.

* Added try-statement to long fsize operation in mover_get_data() to avoid
problems with locally available DBRelease files on Nordugrid. Requested by
Andrej Filipcic.

* Added new error codes: 1191, Illegal zero file size (pilot will fail
jobs with zero size output file with a clear error message); 1192,
DBRelease file has not been transferred yet (identified as empty SFN in
LFC).

* Added IFIC and ANALY_IFIC to inclusion list of STORM sites using
lcg-get-checksum [LIP-COIMBRA remains]

* Now updating appdir in queuedata if not set in schedconfig DB, for every
multi-job. Previously multi-jobs did not run properly on CVMFS sites using
empty appdir's.

* Filtering of dccp output for possible illegal characters, in an effort
to try and understand weird job failures at especially TRIUMF and VICTORIA
sites. For unknown reasons the pilots' internal time-out function, or code
that is handling the return tuple from the time-out function, sometimes
fail with "'int' object is not callable"-errors.

* New JEM code from Tim dos Santos, with the following release notes:

- activation svc is used, at the moment it answers with "don't care"
later it will be able to veto monitoring, or to force monitoring (white
list, black list)
- the JEM code in the pilot understands commands from activation svc like
"what JEM version to use", "what MQ server to send the data to", etc.

JEM is ready to be announced to the users, we thought about maybe adding
a paragraph to the workbook - we're concentrating work on our monitoring
data presentation at the moment (make useful plots and statistics out of
the data that already is in our DB / gets
fed into the DB by JEM now)

Version info

Now setting CPU model on every server update in getNodeStructure() (PandaServerClient)
Added explicit 'source' to job_script if necessary, in get_data() (lcgcpSiteMover, xrdcpSiteMover)
Added explicit 'source' to job_script if necessary, in core_get_data() (stormSiteMover)
Added line breaks in job_setup.sh in executePayload (runJob)
Added try-statement to long fsize operation in mover_get_data() to avoid problems with locally available DBRelease files on Nordugrid (Mover)
Added new error code 1191, ERR_ZEROFILESIZE, Illegal zero file size (PilotErrors)
Failing job for zero file size in getLocalFileInfo() (SiteMover, xrootdSiteMover, xrdcpSiteMover)
Added return code to getOutputFileInfo() (pUtil)
Added error handling of non-zero exit code from getOutputFileInfo() in createFileMetadata() (runJob)
Added IFIC and ANALY_IFIC to inclusion list in put_data() (lcgcpSiteMover) [LIP-COIMBRA remains]
Now updating appdir in queuedata if not set, for every multi-job, in extractAppdir() (pUtil)
Added argument filename_dic to getLFCFileList() and to getReplicasDictionary() (Mover)
Sending filename and guid dictionary to getLFCFileList() in getPoolFileCatalog(), and to getReplicasDictionary() in getLFCFileList() (Mover)
Recognizing DBRelease file in getReplicasDictionary() if empty SFN, better error message (Mover)
Added new error code 1192, ERR_DBRELNOTYETTRANSFERRED, DBRelease file has not been transferred yet (PilotErrors)
Improved replica errors in getReplicasDictionary() (Mover)
Added exception handling around tolog() call to inspect.stack() (pUtil)
Updated findPythonInRelease() to try to use asetup for the case that schedconfig.appdir is not set (pUtil)
Created filter_text() and is_int() (dCacheSiteMover)
get_data() and put_data() are now using filter_text() and is_int() with output from timed_command() (dCacheSiteMover)
New RunJobUtilities code from Tim dos Santos, new functions and code for JEM (RunJobUtilities)