Removed any initial 0's from the locally calculated checksums (for both stage-in/out) in case local value is not of same length as remote value
- Externally calculated Adler32 values are not necessarily of fixed length and might not include initial 0 values
- Was discussed years ago but problem has now reoccurred
- Issue seen with gfal-sum command, discussed in JIRA ticket https://its.cern.ch/jira/browse/ATLASPANDA-324
NOTE: only the ‘new’ gfal-copy mover has been updated in this release
More verbosity added to rucio mover
Corner case fix for storm mover
- Time stamp removed from the etag, can otherwise wrongly end up in the LFN
mv mover updates
- mv support for all transfer schemes
- Added checksum to output file list
Fix for event service jobs
- Correction for undefined runCommandList
Contributions from W. Guan, M. Lassnig, D. Cameron, P. Nilsson.
Last login: Fri Aug 4 16:20:47 on ttys001
macnilsson6:~ nilsnilsson$ ssh nilspal@aiadm.cern.ch
Warning: Permanently added the RSA host key for IP address '2001:1458:201:a4::100:279' to the list of known hosts.
Password:
Password:
Last login: Tue Aug 8 10:18:15 2017
* ********************************************************************
* Welcome to aiadm23.cern.ch, CentOS, 7.3.1611
* Archive of news is available in /etc/motd-archive
* Reminder: You have agreed to comply with the CERN computing rules
* https://cern.ch/ComputingRules
* Puppet environment: qa, Roger state: production
* Foreman hostgroup: aiadm/nodes/login
* aiadm machines are for administrating nodes in the computer centre
* ********************************************************************
[nilspal@aiadm23 ~]$ ssh atlpan@aipanda007
Password:
Last login: Mon Aug 7 14:51:38 2017 from aiadm30.cern.ch
* ********************************************************************
* Welcome to aipanda007.cern.ch, SLC, 6.9
* Archive of news is available in /etc/motd-archive
* Reminder: You have agreed to comply with the CERN computing rules
* https://cern.ch/ComputingRules
* Puppet environment: production, Roger state: production
* Foreman hostgroup: vopanda/pandaserver/srvdevel
* PanDA Cluster, Support is https://its.cern.ch/jira/browse/ASPDA
* * WARNING, aipanda007.cern.ch has lemon exceptions:
* exception.Operating_System
* ********************************************************************
[atlpan@aipanda007 ~]$ cd esTest/
[atlpan@aipanda007 esTest]$ cat README
source /data/atlpan/srv/etc/sysconfig/panda_server-sysconfig
python submitTaskWithES.py UTA_PAUL_TEST
python submitTaskWithES-woTAG.py UTA_PAUL_TEST
python submitTaskWithES-woTAG_19.2.4.py CERN-PROD_preprod_MCORE CERN
python submitTaskWithES-woTAG_19.2.4.py BNL_PROD_MCORE US
python submitTaskWithES_19.2.4.8-short.py UTA_PAUL_TEST US
python submitTaskWithES_19.2.5.3-short2.py UTA_PAUL_TEST WORLD UTA_SWT2
python submitTaskWithES_20.3.3.2.py UTA_PAUL_TEST WORLD UTA_SWT2
python submitTaskWithES_20.3.3.2.py CERN-PROD_preprod CERN CERN-PRODd
python submitTaskWithES_20.3.7.4.py UTA_PAUL_TEST WORLD UTA_SWT2 # prefetcher test
python submitTaskWithES_20.3.7.5.py BNL_PROD_MCORE WORLD BNL-ATLAS # prefetcher test
python submitTaskWithES_20.7.X.Y.py CERN-PROD_preprod_MCORE WORLD CERN-PRODp
python submitTaskWithES_20.7.5.1.py FZK-LCG2_TEST WORLD FZK-LCG2 # remote i/o test
python submitTaskWithES_20.7.8.7.py BNL_PROD_MCORE WORLD BNL-ATLAS
python submitTaskWithES_21.0.15c.py FZK-LCG2_TEST WORLD FZK-LCG2 # remote i/o test
python submitTaskWithES_21.0.21.py BNL_PROD_MCORE WORLD BNL-ATLAS # prefetcher test
python submitTaskWithES_21.0.21-2inputs.py BNL_PROD_MCORE WORLD BNL-ATLAS # prefetcher test, 2 input files
python /data/atlpan/srv/lib/python2.6/site-packages/pandaserver/test/killTaskJEDI.py [taskID]
[atlpan@aipanda007 esTest]$ source /data/atlpan/srv/etc/sysconfig/panda_server-sysconfig
[atlpan@aipanda007 esTest]$ python submitTaskWithES_21.0.15c.py BNL_PROD_MCORE WORLD BNL-ATLAS
(0, (True, 11880474L))
Task ID = 11880474
[atlpan@aipanda007 esTest]$ python /data/atlpan/srv/lib/python2.6/site-packages/pandaserver/test/killTaskJEDI.py 11880474
(2, 'jediTaskID=11880474 not found')
[atlpan@aipanda007 esTest]$ python submitTaskWithES_21.0.15c.py BNL_PROD_MCORE_TEST WORLD BNL-ATLAS
(0, (True, 11880478L))
Task ID = 11880478
[atlpan@aipanda007 esTest]$ emacs submitTaskWithES_21.0.15c.py
[atlpan@aipanda007 esTest]$ exit
logout
Connection to aipanda007 closed.
hepix: >>>>> AFS token expired! <<<<<
[nilspal@aiadm23 ~]$ logout
Connection to aiadm.cern.ch closed.
macnilsson6:~ nilsnilsson$ ssh pnilsson@lxplus
Password:
* ********************************************************************
* Welcome to lxplus085.cern.ch, SLC, 6.9
* Archive of news is available in /etc/motd-archive
* Reminder: You have agreed to comply with the CERN computing rules
* https://cern.ch/ComputingRules
* Puppet environment: production, Roger state: production
* Foreman hostgroup: lxplus/nodes/login
* LXPLUS Public Login Service
* ********************************************************************
[pnilsson@lxplus085 ~]$ cd www
[pnilsson@lxplus085 ~/www]$ logout
Connection to lxplus closed.
macnilsson6:~ nilsnilsson$ ssh nilspal@lxplus
Password:
* ********************************************************************
* Welcome to lxplus085.cern.ch, SLC, 6.9
* Archive of news is available in /etc/motd-archive
* Reminder: You have agreed to comply with the CERN computing rules
* https://cern.ch/ComputingRules
* Puppet environment: production, Roger state: production
* Foreman hostgroup: lxplus/nodes/login
* LXPLUS Public Login Service
* ********************************************************************
[nilspal@lxplus085 ~]$ cd dev/pilot-66
pilot-66.0/ pilot-66.3-dev/ pilot-66.3/
[nilspal@lxplus085 ~]$ cd dev/pilot-66.3
[nilspal@lxplus085 pilot-66.3]$ emacs CHANGES
[nilspal@lxplus085 pilot-66.3]$ cd ../pilot-67
pilot-67-dev/ pilot-67.0/ pilot-67.1-bad/ pilot-67.1/ pilot-67.3/ pilot-67.3b/ pilot-67.4/ pilot-67.5/ pilot-67.5dev/ pilot-67.6/
[nilspal@lxplus085 pilot-66.3]$ cd ../pilot-67.0
[nilspal@lxplus085 pilot-67.0]$ emacs CHANGES
[nilspal@lxplus085 pilot-67.0]$ cd ../pilot-67.1
[nilspal@lxplus085 pilot-67.1]$ emacs CHANGES
File Edit Options Buffers Tools Help
67.1
General changes
- Removed any initial 0's from the locally calculated checksums (for both stage-in/out) in case there are discrepancies.
Externally calculated Adler32 values are not necessarily of fixed length and might not include initial 0 values.
Discussed in JIRA ticket https://its.cern.ch/jira/browse/ATLASPANDA-324 (movers/base)
Updates from Mario Lassnig
https://github.com/PanDAWMS/pilot/pull/90
- More verbosity to make debugging easier
Commit Summary
- ruciomover: tiny updates
File Changes
M movers/rucio_sitemover.py (26)
https://github.com/PanDAWMS/pilot/pull/91
Commit Summary
- storm: fix for timestamp corner case
File Changes
M movers/storm_sitemover.py (13)
Updates from Wen Guan
https://github.com/PanDAWMS/pilot/pull/92
Commit Summary
- fixed undefined runCommandList
File Changes
M RunJobEvent.py (10)
Updates from David Cameron
https://github.com/PanDAWMS/pilot/pull/93
Commit Summary
- mv supports all schemes; use HOME as top-level job dir
- add checksum to output file list
File Changes
M movers/mv_sitemover.py (29)
No comments:
Post a Comment