66.3

Summary

- Updated site mover workflow to handle extra output (split) files produced by the payload ("input pfn file does not exist" issue for MCORE jobs)

- Fixed checksum value extraction from the calc checksum command (CRC mismatch issue)

- Stage-in and stage-out workflow optimization

- Adjusted site mover timeout values and error reports  

Version info

Updates from Alexey Anisenkov:

https://github.com/PanDAWMS/pilot/pull/71

various small sitemover updates:

- movers: base remote_cleanup handler added (fixed lcgcp clean up action)

- fixed getSURL() to avoid transfers to non deterministic sites (quick hack for now based on checking '/rucio' postfix)

- changed sitemover timeout threshold to fixed 1h value

- various cosmetic fixes

https://github.com/PanDAWMS/pilot/pull/72

- fixed checksum value extraction from the calc checksum command

- Updated logic to handle extra output (splitted) files produced by payload (defined in Job report) - ("input pfn file is not exist" issue for MCORE jobs)

- stage-out loop optimization

- lcgcp file clean up: add 1m sleep nap before remote file removal

https://github.com/PanDAWMS/pilot/pull/73

- fixed timeout value

- stage-in workflow optimized (stop processing in case of failed transfers), proper error reporting

- set max timeout value (depending on filesize) up to 3h

- Merge branch 'main-dev' of https://github.com/PanDAWMS/pilot into main-dev

- fixed error reporting

File Changes

M movers/base.py (6)

M movers/mover.py (263)