[2022-11-11T21:35:31+0100] [MainThread] [I] [cwltool] Resolved '/project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow/download_and_run_bf_remove.cwl' to 'file:///project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow/download_and_run_bf_remove.cwl' [2022-11-11T21:35:49+0100] [MainThread] [I] [toil.job] Saving graph of 2 jobs, 2 new [2022-11-11T21:35:49+0100] [MainThread] [I] [toil.job] Processing job 'ResolveIndirect' kind-ResolveIndirect/instance-qrcs2prq v0 [2022-11-11T21:35:49+0100] [MainThread] [I] [toil.job] Processing job 'CWLWorkflow' kind-CWLWorkflow/instance-9xuk6zmh v0 [2022-11-11T21:35:49+0100] [MainThread] [I] [toil] Running Toil version 5.7.1-b5cae9634820d76cb6c13b2a6312895122017d54 on host wn-hb-03.spider.surfsara.nl. [2022-11-11T21:35:50+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/9cb97150609e52169ad13929002bf97a/4245/worker_log.txt [2022-11-11T21:35:50+0100] [MainThread] [I] [toil.leader] 0 jobs are running, 0 jobs are issued and waiting to run [2022-11-11T21:35:51+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/9cb97150609e52169ad13929002bf97a/e761/worker_log.txt [2022-11-11T21:35:51+0100] [MainThread] [I] [toil.leader] Issued job 'CWLJob' download_and_run_bf_remove.cwl.0.fetchdata kind-CWLJob/instance-k_qe84gj v1 with job batch system ID: 2 and cores: 1, disk: 3.0 Gi, and memory: 2.0 Gi [2022-11-11T21:35:52+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/9cb97150609e52169ad13929002bf97a/1c4c/worker_log.txt [2022-11-11T21:36:07+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/9cb97150609e52169ad13929002bf97a/4e7f/worker_log.txt [2022-11-11T21:36:08+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/9cb97150609e52169ad13929002bf97a/8302/worker_log.txt [2022-11-11T21:36:09+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/9cb97150609e52169ad13929002bf97a/7bfd/worker_log.txt [2022-11-11T21:36:10+0100] [MainThread] [I] [toil.leader] Issued job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v1 with job batch system ID: 6 and cores: 1, disk: 3.0 Gi, and memory: 2.0 Gi [2022-11-11T21:36:10+0100] [MainThread] [I] [toil.leader] Issued job 'CWLJob' bf_remove_double_tgz.cwl.eliminate_double_tgz.double_tgz_elimination.cwl kind-CWLJob/instance-f2cv1hb1 v1 with job batch system ID: 7 and cores: 1, disk: 3.0 Gi, and memory: 2.0 Gi [2022-11-11T21:36:11+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/9cb97150609e52169ad13929002bf97a/2768/worker_log.txt [2022-11-11T21:36:11+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/9cb97150609e52169ad13929002bf97a/272b/worker_log.txt [2022-11-11T21:36:19+0100] [Thread-9 ] [E] [toil.batchSystems.singleMachine] Got exit code 1 (indicating failure) from job _toil_worker CWLJob file:/tmp/tmpyhse2inu kind-CWLJob/instance-pm_q4en9. [2022-11-11T21:36:19+0100] [MainThread] [W] [toil.leader] Job failed with exit value 1: 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v1 Exit reason: None [2022-11-11T21:36:19+0100] [MainThread] [W] [toil.leader] The job seems to have left a log file, indicating failure: 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v2 [2022-11-11T21:36:19+0100] [MainThread] [W] [toil.leader] Log from job "kind-CWLJob/instance-pm_q4en9" follows: =========> [2022-11-11T21:36:11+0100] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG--- [2022-11-11T21:36:11+0100] [MainThread] [I] [toil] Running Toil version 5.7.1-b5cae9634820d76cb6c13b2a6312895122017d54 on host wn-hb-03.spider.surfsara.nl. [2022-11-11T21:36:11+0100] [MainThread] [I] [toil.worker] Working on job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v1 [2022-11-11T21:36:12+0100] [MainThread] [I] [toil.worker] Loaded body Job('CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v1) from description 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v1 [2022-11-11T21:36:12+0100] [MainThread] [W] [cwltool] You have specified `--preserve-entire-environment` while running a container which will override variables set in the container. This may break the container, be non-portable, and/or affect reproducibility. [2022-11-11T21:36:13+0100] [MainThread] [I] [cwltool] Using local copy of Singularity image found in /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow [2022-11-11T21:36:14+0100] [MainThread] [I] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] /tmp/tmpb2d7tunz$ singularity \ --quiet \ exec \ --contain \ --ipc \ --cleanenv \ --userns \ --home \ /tmp/tmpb2d7tunz:/yZTIUb \ --bind \ /tmp/tmpcedigwb2:/tmp \ --bind \ /project/ldv/Data/beamformed/pulp-logs:/var/lib/cwl/stgd109b25d-5f02-47ba-b717-4ca47be22e38/pulp-logs:ro \ --pwd \ /yZTIUb \ --net \ --network \ none \ /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow/git.astron.nl:5000_ldv_bf_double_tgz:latest.sif \ collect_unspecified_metadata \ 81045 \ LOFAR_PULSAR_ARCHIVE_locus093_L81045_CVplots_ea0c6a37.tar \ --rootdir \ /var/lib/cwl/stgd109b25d-5f02-47ba-b717-4ca47be22e38/pulp-logs WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_MEM_PER_CPU as environment variable will not be supported in the future, use APPTAINERENV_SLURM_MEM_PER_CPU instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SRUN_DEBUG as environment variable will not be supported in the future, use APPTAINERENV_SRUN_DEBUG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODEID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODEID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASK_PID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASK_PID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CONNECTION as environment variable will not be supported in the future, use APPTAINERENV_SSH_CONNECTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PRIO_PROCESS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PRIO_PROCESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_VERBOSE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_VERBOSE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV__ as environment variable will not be supported in the future, use APPTAINERENV__ instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LANG as environment variable will not be supported in the future, use APPTAINERENV_LANG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_DIR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTCONTROL as environment variable will not be supported in the future, use APPTAINERENV_HISTCONTROL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HOSTNAME as environment variable will not be supported in the future, use APPTAINERENV_HOSTNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEPID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEPID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_DISTRIBUTION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_DISTRIBUTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_ENVIRONMENT as environment variable will not be supported in the future, use APPTAINERENV_ENVIRONMENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_KDEDIRS as environment variable will not be supported in the future, use APPTAINERENV_KDEDIRS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PROCID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PROCID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_GID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_GID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURMD_NODENAME as environment variable will not be supported in the future, use APPTAINERENV_SLURMD_NODENAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_VIRTUAL_ENV as environment variable will not be supported in the future, use APPTAINERENV_VIRTUAL_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_S_COLORS as environment variable will not be supported in the future, use APPTAINERENV_S_COLORS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_SERVICES_XML as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_SERVICES_XML instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_which_declare as environment variable will not be supported in the future, use APPTAINERENV_which_declare instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_TCP_PORT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_TCP_PORT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_SESSION_ID as environment variable will not be supported in the future, use APPTAINERENV_XDG_SESSION_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_PLUGIN as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_PLUGIN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_USER as environment variable will not be supported in the future, use APPTAINERENV_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NNODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NNODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LAUNCH_NODE_IPADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LAUNCH_NODE_IPADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_ROLE_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_ROLE_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GRID_ENV_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GRID_ENV_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PWD as environment variable will not be supported in the future, use APPTAINERENV_PWD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MYPROXY_SERVER as environment variable will not be supported in the future, use APPTAINERENV_MYPROXY_SERVER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CLUSTER_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CLUSTER_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CLIENT as environment variable will not be supported in the future, use APPTAINERENV_SSH_CLIENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_VERSION as environment variable will not be supported in the future, use APPTAINERENV_LMOD_VERSION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NTASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NTASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_UMASK as environment variable will not be supported in the future, use APPTAINERENV_SLURM_UMASK instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_LEVEL_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_LEVEL_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_ENV as environment variable will not be supported in the future, use APPTAINERENV_BASH_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SPIDER_LOGIN_UI as environment variable will not be supported in the future, use APPTAINERENV_SPIDER_LOGIN_UI instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_WORKING_CLUSTER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_WORKING_CLUSTER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SHELL_PRGM as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SHELL_PRGM instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_TMPDIR as environment variable will not be supported in the future, use APPTAINERENV_TMPDIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_sys as environment variable will not be supported in the future, use APPTAINERENV_LMOD_sys instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_LCG_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOBID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOBID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CONF as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CONF instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODE_ALIASES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODE_ALIASES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_ROOT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_QOS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_QOS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR_PATTERN as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR_PATTERN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GT_PROXY_MODE as environment variable will not be supported in the future, use APPTAINERENV_GT_PROXY_MODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPUS_ON_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPUS_ON_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHELL as environment variable will not be supported in the future, use APPTAINERENV_SHELL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_UID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_UID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_X509_USER_PROXY as environment variable will not be supported in the future, use APPTAINERENV_X509_USER_PROXY instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_PARTITION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_PARTITION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_USE_CURRENT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_USE_CURRENT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_LIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_LIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_USER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NPROCS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NPROCS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHLVL as environment variable will not be supported in the future, use APPTAINERENV_SHLVL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ACCOUNT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ACCOUNT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MANPATH as environment variable will not be supported in the future, use APPTAINERENV_MANPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_CWL_SINGULARITY_CACHE as environment variable will not be supported in the future, use APPTAINERENV_CWL_SINGULARITY_CACHE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_LAUNCHER_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_LAUNCHER_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION_VAR as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION_VAR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_GTIDS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_GTIDS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LOGNAME as environment variable will not be supported in the future, use APPTAINERENV_LOGNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DBUS_SESSION_BUS_ADDRESS as environment variable will not be supported in the future, use APPTAINERENV_DBUS_SESSION_BUS_ADDRESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_RUNTIME_DIR as environment variable will not be supported in the future, use APPTAINERENV_XDG_RUNTIME_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH_ROOT as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PATH as environment variable will not be supported in the future, use APPTAINERENV_SING_USER_DEFINED_PATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_TYPE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_TYPE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DEBUGINFOD_URLS as environment variable will not be supported in the future, use APPTAINERENV_DEBUGINFOD_URLS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_TASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_TASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULESHOME as environment variable will not be supported in the future, use APPTAINERENV_MODULESHOME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SETTARG_FULL_SUPPORT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SETTARG_FULL_SUPPORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTSIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTSIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_PKG as environment variable will not be supported in the future, use APPTAINERENV_LMOD_PKG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTFILESIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTFILESIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_CMD as environment variable will not be supported in the future, use APPTAINERENV_LMOD_CMD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LOCALID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LOCALID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_GFAL_INFOSYS as environment variable will not be supported in the future, use APPTAINERENV_LCG_GFAL_INFOSYS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_OMP_NUM_THREADS as environment variable will not be supported in the future, use APPTAINERENV_OMP_NUM_THREADS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LESSOPEN as environment variable will not be supported in the future, use APPTAINERENV_LESSOPEN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_DIR as environment variable will not be supported in the future, use APPTAINERENV_LMOD_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_which%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_which%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_module%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_module%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_ml%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_ml%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PS1 as environment variable will not be supported in the future, use APPTAINERENV_PS1 instead source: /.inject-apptainer-env.sh:114:11: invalid var name (E) Pipeline for the SASid 81045 has probably failed [2022-11-11T21:36:19+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] exited with status: 1 [2022-11-11T21:36:19+0100] [MainThread] [E] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] Job error: Error validating output record. the `output_json` field is not valid because 'Any' type must be non-null in { "output_json": null } [2022-11-11T21:36:19+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] completed permanentFail Traceback (most recent call last): File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/worker.py", line 407, in workerScript job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2406, in _runner returnValues = self._run(jobGraph=None, fileStore=fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2324, in _run return self.run(fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/cwl/cwltoil.py", line 2217, in run raise cwltool.errors.WorkflowException(status) cwltool.errors.WorkflowException: permanentFail [2022-11-11T21:36:19+0100] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host wn-hb-03.spider.surfsara.nl <========= [2022-11-11T21:36:19+0100] [MainThread] [W] [toil.job] Due to failure we are reducing the remaining try count of job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v2 with ID kind-CWLJob/instance-pm_q4en9 to 3 [2022-11-11T21:36:19+0100] [MainThread] [I] [toil.leader] Issued job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v3 with job batch system ID: 8 and cores: 1, disk: 3.0 Gi, and memory: 2.0 Gi [2022-11-11T21:36:20+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/9cb97150609e52169ad13929002bf97a/ba7d/worker_log.txt [2022-11-11T21:36:25+0100] [MainThread] [I] [toil.leader] Issued job 'CWLJob' bf_remove_double_tgz.cwl.compute_md5sum.computemd5.cwl kind-CWLJob/instance-f7q183wo v1 with job batch system ID: 9 and cores: 1, disk: 3.0 Gi, and memory: 2.0 Gi [2022-11-11T21:36:25+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/9cb97150609e52169ad13929002bf97a/0b7a/worker_log.txt [2022-11-11T21:36:29+0100] [Thread-9 ] [E] [toil.batchSystems.singleMachine] Got exit code 1 (indicating failure) from job _toil_worker CWLJob file:/tmp/tmpyhse2inu kind-CWLJob/instance-pm_q4en9. [2022-11-11T21:36:29+0100] [MainThread] [W] [toil.leader] Job failed with exit value 1: 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v3 Exit reason: None [2022-11-11T21:36:29+0100] [MainThread] [W] [toil.leader] The job seems to have left a log file, indicating failure: 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v5 [2022-11-11T21:36:29+0100] [MainThread] [W] [toil.leader] Log from job "kind-CWLJob/instance-pm_q4en9" follows: =========> [2022-11-11T21:36:20+0100] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG--- [2022-11-11T21:36:20+0100] [MainThread] [I] [toil] Running Toil version 5.7.1-b5cae9634820d76cb6c13b2a6312895122017d54 on host wn-hb-03.spider.surfsara.nl. [2022-11-11T21:36:20+0100] [MainThread] [I] [toil.worker] Working on job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v4 [2022-11-11T21:36:21+0100] [MainThread] [I] [toil.worker] Loaded body Job('CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v4) from description 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v4 [2022-11-11T21:36:22+0100] [MainThread] [W] [cwltool] You have specified `--preserve-entire-environment` while running a container which will override variables set in the container. This may break the container, be non-portable, and/or affect reproducibility. [2022-11-11T21:36:22+0100] [MainThread] [I] [cwltool] Using local copy of Singularity image found in /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow [2022-11-11T21:36:23+0100] [MainThread] [I] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] /tmp/tmpstgkp780$ singularity \ --quiet \ exec \ --contain \ --ipc \ --cleanenv \ --userns \ --home \ /tmp/tmpstgkp780:/BsNJZa \ --bind \ /tmp/tmpdfph1cde:/tmp \ --bind \ /project/ldv/Data/beamformed/pulp-logs:/var/lib/cwl/stg0226af06-b79e-4b00-8b75-a92885dcafdd/pulp-logs:ro \ --pwd \ /BsNJZa \ --net \ --network \ none \ /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow/git.astron.nl:5000_ldv_bf_double_tgz:latest.sif \ collect_unspecified_metadata \ 81045 \ LOFAR_PULSAR_ARCHIVE_locus093_L81045_CVplots_ea0c6a37.tar \ --rootdir \ /var/lib/cwl/stg0226af06-b79e-4b00-8b75-a92885dcafdd/pulp-logs WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_MEM_PER_CPU as environment variable will not be supported in the future, use APPTAINERENV_SLURM_MEM_PER_CPU instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SRUN_DEBUG as environment variable will not be supported in the future, use APPTAINERENV_SRUN_DEBUG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODEID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODEID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASK_PID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASK_PID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CONNECTION as environment variable will not be supported in the future, use APPTAINERENV_SSH_CONNECTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PRIO_PROCESS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PRIO_PROCESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_VERBOSE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_VERBOSE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV__ as environment variable will not be supported in the future, use APPTAINERENV__ instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LANG as environment variable will not be supported in the future, use APPTAINERENV_LANG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_DIR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTCONTROL as environment variable will not be supported in the future, use APPTAINERENV_HISTCONTROL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HOSTNAME as environment variable will not be supported in the future, use APPTAINERENV_HOSTNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEPID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEPID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_DISTRIBUTION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_DISTRIBUTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_ENVIRONMENT as environment variable will not be supported in the future, use APPTAINERENV_ENVIRONMENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_KDEDIRS as environment variable will not be supported in the future, use APPTAINERENV_KDEDIRS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PROCID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PROCID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_GID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_GID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURMD_NODENAME as environment variable will not be supported in the future, use APPTAINERENV_SLURMD_NODENAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_VIRTUAL_ENV as environment variable will not be supported in the future, use APPTAINERENV_VIRTUAL_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_S_COLORS as environment variable will not be supported in the future, use APPTAINERENV_S_COLORS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_SERVICES_XML as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_SERVICES_XML instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_which_declare as environment variable will not be supported in the future, use APPTAINERENV_which_declare instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_TCP_PORT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_TCP_PORT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_SESSION_ID as environment variable will not be supported in the future, use APPTAINERENV_XDG_SESSION_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_PLUGIN as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_PLUGIN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_USER as environment variable will not be supported in the future, use APPTAINERENV_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NNODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NNODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LAUNCH_NODE_IPADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LAUNCH_NODE_IPADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_ROLE_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_ROLE_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GRID_ENV_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GRID_ENV_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PWD as environment variable will not be supported in the future, use APPTAINERENV_PWD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MYPROXY_SERVER as environment variable will not be supported in the future, use APPTAINERENV_MYPROXY_SERVER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CLUSTER_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CLUSTER_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CLIENT as environment variable will not be supported in the future, use APPTAINERENV_SSH_CLIENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_VERSION as environment variable will not be supported in the future, use APPTAINERENV_LMOD_VERSION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NTASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NTASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_UMASK as environment variable will not be supported in the future, use APPTAINERENV_SLURM_UMASK instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_LEVEL_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_LEVEL_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_ENV as environment variable will not be supported in the future, use APPTAINERENV_BASH_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SPIDER_LOGIN_UI as environment variable will not be supported in the future, use APPTAINERENV_SPIDER_LOGIN_UI instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_WORKING_CLUSTER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_WORKING_CLUSTER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SHELL_PRGM as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SHELL_PRGM instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_TMPDIR as environment variable will not be supported in the future, use APPTAINERENV_TMPDIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_sys as environment variable will not be supported in the future, use APPTAINERENV_LMOD_sys instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_LCG_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOBID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOBID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CONF as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CONF instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODE_ALIASES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODE_ALIASES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_ROOT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_QOS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_QOS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR_PATTERN as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR_PATTERN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GT_PROXY_MODE as environment variable will not be supported in the future, use APPTAINERENV_GT_PROXY_MODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPUS_ON_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPUS_ON_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHELL as environment variable will not be supported in the future, use APPTAINERENV_SHELL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_UID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_UID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_X509_USER_PROXY as environment variable will not be supported in the future, use APPTAINERENV_X509_USER_PROXY instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_PARTITION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_PARTITION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_USE_CURRENT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_USE_CURRENT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_LIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_LIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_USER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NPROCS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NPROCS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHLVL as environment variable will not be supported in the future, use APPTAINERENV_SHLVL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ACCOUNT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ACCOUNT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MANPATH as environment variable will not be supported in the future, use APPTAINERENV_MANPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_CWL_SINGULARITY_CACHE as environment variable will not be supported in the future, use APPTAINERENV_CWL_SINGULARITY_CACHE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_LAUNCHER_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_LAUNCHER_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION_VAR as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION_VAR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_GTIDS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_GTIDS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LOGNAME as environment variable will not be supported in the future, use APPTAINERENV_LOGNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DBUS_SESSION_BUS_ADDRESS as environment variable will not be supported in the future, use APPTAINERENV_DBUS_SESSION_BUS_ADDRESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_RUNTIME_DIR as environment variable will not be supported in the future, use APPTAINERENV_XDG_RUNTIME_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH_ROOT as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PATH as environment variable will not be supported in the future, use APPTAINERENV_SING_USER_DEFINED_PATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_TYPE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_TYPE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DEBUGINFOD_URLS as environment variable will not be supported in the future, use APPTAINERENV_DEBUGINFOD_URLS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_TASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_TASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULESHOME as environment variable will not be supported in the future, use APPTAINERENV_MODULESHOME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SETTARG_FULL_SUPPORT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SETTARG_FULL_SUPPORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTSIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTSIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_PKG as environment variable will not be supported in the future, use APPTAINERENV_LMOD_PKG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTFILESIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTFILESIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_CMD as environment variable will not be supported in the future, use APPTAINERENV_LMOD_CMD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LOCALID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LOCALID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_GFAL_INFOSYS as environment variable will not be supported in the future, use APPTAINERENV_LCG_GFAL_INFOSYS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_OMP_NUM_THREADS as environment variable will not be supported in the future, use APPTAINERENV_OMP_NUM_THREADS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LESSOPEN as environment variable will not be supported in the future, use APPTAINERENV_LESSOPEN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_DIR as environment variable will not be supported in the future, use APPTAINERENV_LMOD_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_which%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_which%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_module%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_module%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_ml%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_ml%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PS1 as environment variable will not be supported in the future, use APPTAINERENV_PS1 instead source: /.inject-apptainer-env.sh:139:11: invalid var name (E) Pipeline for the SASid 81045 has probably failed [2022-11-11T21:36:28+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] exited with status: 1 [2022-11-11T21:36:28+0100] [MainThread] [E] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] Job error: Error validating output record. the `output_json` field is not valid because 'Any' type must be non-null in { "output_json": null } [2022-11-11T21:36:28+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] completed permanentFail Traceback (most recent call last): File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/worker.py", line 407, in workerScript job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2406, in _runner returnValues = self._run(jobGraph=None, fileStore=fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2324, in _run return self.run(fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/cwl/cwltoil.py", line 2217, in run raise cwltool.errors.WorkflowException(status) cwltool.errors.WorkflowException: permanentFail [2022-11-11T21:36:28+0100] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host wn-hb-03.spider.surfsara.nl <========= [2022-11-11T21:36:29+0100] [MainThread] [W] [toil.job] Due to failure we are reducing the remaining try count of job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v5 with ID kind-CWLJob/instance-pm_q4en9 to 2 [2022-11-11T21:36:29+0100] [MainThread] [I] [toil.leader] Issued job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v6 with job batch system ID: 10 and cores: 1, disk: 3.0 Gi, and memory: 2.0 Gi [2022-11-11T21:36:29+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/9cb97150609e52169ad13929002bf97a/241a/worker_log.txt [2022-11-11T21:36:34+0100] [Thread-9 ] [E] [toil.batchSystems.singleMachine] Got exit code 1 (indicating failure) from job _toil_worker CWLJob file:/tmp/tmpyhse2inu kind-CWLJob/instance-pm_q4en9. [2022-11-11T21:36:34+0100] [MainThread] [W] [toil.leader] Job failed with exit value 1: 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v6 Exit reason: None [2022-11-11T21:36:34+0100] [MainThread] [W] [toil.leader] The job seems to have left a log file, indicating failure: 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v8 [2022-11-11T21:36:34+0100] [MainThread] [W] [toil.leader] Log from job "kind-CWLJob/instance-pm_q4en9" follows: =========> [2022-11-11T21:36:29+0100] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG--- [2022-11-11T21:36:29+0100] [MainThread] [I] [toil] Running Toil version 5.7.1-b5cae9634820d76cb6c13b2a6312895122017d54 on host wn-hb-03.spider.surfsara.nl. [2022-11-11T21:36:29+0100] [MainThread] [I] [toil.worker] Working on job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v7 [2022-11-11T21:36:29+0100] [MainThread] [I] [toil.worker] Loaded body Job('CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v7) from description 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v7 [2022-11-11T21:36:30+0100] [MainThread] [W] [cwltool] You have specified `--preserve-entire-environment` while running a container which will override variables set in the container. This may break the container, be non-portable, and/or affect reproducibility. [2022-11-11T21:36:30+0100] [MainThread] [I] [cwltool] Using local copy of Singularity image found in /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow [2022-11-11T21:36:30+0100] [MainThread] [I] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] /tmp/tmpwfsisxk5$ singularity \ --quiet \ exec \ --contain \ --ipc \ --cleanenv \ --userns \ --home \ /tmp/tmpwfsisxk5:/MANcSo \ --bind \ /tmp/tmpvlk_ir5x:/tmp \ --bind \ /project/ldv/Data/beamformed/pulp-logs:/var/lib/cwl/stg3f2e0581-d43a-408e-84ea-950555bfdf0c/pulp-logs:ro \ --pwd \ /MANcSo \ --net \ --network \ none \ /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow/git.astron.nl:5000_ldv_bf_double_tgz:latest.sif \ collect_unspecified_metadata \ 81045 \ LOFAR_PULSAR_ARCHIVE_locus093_L81045_CVplots_ea0c6a37.tar \ --rootdir \ /var/lib/cwl/stg3f2e0581-d43a-408e-84ea-950555bfdf0c/pulp-logs WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_MEM_PER_CPU as environment variable will not be supported in the future, use APPTAINERENV_SLURM_MEM_PER_CPU instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SRUN_DEBUG as environment variable will not be supported in the future, use APPTAINERENV_SRUN_DEBUG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODEID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODEID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASK_PID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASK_PID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CONNECTION as environment variable will not be supported in the future, use APPTAINERENV_SSH_CONNECTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PRIO_PROCESS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PRIO_PROCESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_VERBOSE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_VERBOSE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV__ as environment variable will not be supported in the future, use APPTAINERENV__ instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LANG as environment variable will not be supported in the future, use APPTAINERENV_LANG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_DIR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTCONTROL as environment variable will not be supported in the future, use APPTAINERENV_HISTCONTROL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HOSTNAME as environment variable will not be supported in the future, use APPTAINERENV_HOSTNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEPID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEPID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_DISTRIBUTION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_DISTRIBUTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_ENVIRONMENT as environment variable will not be supported in the future, use APPTAINERENV_ENVIRONMENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_KDEDIRS as environment variable will not be supported in the future, use APPTAINERENV_KDEDIRS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PROCID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PROCID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_GID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_GID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURMD_NODENAME as environment variable will not be supported in the future, use APPTAINERENV_SLURMD_NODENAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_VIRTUAL_ENV as environment variable will not be supported in the future, use APPTAINERENV_VIRTUAL_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_S_COLORS as environment variable will not be supported in the future, use APPTAINERENV_S_COLORS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_SERVICES_XML as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_SERVICES_XML instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_which_declare as environment variable will not be supported in the future, use APPTAINERENV_which_declare instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_TCP_PORT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_TCP_PORT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_SESSION_ID as environment variable will not be supported in the future, use APPTAINERENV_XDG_SESSION_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_PLUGIN as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_PLUGIN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_USER as environment variable will not be supported in the future, use APPTAINERENV_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NNODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NNODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LAUNCH_NODE_IPADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LAUNCH_NODE_IPADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_ROLE_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_ROLE_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GRID_ENV_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GRID_ENV_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PWD as environment variable will not be supported in the future, use APPTAINERENV_PWD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MYPROXY_SERVER as environment variable will not be supported in the future, use APPTAINERENV_MYPROXY_SERVER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CLUSTER_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CLUSTER_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CLIENT as environment variable will not be supported in the future, use APPTAINERENV_SSH_CLIENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_VERSION as environment variable will not be supported in the future, use APPTAINERENV_LMOD_VERSION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NTASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NTASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_UMASK as environment variable will not be supported in the future, use APPTAINERENV_SLURM_UMASK instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_LEVEL_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_LEVEL_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_ENV as environment variable will not be supported in the future, use APPTAINERENV_BASH_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SPIDER_LOGIN_UI as environment variable will not be supported in the future, use APPTAINERENV_SPIDER_LOGIN_UI instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_WORKING_CLUSTER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_WORKING_CLUSTER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SHELL_PRGM as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SHELL_PRGM instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_TMPDIR as environment variable will not be supported in the future, use APPTAINERENV_TMPDIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_sys as environment variable will not be supported in the future, use APPTAINERENV_LMOD_sys instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_LCG_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOBID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOBID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CONF as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CONF instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODE_ALIASES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODE_ALIASES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_ROOT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_QOS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_QOS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR_PATTERN as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR_PATTERN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GT_PROXY_MODE as environment variable will not be supported in the future, use APPTAINERENV_GT_PROXY_MODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPUS_ON_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPUS_ON_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHELL as environment variable will not be supported in the future, use APPTAINERENV_SHELL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_UID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_UID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_X509_USER_PROXY as environment variable will not be supported in the future, use APPTAINERENV_X509_USER_PROXY instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_PARTITION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_PARTITION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_USE_CURRENT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_USE_CURRENT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_LIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_LIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_USER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NPROCS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NPROCS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHLVL as environment variable will not be supported in the future, use APPTAINERENV_SHLVL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ACCOUNT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ACCOUNT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MANPATH as environment variable will not be supported in the future, use APPTAINERENV_MANPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_CWL_SINGULARITY_CACHE as environment variable will not be supported in the future, use APPTAINERENV_CWL_SINGULARITY_CACHE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_LAUNCHER_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_LAUNCHER_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION_VAR as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION_VAR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_GTIDS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_GTIDS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LOGNAME as environment variable will not be supported in the future, use APPTAINERENV_LOGNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DBUS_SESSION_BUS_ADDRESS as environment variable will not be supported in the future, use APPTAINERENV_DBUS_SESSION_BUS_ADDRESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_RUNTIME_DIR as environment variable will not be supported in the future, use APPTAINERENV_XDG_RUNTIME_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH_ROOT as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PATH as environment variable will not be supported in the future, use APPTAINERENV_SING_USER_DEFINED_PATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_TYPE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_TYPE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DEBUGINFOD_URLS as environment variable will not be supported in the future, use APPTAINERENV_DEBUGINFOD_URLS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_TASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_TASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULESHOME as environment variable will not be supported in the future, use APPTAINERENV_MODULESHOME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SETTARG_FULL_SUPPORT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SETTARG_FULL_SUPPORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTSIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTSIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_PKG as environment variable will not be supported in the future, use APPTAINERENV_LMOD_PKG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTFILESIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTFILESIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_CMD as environment variable will not be supported in the future, use APPTAINERENV_LMOD_CMD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LOCALID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LOCALID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_GFAL_INFOSYS as environment variable will not be supported in the future, use APPTAINERENV_LCG_GFAL_INFOSYS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_OMP_NUM_THREADS as environment variable will not be supported in the future, use APPTAINERENV_OMP_NUM_THREADS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LESSOPEN as environment variable will not be supported in the future, use APPTAINERENV_LESSOPEN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_DIR as environment variable will not be supported in the future, use APPTAINERENV_LMOD_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_which%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_which%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_module%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_module%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_ml%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_ml%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PS1 as environment variable will not be supported in the future, use APPTAINERENV_PS1 instead source: /.inject-apptainer-env.sh:34:11: invalid var name (E) Pipeline for the SASid 81045 has probably failed [2022-11-11T21:36:34+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] exited with status: 1 [2022-11-11T21:36:34+0100] [MainThread] [E] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] Job error: Error validating output record. the `output_json` field is not valid because 'Any' type must be non-null in { "output_json": null } [2022-11-11T21:36:34+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] completed permanentFail Traceback (most recent call last): File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/worker.py", line 407, in workerScript job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2406, in _runner returnValues = self._run(jobGraph=None, fileStore=fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2324, in _run return self.run(fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/cwl/cwltoil.py", line 2217, in run raise cwltool.errors.WorkflowException(status) cwltool.errors.WorkflowException: permanentFail [2022-11-11T21:36:34+0100] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host wn-hb-03.spider.surfsara.nl <========= [2022-11-11T21:36:34+0100] [MainThread] [W] [toil.job] Due to failure we are reducing the remaining try count of job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v8 with ID kind-CWLJob/instance-pm_q4en9 to 1 [2022-11-11T21:36:34+0100] [MainThread] [I] [toil.leader] Issued job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v9 with job batch system ID: 11 and cores: 1, disk: 3.0 Gi, and memory: 2.0 Gi [2022-11-11T21:36:35+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/9cb97150609e52169ad13929002bf97a/5310/worker_log.txt [2022-11-11T21:36:40+0100] [Thread-9 ] [E] [toil.batchSystems.singleMachine] Got exit code 1 (indicating failure) from job _toil_worker CWLJob file:/tmp/tmpyhse2inu kind-CWLJob/instance-pm_q4en9. [2022-11-11T21:36:40+0100] [MainThread] [W] [toil.leader] Job failed with exit value 1: 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v9 Exit reason: None [2022-11-11T21:36:40+0100] [MainThread] [W] [toil.leader] The job seems to have left a log file, indicating failure: 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v11 [2022-11-11T21:36:40+0100] [MainThread] [W] [toil.leader] Log from job "kind-CWLJob/instance-pm_q4en9" follows: =========> [2022-11-11T21:36:35+0100] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG--- [2022-11-11T21:36:35+0100] [MainThread] [I] [toil] Running Toil version 5.7.1-b5cae9634820d76cb6c13b2a6312895122017d54 on host wn-hb-03.spider.surfsara.nl. [2022-11-11T21:36:35+0100] [MainThread] [I] [toil.worker] Working on job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v10 [2022-11-11T21:36:36+0100] [MainThread] [I] [toil.worker] Loaded body Job('CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v10) from description 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v10 [2022-11-11T21:36:36+0100] [MainThread] [W] [cwltool] You have specified `--preserve-entire-environment` while running a container which will override variables set in the container. This may break the container, be non-portable, and/or affect reproducibility. [2022-11-11T21:36:36+0100] [MainThread] [I] [cwltool] Using local copy of Singularity image found in /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow [2022-11-11T21:36:37+0100] [MainThread] [I] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] /tmp/tmpkn1x3d3l$ singularity \ --quiet \ exec \ --contain \ --ipc \ --cleanenv \ --userns \ --home \ /tmp/tmpkn1x3d3l:/hEGRjL \ --bind \ /tmp/tmpp7i325ed:/tmp \ --bind \ /project/ldv/Data/beamformed/pulp-logs:/var/lib/cwl/stg1dca3b18-f0ca-4e35-a004-2b7b02449548/pulp-logs:ro \ --pwd \ /hEGRjL \ --net \ --network \ none \ /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow/git.astron.nl:5000_ldv_bf_double_tgz:latest.sif \ collect_unspecified_metadata \ 81045 \ LOFAR_PULSAR_ARCHIVE_locus093_L81045_CVplots_ea0c6a37.tar \ --rootdir \ /var/lib/cwl/stg1dca3b18-f0ca-4e35-a004-2b7b02449548/pulp-logs WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_MEM_PER_CPU as environment variable will not be supported in the future, use APPTAINERENV_SLURM_MEM_PER_CPU instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SRUN_DEBUG as environment variable will not be supported in the future, use APPTAINERENV_SRUN_DEBUG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODEID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODEID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASK_PID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASK_PID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CONNECTION as environment variable will not be supported in the future, use APPTAINERENV_SSH_CONNECTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PRIO_PROCESS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PRIO_PROCESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_VERBOSE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_VERBOSE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV__ as environment variable will not be supported in the future, use APPTAINERENV__ instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LANG as environment variable will not be supported in the future, use APPTAINERENV_LANG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_DIR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTCONTROL as environment variable will not be supported in the future, use APPTAINERENV_HISTCONTROL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HOSTNAME as environment variable will not be supported in the future, use APPTAINERENV_HOSTNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEPID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEPID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_DISTRIBUTION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_DISTRIBUTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_ENVIRONMENT as environment variable will not be supported in the future, use APPTAINERENV_ENVIRONMENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_KDEDIRS as environment variable will not be supported in the future, use APPTAINERENV_KDEDIRS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PROCID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PROCID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_GID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_GID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURMD_NODENAME as environment variable will not be supported in the future, use APPTAINERENV_SLURMD_NODENAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_VIRTUAL_ENV as environment variable will not be supported in the future, use APPTAINERENV_VIRTUAL_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_S_COLORS as environment variable will not be supported in the future, use APPTAINERENV_S_COLORS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_SERVICES_XML as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_SERVICES_XML instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_which_declare as environment variable will not be supported in the future, use APPTAINERENV_which_declare instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_TCP_PORT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_TCP_PORT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_SESSION_ID as environment variable will not be supported in the future, use APPTAINERENV_XDG_SESSION_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_PLUGIN as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_PLUGIN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_USER as environment variable will not be supported in the future, use APPTAINERENV_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NNODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NNODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LAUNCH_NODE_IPADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LAUNCH_NODE_IPADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_ROLE_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_ROLE_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GRID_ENV_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GRID_ENV_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PWD as environment variable will not be supported in the future, use APPTAINERENV_PWD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MYPROXY_SERVER as environment variable will not be supported in the future, use APPTAINERENV_MYPROXY_SERVER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CLUSTER_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CLUSTER_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CLIENT as environment variable will not be supported in the future, use APPTAINERENV_SSH_CLIENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_VERSION as environment variable will not be supported in the future, use APPTAINERENV_LMOD_VERSION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NTASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NTASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_UMASK as environment variable will not be supported in the future, use APPTAINERENV_SLURM_UMASK instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_LEVEL_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_LEVEL_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_ENV as environment variable will not be supported in the future, use APPTAINERENV_BASH_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SPIDER_LOGIN_UI as environment variable will not be supported in the future, use APPTAINERENV_SPIDER_LOGIN_UI instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_WORKING_CLUSTER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_WORKING_CLUSTER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SHELL_PRGM as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SHELL_PRGM instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_TMPDIR as environment variable will not be supported in the future, use APPTAINERENV_TMPDIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_sys as environment variable will not be supported in the future, use APPTAINERENV_LMOD_sys instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_LCG_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOBID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOBID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CONF as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CONF instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODE_ALIASES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODE_ALIASES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_ROOT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_QOS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_QOS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR_PATTERN as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR_PATTERN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GT_PROXY_MODE as environment variable will not be supported in the future, use APPTAINERENV_GT_PROXY_MODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPUS_ON_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPUS_ON_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHELL as environment variable will not be supported in the future, use APPTAINERENV_SHELL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_UID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_UID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_X509_USER_PROXY as environment variable will not be supported in the future, use APPTAINERENV_X509_USER_PROXY instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_PARTITION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_PARTITION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_USE_CURRENT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_USE_CURRENT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_LIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_LIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_USER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NPROCS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NPROCS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHLVL as environment variable will not be supported in the future, use APPTAINERENV_SHLVL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ACCOUNT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ACCOUNT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MANPATH as environment variable will not be supported in the future, use APPTAINERENV_MANPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_CWL_SINGULARITY_CACHE as environment variable will not be supported in the future, use APPTAINERENV_CWL_SINGULARITY_CACHE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_LAUNCHER_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_LAUNCHER_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION_VAR as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION_VAR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_GTIDS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_GTIDS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LOGNAME as environment variable will not be supported in the future, use APPTAINERENV_LOGNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DBUS_SESSION_BUS_ADDRESS as environment variable will not be supported in the future, use APPTAINERENV_DBUS_SESSION_BUS_ADDRESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_RUNTIME_DIR as environment variable will not be supported in the future, use APPTAINERENV_XDG_RUNTIME_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH_ROOT as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PATH as environment variable will not be supported in the future, use APPTAINERENV_SING_USER_DEFINED_PATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_TYPE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_TYPE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DEBUGINFOD_URLS as environment variable will not be supported in the future, use APPTAINERENV_DEBUGINFOD_URLS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_TASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_TASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULESHOME as environment variable will not be supported in the future, use APPTAINERENV_MODULESHOME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SETTARG_FULL_SUPPORT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SETTARG_FULL_SUPPORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTSIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTSIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_PKG as environment variable will not be supported in the future, use APPTAINERENV_LMOD_PKG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTFILESIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTFILESIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_CMD as environment variable will not be supported in the future, use APPTAINERENV_LMOD_CMD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LOCALID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LOCALID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_GFAL_INFOSYS as environment variable will not be supported in the future, use APPTAINERENV_LCG_GFAL_INFOSYS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_OMP_NUM_THREADS as environment variable will not be supported in the future, use APPTAINERENV_OMP_NUM_THREADS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LESSOPEN as environment variable will not be supported in the future, use APPTAINERENV_LESSOPEN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_DIR as environment variable will not be supported in the future, use APPTAINERENV_LMOD_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_which%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_which%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_module%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_module%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_ml%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_ml%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PS1 as environment variable will not be supported in the future, use APPTAINERENV_PS1 instead source: /.inject-apptainer-env.sh:239:11: invalid var name (E) Pipeline for the SASid 81045 has probably failed [2022-11-11T21:36:40+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] exited with status: 1 [2022-11-11T21:36:40+0100] [MainThread] [E] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] Job error: Error validating output record. the `output_json` field is not valid because 'Any' type must be non-null in { "output_json": null } [2022-11-11T21:36:40+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] completed permanentFail Traceback (most recent call last): File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/worker.py", line 407, in workerScript job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2406, in _runner returnValues = self._run(jobGraph=None, fileStore=fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2324, in _run return self.run(fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/cwl/cwltoil.py", line 2217, in run raise cwltool.errors.WorkflowException(status) cwltool.errors.WorkflowException: permanentFail [2022-11-11T21:36:40+0100] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host wn-hb-03.spider.surfsara.nl <========= [2022-11-11T21:36:40+0100] [MainThread] [W] [toil.job] Due to failure we are reducing the remaining try count of job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v11 with ID kind-CWLJob/instance-pm_q4en9 to 0 [2022-11-11T21:36:40+0100] [MainThread] [W] [toil.leader] Job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v12 is completely failed [2022-11-11T21:36:58+0100] [MainThread] [I] [toil.leader] Finished toil run with 8 failed jobs. [2022-11-11T21:36:58+0100] [MainThread] [I] [toil.leader] Failed jobs at end of the run: 'CWLJob' bf_remove_double_tgz.cwl.compute_md5sum.computemd5.cwl kind-CWLJob/instance-f7q183wo v3 'CWLGather' kind-CWLGather/instance-fwq06qps v3 'CWLWorkflow' kind-CWLWorkflow/instance-9xuk6zmh v2 'CWLWorkflow' kind-CWLWorkflow/instance-lr0yelz7 v2 'CWLScatter' kind-CWLScatter/instance-wj2dm2zb v2 'CWLJob' bf_remove_double_tgz.cwl.eliminate_double_tgz.double_tgz_elimination.cwl kind-CWLJob/instance-f2cv1hb1 v2 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v12 'CWLScatter' kind-CWLScatter/instance-euos6u_u v3 [2022-11-11T21:36:58+0100] [MainThread] [I] [toil.common] Successfully deleted the job store: FileJobStore(/tmp/tmpyhse2inu) Traceback (most recent call last): File "/project/ldv/Software/executor-venv/bin/toil-cwl-runner", line 8, in sys.exit(main()) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/cwl/cwltoil.py", line 3654, in main outobj = toil.start(wf1) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/common.py", line 966, in start return self._runMainLoop(rootJobDescription) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/common.py", line 1386, in _runMainLoop return Leader(config=self.config, File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/leader.py", line 309, in run raise FailedJobsException(self.jobStore, failed_jobs, exit_code=self.recommended_fail_exit_code) toil.leader.FailedJobsException: The job store '/tmp/tmpyhse2inu' contains 8 failed jobs: 'CWLJob' bf_remove_double_tgz.cwl.compute_md5sum.computemd5.cwl kind-CWLJob/instance-f7q183wo v3, 'CWLGather' kind-CWLGather/instance-fwq06qps v3, 'CWLWorkflow' kind-CWLWorkflow/instance-9xuk6zmh v2, 'CWLWorkflow' kind-CWLWorkflow/instance-lr0yelz7 v2, 'CWLScatter' kind-CWLScatter/instance-wj2dm2zb v2, 'CWLJob' bf_remove_double_tgz.cwl.eliminate_double_tgz.double_tgz_elimination.cwl kind-CWLJob/instance-f2cv1hb1 v2, 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v12, 'CWLScatter' kind-CWLScatter/instance-euos6u_u v3 Log from job "'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v12" follows: =========> [2022-11-11T21:36:35+0100] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG--- [2022-11-11T21:36:35+0100] [MainThread] [I] [toil] Running Toil version 5.7.1-b5cae9634820d76cb6c13b2a6312895122017d54 on host wn-hb-03.spider.surfsara.nl. [2022-11-11T21:36:35+0100] [MainThread] [I] [toil.worker] Working on job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v10 [2022-11-11T21:36:36+0100] [MainThread] [I] [toil.worker] Loaded body Job('CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v10) from description 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-pm_q4en9 v10 [2022-11-11T21:36:36+0100] [MainThread] [W] [cwltool] You have specified `--preserve-entire-environment` while running a container which will override variables set in the container. This may break the container, be non-portable, and/or affect reproducibility. [2022-11-11T21:36:36+0100] [MainThread] [I] [cwltool] Using local copy of Singularity image found in /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow [2022-11-11T21:36:37+0100] [MainThread] [I] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] /tmp/tmpkn1x3d3l$ singularity \ --quiet \ exec \ --contain \ --ipc \ --cleanenv \ --userns \ --home \ /tmp/tmpkn1x3d3l:/hEGRjL \ --bind \ /tmp/tmpp7i325ed:/tmp \ --bind \ /project/ldv/Data/beamformed/pulp-logs:/var/lib/cwl/stg1dca3b18-f0ca-4e35-a004-2b7b02449548/pulp-logs:ro \ --pwd \ /hEGRjL \ --net \ --network \ none \ /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow/git.astron.nl:5000_ldv_bf_double_tgz:latest.sif \ collect_unspecified_metadata \ 81045 \ LOFAR_PULSAR_ARCHIVE_locus093_L81045_CVplots_ea0c6a37.tar \ --rootdir \ /var/lib/cwl/stg1dca3b18-f0ca-4e35-a004-2b7b02449548/pulp-logs WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_MEM_PER_CPU as environment variable will not be supported in the future, use APPTAINERENV_SLURM_MEM_PER_CPU instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SRUN_DEBUG as environment variable will not be supported in the future, use APPTAINERENV_SRUN_DEBUG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODEID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODEID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASK_PID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASK_PID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CONNECTION as environment variable will not be supported in the future, use APPTAINERENV_SSH_CONNECTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PRIO_PROCESS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PRIO_PROCESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_VERBOSE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_VERBOSE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV__ as environment variable will not be supported in the future, use APPTAINERENV__ instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LANG as environment variable will not be supported in the future, use APPTAINERENV_LANG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_DIR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTCONTROL as environment variable will not be supported in the future, use APPTAINERENV_HISTCONTROL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HOSTNAME as environment variable will not be supported in the future, use APPTAINERENV_HOSTNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEPID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEPID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_DISTRIBUTION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_DISTRIBUTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_ENVIRONMENT as environment variable will not be supported in the future, use APPTAINERENV_ENVIRONMENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_KDEDIRS as environment variable will not be supported in the future, use APPTAINERENV_KDEDIRS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PROCID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PROCID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_GID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_GID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURMD_NODENAME as environment variable will not be supported in the future, use APPTAINERENV_SLURMD_NODENAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_VIRTUAL_ENV as environment variable will not be supported in the future, use APPTAINERENV_VIRTUAL_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_S_COLORS as environment variable will not be supported in the future, use APPTAINERENV_S_COLORS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_SERVICES_XML as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_SERVICES_XML instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_which_declare as environment variable will not be supported in the future, use APPTAINERENV_which_declare instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_TCP_PORT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_TCP_PORT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_SESSION_ID as environment variable will not be supported in the future, use APPTAINERENV_XDG_SESSION_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_PLUGIN as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_PLUGIN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_USER as environment variable will not be supported in the future, use APPTAINERENV_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NNODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NNODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LAUNCH_NODE_IPADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LAUNCH_NODE_IPADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_ROLE_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_ROLE_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GRID_ENV_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GRID_ENV_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PWD as environment variable will not be supported in the future, use APPTAINERENV_PWD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MYPROXY_SERVER as environment variable will not be supported in the future, use APPTAINERENV_MYPROXY_SERVER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CLUSTER_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CLUSTER_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CLIENT as environment variable will not be supported in the future, use APPTAINERENV_SSH_CLIENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_VERSION as environment variable will not be supported in the future, use APPTAINERENV_LMOD_VERSION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NTASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NTASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_UMASK as environment variable will not be supported in the future, use APPTAINERENV_SLURM_UMASK instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_LEVEL_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_LEVEL_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_ENV as environment variable will not be supported in the future, use APPTAINERENV_BASH_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SPIDER_LOGIN_UI as environment variable will not be supported in the future, use APPTAINERENV_SPIDER_LOGIN_UI instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_WORKING_CLUSTER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_WORKING_CLUSTER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SHELL_PRGM as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SHELL_PRGM instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_TMPDIR as environment variable will not be supported in the future, use APPTAINERENV_TMPDIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_sys as environment variable will not be supported in the future, use APPTAINERENV_LMOD_sys instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_LCG_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOBID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOBID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CONF as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CONF instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODE_ALIASES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODE_ALIASES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_ROOT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_QOS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_QOS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR_PATTERN as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR_PATTERN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GT_PROXY_MODE as environment variable will not be supported in the future, use APPTAINERENV_GT_PROXY_MODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPUS_ON_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPUS_ON_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHELL as environment variable will not be supported in the future, use APPTAINERENV_SHELL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_UID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_UID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_X509_USER_PROXY as environment variable will not be supported in the future, use APPTAINERENV_X509_USER_PROXY instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_PARTITION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_PARTITION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_USE_CURRENT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_USE_CURRENT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_LIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_LIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_USER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NPROCS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NPROCS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHLVL as environment variable will not be supported in the future, use APPTAINERENV_SHLVL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ACCOUNT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ACCOUNT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MANPATH as environment variable will not be supported in the future, use APPTAINERENV_MANPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_CWL_SINGULARITY_CACHE as environment variable will not be supported in the future, use APPTAINERENV_CWL_SINGULARITY_CACHE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_LAUNCHER_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_LAUNCHER_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION_VAR as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION_VAR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_GTIDS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_GTIDS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LOGNAME as environment variable will not be supported in the future, use APPTAINERENV_LOGNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DBUS_SESSION_BUS_ADDRESS as environment variable will not be supported in the future, use APPTAINERENV_DBUS_SESSION_BUS_ADDRESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_RUNTIME_DIR as environment variable will not be supported in the future, use APPTAINERENV_XDG_RUNTIME_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH_ROOT as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PATH as environment variable will not be supported in the future, use APPTAINERENV_SING_USER_DEFINED_PATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_TYPE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_TYPE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DEBUGINFOD_URLS as environment variable will not be supported in the future, use APPTAINERENV_DEBUGINFOD_URLS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_TASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_TASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULESHOME as environment variable will not be supported in the future, use APPTAINERENV_MODULESHOME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SETTARG_FULL_SUPPORT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SETTARG_FULL_SUPPORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTSIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTSIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_PKG as environment variable will not be supported in the future, use APPTAINERENV_LMOD_PKG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTFILESIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTFILESIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_CMD as environment variable will not be supported in the future, use APPTAINERENV_LMOD_CMD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LOCALID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LOCALID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_GFAL_INFOSYS as environment variable will not be supported in the future, use APPTAINERENV_LCG_GFAL_INFOSYS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_OMP_NUM_THREADS as environment variable will not be supported in the future, use APPTAINERENV_OMP_NUM_THREADS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LESSOPEN as environment variable will not be supported in the future, use APPTAINERENV_LESSOPEN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_DIR as environment variable will not be supported in the future, use APPTAINERENV_LMOD_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_which%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_which%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_module%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_module%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_ml%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_ml%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PS1 as environment variable will not be supported in the future, use APPTAINERENV_PS1 instead source: /.inject-apptainer-env.sh:239:11: invalid var name (E) Pipeline for the SASid 81045 has probably failed [2022-11-11T21:36:40+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] exited with status: 1 [2022-11-11T21:36:40+0100] [MainThread] [E] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] Job error: Error validating output record. the `output_json` field is not valid because 'Any' type must be non-null in { "output_json": null } [2022-11-11T21:36:40+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] completed permanentFail Traceback (most recent call last): File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/worker.py", line 407, in workerScript job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2406, in _runner returnValues = self._run(jobGraph=None, fileStore=fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2324, in _run return self.run(fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/cwl/cwltoil.py", line 2217, in run raise cwltool.errors.WorkflowException(status) cwltool.errors.WorkflowException: permanentFail [2022-11-11T21:36:40+0100] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host wn-hb-03.spider.surfsara.nl <========= srun: error: wn-hb-03: task 0: Exited with exit code 1