[2022-11-11T18:34:49+0100] [MainThread] [I] [cwltool] Resolved '/project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow/download_and_run_bf_remove.cwl' to 'file:///project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow/download_and_run_bf_remove.cwl' [2022-11-11T18:34:59+0100] [MainThread] [I] [toil.job] Saving graph of 2 jobs, 2 new [2022-11-11T18:34:59+0100] [MainThread] [I] [toil.job] Processing job 'ResolveIndirect' kind-ResolveIndirect/instance-xw0j7hnb v0 [2022-11-11T18:34:59+0100] [MainThread] [I] [toil.job] Processing job 'CWLWorkflow' kind-CWLWorkflow/instance-qcz3spu5 v0 [2022-11-11T18:34:59+0100] [MainThread] [I] [toil] Running Toil version 5.7.1-b5cae9634820d76cb6c13b2a6312895122017d54 on host wn-ca-07.spider.surfsara.nl. [2022-11-11T18:35:00+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/3dbc01b17d325567bcf725e1375d84eb/b94e/worker_log.txt [2022-11-11T18:35:01+0100] [MainThread] [I] [toil.leader] 0 jobs are running, 0 jobs are issued and waiting to run [2022-11-11T18:35:02+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/3dbc01b17d325567bcf725e1375d84eb/9e22/worker_log.txt [2022-11-11T18:35:02+0100] [MainThread] [I] [toil.leader] Issued job 'CWLJob' download_and_run_bf_remove.cwl.0.fetchdata kind-CWLJob/instance-etbm93i6 v1 with job batch system ID: 2 and cores: 1, disk: 3.0 Gi, and memory: 2.0 Gi [2022-11-11T18:35:02+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/3dbc01b17d325567bcf725e1375d84eb/6165/worker_log.txt [2022-11-11T18:35:14+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/3dbc01b17d325567bcf725e1375d84eb/780e/worker_log.txt [2022-11-11T18:35:15+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/3dbc01b17d325567bcf725e1375d84eb/304b/worker_log.txt [2022-11-11T18:35:16+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/3dbc01b17d325567bcf725e1375d84eb/e7ed/worker_log.txt [2022-11-11T18:35:16+0100] [MainThread] [I] [toil.leader] Issued job 'CWLJob' bf_remove_double_tgz.cwl.eliminate_double_tgz.double_tgz_elimination.cwl kind-CWLJob/instance-djtbvqku v1 with job batch system ID: 6 and cores: 1, disk: 3.0 Gi, and memory: 2.0 Gi [2022-11-11T18:35:16+0100] [MainThread] [I] [toil.leader] Issued job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v1 with job batch system ID: 7 and cores: 1, disk: 3.0 Gi, and memory: 2.0 Gi [2022-11-11T18:35:17+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/3dbc01b17d325567bcf725e1375d84eb/05e9/worker_log.txt [2022-11-11T18:35:17+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/3dbc01b17d325567bcf725e1375d84eb/ca48/worker_log.txt [2022-11-11T18:35:21+0100] [Thread-9 ] [E] [toil.batchSystems.singleMachine] Got exit code 1 (indicating failure) from job _toil_worker CWLJob file:/tmp/tmpjtdgbk5i kind-CWLJob/instance-upr5_lv2. [2022-11-11T18:35:21+0100] [MainThread] [W] [toil.leader] Job failed with exit value 1: 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v1 Exit reason: None [2022-11-11T18:35:21+0100] [MainThread] [W] [toil.leader] The job seems to have left a log file, indicating failure: 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v2 [2022-11-11T18:35:21+0100] [MainThread] [W] [toil.leader] Log from job "kind-CWLJob/instance-upr5_lv2" follows: =========> [2022-11-11T18:35:17+0100] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG--- [2022-11-11T18:35:17+0100] [MainThread] [I] [toil] Running Toil version 5.7.1-b5cae9634820d76cb6c13b2a6312895122017d54 on host wn-ca-07.spider.surfsara.nl. [2022-11-11T18:35:17+0100] [MainThread] [I] [toil.worker] Working on job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v1 [2022-11-11T18:35:17+0100] [MainThread] [I] [toil.worker] Loaded body Job('CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v1) from description 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v1 [2022-11-11T18:35:18+0100] [MainThread] [W] [cwltool] You have specified `--preserve-entire-environment` while running a container which will override variables set in the container. This may break the container, be non-portable, and/or affect reproducibility. [2022-11-11T18:35:18+0100] [MainThread] [I] [cwltool] Using local copy of Singularity image found in /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow [2022-11-11T18:35:18+0100] [MainThread] [I] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] /tmp/tmpq1bsb5cp$ singularity \ --quiet \ exec \ --contain \ --ipc \ --cleanenv \ --userns \ --home \ /tmp/tmpq1bsb5cp:/vUJvfH \ --bind \ /tmp/tmpizjbrs0v:/tmp \ --bind \ /project/ldv/Data/beamformed/pulp-logs:/var/lib/cwl/stgeea6d4b4-f0d9-480a-a871-170e9e148afb/pulp-logs:ro \ --pwd \ /vUJvfH \ --net \ --network \ none \ /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow/git.astron.nl:5000_ldv_bf_double_tgz:latest.sif \ collect_unspecified_metadata \ 81045 \ LOFAR_PULSAR_ARCHIVE_locus093_L81045_CVplots_ea0c6a37.tar \ --rootdir \ /var/lib/cwl/stgeea6d4b4-f0d9-480a-a871-170e9e148afb/pulp-logs WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_MEM_PER_CPU as environment variable will not be supported in the future, use APPTAINERENV_SLURM_MEM_PER_CPU instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SRUN_DEBUG as environment variable will not be supported in the future, use APPTAINERENV_SRUN_DEBUG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODEID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODEID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASK_PID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASK_PID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CONNECTION as environment variable will not be supported in the future, use APPTAINERENV_SSH_CONNECTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PRIO_PROCESS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PRIO_PROCESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_VERBOSE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_VERBOSE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV__ as environment variable will not be supported in the future, use APPTAINERENV__ instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LANG as environment variable will not be supported in the future, use APPTAINERENV_LANG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_DIR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTCONTROL as environment variable will not be supported in the future, use APPTAINERENV_HISTCONTROL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HOSTNAME as environment variable will not be supported in the future, use APPTAINERENV_HOSTNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEPID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEPID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_DISTRIBUTION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_DISTRIBUTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_ENVIRONMENT as environment variable will not be supported in the future, use APPTAINERENV_ENVIRONMENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_KDEDIRS as environment variable will not be supported in the future, use APPTAINERENV_KDEDIRS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PROCID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PROCID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_GID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_GID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURMD_NODENAME as environment variable will not be supported in the future, use APPTAINERENV_SLURMD_NODENAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_VIRTUAL_ENV as environment variable will not be supported in the future, use APPTAINERENV_VIRTUAL_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_S_COLORS as environment variable will not be supported in the future, use APPTAINERENV_S_COLORS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_SERVICES_XML as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_SERVICES_XML instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_which_declare as environment variable will not be supported in the future, use APPTAINERENV_which_declare instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_TCP_PORT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_TCP_PORT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_SESSION_ID as environment variable will not be supported in the future, use APPTAINERENV_XDG_SESSION_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_PLUGIN as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_PLUGIN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_USER as environment variable will not be supported in the future, use APPTAINERENV_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NNODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NNODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LAUNCH_NODE_IPADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LAUNCH_NODE_IPADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_ROLE_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_ROLE_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GRID_ENV_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GRID_ENV_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PWD as environment variable will not be supported in the future, use APPTAINERENV_PWD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MYPROXY_SERVER as environment variable will not be supported in the future, use APPTAINERENV_MYPROXY_SERVER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CLUSTER_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CLUSTER_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CLIENT as environment variable will not be supported in the future, use APPTAINERENV_SSH_CLIENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_VERSION as environment variable will not be supported in the future, use APPTAINERENV_LMOD_VERSION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NTASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NTASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_UMASK as environment variable will not be supported in the future, use APPTAINERENV_SLURM_UMASK instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_LEVEL_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_LEVEL_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_ENV as environment variable will not be supported in the future, use APPTAINERENV_BASH_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SPIDER_LOGIN_UI as environment variable will not be supported in the future, use APPTAINERENV_SPIDER_LOGIN_UI instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_WORKING_CLUSTER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_WORKING_CLUSTER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SHELL_PRGM as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SHELL_PRGM instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_TMPDIR as environment variable will not be supported in the future, use APPTAINERENV_TMPDIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_sys as environment variable will not be supported in the future, use APPTAINERENV_LMOD_sys instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_LCG_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOBID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOBID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CONF as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CONF instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODE_ALIASES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODE_ALIASES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_ROOT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_QOS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_QOS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR_PATTERN as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR_PATTERN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GT_PROXY_MODE as environment variable will not be supported in the future, use APPTAINERENV_GT_PROXY_MODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPUS_ON_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPUS_ON_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHELL as environment variable will not be supported in the future, use APPTAINERENV_SHELL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_UID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_UID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_X509_USER_PROXY as environment variable will not be supported in the future, use APPTAINERENV_X509_USER_PROXY instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_PARTITION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_PARTITION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_USE_CURRENT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_USE_CURRENT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_LIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_LIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_USER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NPROCS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NPROCS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHLVL as environment variable will not be supported in the future, use APPTAINERENV_SHLVL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ACCOUNT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ACCOUNT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MANPATH as environment variable will not be supported in the future, use APPTAINERENV_MANPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_CWL_SINGULARITY_CACHE as environment variable will not be supported in the future, use APPTAINERENV_CWL_SINGULARITY_CACHE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_LAUNCHER_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_LAUNCHER_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION_VAR as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION_VAR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_GTIDS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_GTIDS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LOGNAME as environment variable will not be supported in the future, use APPTAINERENV_LOGNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DBUS_SESSION_BUS_ADDRESS as environment variable will not be supported in the future, use APPTAINERENV_DBUS_SESSION_BUS_ADDRESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_RUNTIME_DIR as environment variable will not be supported in the future, use APPTAINERENV_XDG_RUNTIME_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH_ROOT as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PATH as environment variable will not be supported in the future, use APPTAINERENV_SING_USER_DEFINED_PATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_TYPE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_TYPE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DEBUGINFOD_URLS as environment variable will not be supported in the future, use APPTAINERENV_DEBUGINFOD_URLS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_TASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_TASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULESHOME as environment variable will not be supported in the future, use APPTAINERENV_MODULESHOME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SETTARG_FULL_SUPPORT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SETTARG_FULL_SUPPORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTSIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTSIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_PKG as environment variable will not be supported in the future, use APPTAINERENV_LMOD_PKG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTFILESIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTFILESIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_CMD as environment variable will not be supported in the future, use APPTAINERENV_LMOD_CMD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LOCALID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LOCALID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_GFAL_INFOSYS as environment variable will not be supported in the future, use APPTAINERENV_LCG_GFAL_INFOSYS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_OMP_NUM_THREADS as environment variable will not be supported in the future, use APPTAINERENV_OMP_NUM_THREADS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LESSOPEN as environment variable will not be supported in the future, use APPTAINERENV_LESSOPEN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_DIR as environment variable will not be supported in the future, use APPTAINERENV_LMOD_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_which%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_which%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_module%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_module%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_ml%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_ml%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PS1 as environment variable will not be supported in the future, use APPTAINERENV_PS1 instead source: /.inject-apptainer-env.sh:79:11: invalid var name (E) Pipeline for the SASid 81045 has probably failed [2022-11-11T18:35:20+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] exited with status: 1 [2022-11-11T18:35:20+0100] [MainThread] [E] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] Job error: Error validating output record. the `output_json` field is not valid because 'Any' type must be non-null in { "output_json": null } [2022-11-11T18:35:20+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] completed permanentFail Traceback (most recent call last): File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/worker.py", line 407, in workerScript job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2406, in _runner returnValues = self._run(jobGraph=None, fileStore=fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2324, in _run return self.run(fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/cwl/cwltoil.py", line 2217, in run raise cwltool.errors.WorkflowException(status) cwltool.errors.WorkflowException: permanentFail [2022-11-11T18:35:21+0100] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host wn-ca-07.spider.surfsara.nl <========= [2022-11-11T18:35:21+0100] [MainThread] [W] [toil.job] Due to failure we are reducing the remaining try count of job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v2 with ID kind-CWLJob/instance-upr5_lv2 to 3 [2022-11-11T18:35:21+0100] [MainThread] [I] [toil.leader] Issued job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v3 with job batch system ID: 8 and cores: 1, disk: 3.0 Gi, and memory: 2.0 Gi [2022-11-11T18:35:22+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/3dbc01b17d325567bcf725e1375d84eb/d3fa/worker_log.txt [2022-11-11T18:35:25+0100] [MainThread] [I] [toil.leader] Issued job 'CWLJob' bf_remove_double_tgz.cwl.compute_md5sum.computemd5.cwl kind-CWLJob/instance-qan19d1t v1 with job batch system ID: 9 and cores: 1, disk: 3.0 Gi, and memory: 2.0 Gi [2022-11-11T18:35:25+0100] [Thread-9 ] [E] [toil.batchSystems.singleMachine] Got exit code 1 (indicating failure) from job _toil_worker CWLJob file:/tmp/tmpjtdgbk5i kind-CWLJob/instance-upr5_lv2. [2022-11-11T18:35:25+0100] [MainThread] [W] [toil.leader] Job failed with exit value 1: 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v3 Exit reason: None [2022-11-11T18:35:25+0100] [MainThread] [W] [toil.leader] The job seems to have left a log file, indicating failure: 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v5 [2022-11-11T18:35:25+0100] [MainThread] [W] [toil.leader] Log from job "kind-CWLJob/instance-upr5_lv2" follows: =========> [2022-11-11T18:35:22+0100] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG--- [2022-11-11T18:35:22+0100] [MainThread] [I] [toil] Running Toil version 5.7.1-b5cae9634820d76cb6c13b2a6312895122017d54 on host wn-ca-07.spider.surfsara.nl. [2022-11-11T18:35:22+0100] [MainThread] [I] [toil.worker] Working on job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v4 [2022-11-11T18:35:22+0100] [MainThread] [I] [toil.worker] Loaded body Job('CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v4) from description 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v4 [2022-11-11T18:35:23+0100] [MainThread] [W] [cwltool] You have specified `--preserve-entire-environment` while running a container which will override variables set in the container. This may break the container, be non-portable, and/or affect reproducibility. [2022-11-11T18:35:23+0100] [MainThread] [I] [cwltool] Using local copy of Singularity image found in /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow [2022-11-11T18:35:23+0100] [MainThread] [I] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] /tmp/tmpifxunjmp$ singularity \ --quiet \ exec \ --contain \ --ipc \ --cleanenv \ --userns \ --home \ /tmp/tmpifxunjmp:/KHylgN \ --bind \ /tmp/tmprttr8dl9:/tmp \ --bind \ /project/ldv/Data/beamformed/pulp-logs:/var/lib/cwl/stgb1d8b62f-8e08-4baf-bd29-5ff0dc7cb380/pulp-logs:ro \ --pwd \ /KHylgN \ --net \ --network \ none \ /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow/git.astron.nl:5000_ldv_bf_double_tgz:latest.sif \ collect_unspecified_metadata \ 81045 \ LOFAR_PULSAR_ARCHIVE_locus093_L81045_CVplots_ea0c6a37.tar \ --rootdir \ /var/lib/cwl/stgb1d8b62f-8e08-4baf-bd29-5ff0dc7cb380/pulp-logs WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_MEM_PER_CPU as environment variable will not be supported in the future, use APPTAINERENV_SLURM_MEM_PER_CPU instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SRUN_DEBUG as environment variable will not be supported in the future, use APPTAINERENV_SRUN_DEBUG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODEID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODEID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASK_PID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASK_PID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CONNECTION as environment variable will not be supported in the future, use APPTAINERENV_SSH_CONNECTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PRIO_PROCESS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PRIO_PROCESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_VERBOSE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_VERBOSE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV__ as environment variable will not be supported in the future, use APPTAINERENV__ instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LANG as environment variable will not be supported in the future, use APPTAINERENV_LANG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_DIR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTCONTROL as environment variable will not be supported in the future, use APPTAINERENV_HISTCONTROL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HOSTNAME as environment variable will not be supported in the future, use APPTAINERENV_HOSTNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEPID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEPID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_DISTRIBUTION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_DISTRIBUTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_ENVIRONMENT as environment variable will not be supported in the future, use APPTAINERENV_ENVIRONMENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_KDEDIRS as environment variable will not be supported in the future, use APPTAINERENV_KDEDIRS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PROCID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PROCID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_GID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_GID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURMD_NODENAME as environment variable will not be supported in the future, use APPTAINERENV_SLURMD_NODENAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_VIRTUAL_ENV as environment variable will not be supported in the future, use APPTAINERENV_VIRTUAL_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_S_COLORS as environment variable will not be supported in the future, use APPTAINERENV_S_COLORS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_SERVICES_XML as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_SERVICES_XML instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_which_declare as environment variable will not be supported in the future, use APPTAINERENV_which_declare instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_TCP_PORT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_TCP_PORT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_SESSION_ID as environment variable will not be supported in the future, use APPTAINERENV_XDG_SESSION_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_PLUGIN as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_PLUGIN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_USER as environment variable will not be supported in the future, use APPTAINERENV_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NNODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NNODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LAUNCH_NODE_IPADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LAUNCH_NODE_IPADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_ROLE_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_ROLE_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GRID_ENV_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GRID_ENV_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PWD as environment variable will not be supported in the future, use APPTAINERENV_PWD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MYPROXY_SERVER as environment variable will not be supported in the future, use APPTAINERENV_MYPROXY_SERVER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CLUSTER_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CLUSTER_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CLIENT as environment variable will not be supported in the future, use APPTAINERENV_SSH_CLIENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_VERSION as environment variable will not be supported in the future, use APPTAINERENV_LMOD_VERSION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NTASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NTASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_UMASK as environment variable will not be supported in the future, use APPTAINERENV_SLURM_UMASK instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_LEVEL_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_LEVEL_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_ENV as environment variable will not be supported in the future, use APPTAINERENV_BASH_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SPIDER_LOGIN_UI as environment variable will not be supported in the future, use APPTAINERENV_SPIDER_LOGIN_UI instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_WORKING_CLUSTER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_WORKING_CLUSTER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SHELL_PRGM as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SHELL_PRGM instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_TMPDIR as environment variable will not be supported in the future, use APPTAINERENV_TMPDIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_sys as environment variable will not be supported in the future, use APPTAINERENV_LMOD_sys instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_LCG_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOBID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOBID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CONF as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CONF instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODE_ALIASES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODE_ALIASES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_ROOT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_QOS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_QOS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR_PATTERN as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR_PATTERN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GT_PROXY_MODE as environment variable will not be supported in the future, use APPTAINERENV_GT_PROXY_MODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPUS_ON_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPUS_ON_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHELL as environment variable will not be supported in the future, use APPTAINERENV_SHELL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_UID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_UID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_X509_USER_PROXY as environment variable will not be supported in the future, use APPTAINERENV_X509_USER_PROXY instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_PARTITION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_PARTITION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_USE_CURRENT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_USE_CURRENT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_LIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_LIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_USER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NPROCS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NPROCS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHLVL as environment variable will not be supported in the future, use APPTAINERENV_SHLVL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ACCOUNT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ACCOUNT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MANPATH as environment variable will not be supported in the future, use APPTAINERENV_MANPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_CWL_SINGULARITY_CACHE as environment variable will not be supported in the future, use APPTAINERENV_CWL_SINGULARITY_CACHE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_LAUNCHER_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_LAUNCHER_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION_VAR as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION_VAR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_GTIDS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_GTIDS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LOGNAME as environment variable will not be supported in the future, use APPTAINERENV_LOGNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DBUS_SESSION_BUS_ADDRESS as environment variable will not be supported in the future, use APPTAINERENV_DBUS_SESSION_BUS_ADDRESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_RUNTIME_DIR as environment variable will not be supported in the future, use APPTAINERENV_XDG_RUNTIME_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH_ROOT as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PATH as environment variable will not be supported in the future, use APPTAINERENV_SING_USER_DEFINED_PATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_TYPE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_TYPE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DEBUGINFOD_URLS as environment variable will not be supported in the future, use APPTAINERENV_DEBUGINFOD_URLS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_TASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_TASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULESHOME as environment variable will not be supported in the future, use APPTAINERENV_MODULESHOME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SETTARG_FULL_SUPPORT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SETTARG_FULL_SUPPORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTSIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTSIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_PKG as environment variable will not be supported in the future, use APPTAINERENV_LMOD_PKG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTFILESIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTFILESIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_CMD as environment variable will not be supported in the future, use APPTAINERENV_LMOD_CMD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LOCALID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LOCALID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_GFAL_INFOSYS as environment variable will not be supported in the future, use APPTAINERENV_LCG_GFAL_INFOSYS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_OMP_NUM_THREADS as environment variable will not be supported in the future, use APPTAINERENV_OMP_NUM_THREADS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LESSOPEN as environment variable will not be supported in the future, use APPTAINERENV_LESSOPEN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_DIR as environment variable will not be supported in the future, use APPTAINERENV_LMOD_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_which%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_which%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_module%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_module%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_ml%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_ml%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PS1 as environment variable will not be supported in the future, use APPTAINERENV_PS1 instead source: /.inject-apptainer-env.sh:209:11: invalid var name (E) Pipeline for the SASid 81045 has probably failed [2022-11-11T18:35:24+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] exited with status: 1 [2022-11-11T18:35:24+0100] [MainThread] [E] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] Job error: Error validating output record. the `output_json` field is not valid because 'Any' type must be non-null in { "output_json": null } [2022-11-11T18:35:24+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] completed permanentFail Traceback (most recent call last): File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/worker.py", line 407, in workerScript job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2406, in _runner returnValues = self._run(jobGraph=None, fileStore=fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2324, in _run return self.run(fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/cwl/cwltoil.py", line 2217, in run raise cwltool.errors.WorkflowException(status) cwltool.errors.WorkflowException: permanentFail [2022-11-11T18:35:24+0100] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host wn-ca-07.spider.surfsara.nl <========= [2022-11-11T18:35:25+0100] [MainThread] [W] [toil.job] Due to failure we are reducing the remaining try count of job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v5 with ID kind-CWLJob/instance-upr5_lv2 to 2 [2022-11-11T18:35:25+0100] [MainThread] [I] [toil.leader] Issued job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v6 with job batch system ID: 10 and cores: 1, disk: 3.0 Gi, and memory: 2.0 Gi [2022-11-11T18:35:26+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/3dbc01b17d325567bcf725e1375d84eb/3cf1/worker_log.txt [2022-11-11T18:35:26+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/3dbc01b17d325567bcf725e1375d84eb/e4d3/worker_log.txt [2022-11-11T18:35:30+0100] [Thread-9 ] [E] [toil.batchSystems.singleMachine] Got exit code 1 (indicating failure) from job _toil_worker CWLJob file:/tmp/tmpjtdgbk5i kind-CWLJob/instance-upr5_lv2. [2022-11-11T18:35:30+0100] [MainThread] [W] [toil.leader] Job failed with exit value 1: 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v6 Exit reason: None [2022-11-11T18:35:30+0100] [MainThread] [W] [toil.leader] The job seems to have left a log file, indicating failure: 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v8 [2022-11-11T18:35:30+0100] [MainThread] [W] [toil.leader] Log from job "kind-CWLJob/instance-upr5_lv2" follows: =========> [2022-11-11T18:35:26+0100] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG--- [2022-11-11T18:35:26+0100] [MainThread] [I] [toil] Running Toil version 5.7.1-b5cae9634820d76cb6c13b2a6312895122017d54 on host wn-ca-07.spider.surfsara.nl. [2022-11-11T18:35:26+0100] [MainThread] [I] [toil.worker] Working on job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v7 [2022-11-11T18:35:28+0100] [MainThread] [I] [toil.worker] Loaded body Job('CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v7) from description 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v7 [2022-11-11T18:35:28+0100] [MainThread] [W] [cwltool] You have specified `--preserve-entire-environment` while running a container which will override variables set in the container. This may break the container, be non-portable, and/or affect reproducibility. [2022-11-11T18:35:28+0100] [MainThread] [I] [cwltool] Using local copy of Singularity image found in /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow [2022-11-11T18:35:29+0100] [MainThread] [I] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] /tmp/tmp0gjwfh5y$ singularity \ --quiet \ exec \ --contain \ --ipc \ --cleanenv \ --userns \ --home \ /tmp/tmp0gjwfh5y:/lLnyPa \ --bind \ /tmp/tmpx86fhyhb:/tmp \ --bind \ /project/ldv/Data/beamformed/pulp-logs:/var/lib/cwl/stga74ff7ce-d0d0-495f-aade-9c898d294c15/pulp-logs:ro \ --pwd \ /lLnyPa \ --net \ --network \ none \ /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow/git.astron.nl:5000_ldv_bf_double_tgz:latest.sif \ collect_unspecified_metadata \ 81045 \ LOFAR_PULSAR_ARCHIVE_locus093_L81045_CVplots_ea0c6a37.tar \ --rootdir \ /var/lib/cwl/stga74ff7ce-d0d0-495f-aade-9c898d294c15/pulp-logs WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_MEM_PER_CPU as environment variable will not be supported in the future, use APPTAINERENV_SLURM_MEM_PER_CPU instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SRUN_DEBUG as environment variable will not be supported in the future, use APPTAINERENV_SRUN_DEBUG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODEID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODEID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASK_PID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASK_PID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CONNECTION as environment variable will not be supported in the future, use APPTAINERENV_SSH_CONNECTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PRIO_PROCESS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PRIO_PROCESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_VERBOSE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_VERBOSE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV__ as environment variable will not be supported in the future, use APPTAINERENV__ instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LANG as environment variable will not be supported in the future, use APPTAINERENV_LANG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_DIR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTCONTROL as environment variable will not be supported in the future, use APPTAINERENV_HISTCONTROL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HOSTNAME as environment variable will not be supported in the future, use APPTAINERENV_HOSTNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEPID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEPID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_DISTRIBUTION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_DISTRIBUTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_ENVIRONMENT as environment variable will not be supported in the future, use APPTAINERENV_ENVIRONMENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_KDEDIRS as environment variable will not be supported in the future, use APPTAINERENV_KDEDIRS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PROCID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PROCID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_GID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_GID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURMD_NODENAME as environment variable will not be supported in the future, use APPTAINERENV_SLURMD_NODENAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_VIRTUAL_ENV as environment variable will not be supported in the future, use APPTAINERENV_VIRTUAL_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_S_COLORS as environment variable will not be supported in the future, use APPTAINERENV_S_COLORS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_SERVICES_XML as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_SERVICES_XML instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_which_declare as environment variable will not be supported in the future, use APPTAINERENV_which_declare instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_TCP_PORT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_TCP_PORT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_SESSION_ID as environment variable will not be supported in the future, use APPTAINERENV_XDG_SESSION_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_PLUGIN as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_PLUGIN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_USER as environment variable will not be supported in the future, use APPTAINERENV_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NNODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NNODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LAUNCH_NODE_IPADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LAUNCH_NODE_IPADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_ROLE_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_ROLE_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GRID_ENV_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GRID_ENV_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PWD as environment variable will not be supported in the future, use APPTAINERENV_PWD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MYPROXY_SERVER as environment variable will not be supported in the future, use APPTAINERENV_MYPROXY_SERVER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CLUSTER_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CLUSTER_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CLIENT as environment variable will not be supported in the future, use APPTAINERENV_SSH_CLIENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_VERSION as environment variable will not be supported in the future, use APPTAINERENV_LMOD_VERSION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NTASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NTASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_UMASK as environment variable will not be supported in the future, use APPTAINERENV_SLURM_UMASK instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_LEVEL_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_LEVEL_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_ENV as environment variable will not be supported in the future, use APPTAINERENV_BASH_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SPIDER_LOGIN_UI as environment variable will not be supported in the future, use APPTAINERENV_SPIDER_LOGIN_UI instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_WORKING_CLUSTER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_WORKING_CLUSTER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SHELL_PRGM as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SHELL_PRGM instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_TMPDIR as environment variable will not be supported in the future, use APPTAINERENV_TMPDIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_sys as environment variable will not be supported in the future, use APPTAINERENV_LMOD_sys instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_LCG_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOBID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOBID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CONF as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CONF instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODE_ALIASES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODE_ALIASES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_ROOT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_QOS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_QOS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR_PATTERN as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR_PATTERN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GT_PROXY_MODE as environment variable will not be supported in the future, use APPTAINERENV_GT_PROXY_MODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPUS_ON_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPUS_ON_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHELL as environment variable will not be supported in the future, use APPTAINERENV_SHELL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_UID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_UID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_X509_USER_PROXY as environment variable will not be supported in the future, use APPTAINERENV_X509_USER_PROXY instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_PARTITION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_PARTITION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_USE_CURRENT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_USE_CURRENT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_LIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_LIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_USER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NPROCS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NPROCS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHLVL as environment variable will not be supported in the future, use APPTAINERENV_SHLVL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ACCOUNT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ACCOUNT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MANPATH as environment variable will not be supported in the future, use APPTAINERENV_MANPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_CWL_SINGULARITY_CACHE as environment variable will not be supported in the future, use APPTAINERENV_CWL_SINGULARITY_CACHE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_LAUNCHER_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_LAUNCHER_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION_VAR as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION_VAR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_GTIDS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_GTIDS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LOGNAME as environment variable will not be supported in the future, use APPTAINERENV_LOGNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DBUS_SESSION_BUS_ADDRESS as environment variable will not be supported in the future, use APPTAINERENV_DBUS_SESSION_BUS_ADDRESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_RUNTIME_DIR as environment variable will not be supported in the future, use APPTAINERENV_XDG_RUNTIME_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH_ROOT as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PATH as environment variable will not be supported in the future, use APPTAINERENV_SING_USER_DEFINED_PATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_TYPE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_TYPE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DEBUGINFOD_URLS as environment variable will not be supported in the future, use APPTAINERENV_DEBUGINFOD_URLS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_TASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_TASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULESHOME as environment variable will not be supported in the future, use APPTAINERENV_MODULESHOME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SETTARG_FULL_SUPPORT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SETTARG_FULL_SUPPORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTSIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTSIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_PKG as environment variable will not be supported in the future, use APPTAINERENV_LMOD_PKG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTFILESIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTFILESIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_CMD as environment variable will not be supported in the future, use APPTAINERENV_LMOD_CMD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LOCALID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LOCALID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_GFAL_INFOSYS as environment variable will not be supported in the future, use APPTAINERENV_LCG_GFAL_INFOSYS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_OMP_NUM_THREADS as environment variable will not be supported in the future, use APPTAINERENV_OMP_NUM_THREADS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LESSOPEN as environment variable will not be supported in the future, use APPTAINERENV_LESSOPEN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_DIR as environment variable will not be supported in the future, use APPTAINERENV_LMOD_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_which%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_which%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_module%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_module%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_ml%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_ml%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PS1 as environment variable will not be supported in the future, use APPTAINERENV_PS1 instead source: /.inject-apptainer-env.sh:29:11: invalid var name (E) Pipeline for the SASid 81045 has probably failed [2022-11-11T18:35:30+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] exited with status: 1 [2022-11-11T18:35:30+0100] [MainThread] [E] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] Job error: Error validating output record. the `output_json` field is not valid because 'Any' type must be non-null in { "output_json": null } [2022-11-11T18:35:30+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] completed permanentFail Traceback (most recent call last): File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/worker.py", line 407, in workerScript job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2406, in _runner returnValues = self._run(jobGraph=None, fileStore=fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2324, in _run return self.run(fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/cwl/cwltoil.py", line 2217, in run raise cwltool.errors.WorkflowException(status) cwltool.errors.WorkflowException: permanentFail [2022-11-11T18:35:30+0100] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host wn-ca-07.spider.surfsara.nl <========= [2022-11-11T18:35:30+0100] [MainThread] [W] [toil.job] Due to failure we are reducing the remaining try count of job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v8 with ID kind-CWLJob/instance-upr5_lv2 to 1 [2022-11-11T18:35:30+0100] [MainThread] [I] [toil.leader] Issued job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v9 with job batch system ID: 11 and cores: 1, disk: 3.0 Gi, and memory: 2.0 Gi [2022-11-11T18:35:31+0100] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/3dbc01b17d325567bcf725e1375d84eb/086f/worker_log.txt [2022-11-11T18:35:34+0100] [Thread-9 ] [E] [toil.batchSystems.singleMachine] Got exit code 1 (indicating failure) from job _toil_worker CWLJob file:/tmp/tmpjtdgbk5i kind-CWLJob/instance-upr5_lv2. [2022-11-11T18:35:34+0100] [MainThread] [W] [toil.leader] Job failed with exit value 1: 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v9 Exit reason: None [2022-11-11T18:35:34+0100] [MainThread] [W] [toil.leader] The job seems to have left a log file, indicating failure: 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v11 [2022-11-11T18:35:34+0100] [MainThread] [W] [toil.leader] Log from job "kind-CWLJob/instance-upr5_lv2" follows: =========> [2022-11-11T18:35:31+0100] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG--- [2022-11-11T18:35:31+0100] [MainThread] [I] [toil] Running Toil version 5.7.1-b5cae9634820d76cb6c13b2a6312895122017d54 on host wn-ca-07.spider.surfsara.nl. [2022-11-11T18:35:31+0100] [MainThread] [I] [toil.worker] Working on job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v10 [2022-11-11T18:35:32+0100] [MainThread] [I] [toil.worker] Loaded body Job('CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v10) from description 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v10 [2022-11-11T18:35:32+0100] [MainThread] [W] [cwltool] You have specified `--preserve-entire-environment` while running a container which will override variables set in the container. This may break the container, be non-portable, and/or affect reproducibility. [2022-11-11T18:35:32+0100] [MainThread] [I] [cwltool] Using local copy of Singularity image found in /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow [2022-11-11T18:35:33+0100] [MainThread] [I] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] /tmp/tmpvb_1uaio$ singularity \ --quiet \ exec \ --contain \ --ipc \ --cleanenv \ --userns \ --home \ /tmp/tmpvb_1uaio:/AYKXMY \ --bind \ /tmp/tmpyxpnlfkb:/tmp \ --bind \ /project/ldv/Data/beamformed/pulp-logs:/var/lib/cwl/stgd2c2c400-1a7c-4b67-9061-813b0b13e8a5/pulp-logs:ro \ --pwd \ /AYKXMY \ --net \ --network \ none \ /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow/git.astron.nl:5000_ldv_bf_double_tgz:latest.sif \ collect_unspecified_metadata \ 81045 \ LOFAR_PULSAR_ARCHIVE_locus093_L81045_CVplots_ea0c6a37.tar \ --rootdir \ /var/lib/cwl/stgd2c2c400-1a7c-4b67-9061-813b0b13e8a5/pulp-logs WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_MEM_PER_CPU as environment variable will not be supported in the future, use APPTAINERENV_SLURM_MEM_PER_CPU instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SRUN_DEBUG as environment variable will not be supported in the future, use APPTAINERENV_SRUN_DEBUG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODEID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODEID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASK_PID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASK_PID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CONNECTION as environment variable will not be supported in the future, use APPTAINERENV_SSH_CONNECTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PRIO_PROCESS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PRIO_PROCESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_VERBOSE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_VERBOSE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV__ as environment variable will not be supported in the future, use APPTAINERENV__ instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LANG as environment variable will not be supported in the future, use APPTAINERENV_LANG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_DIR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTCONTROL as environment variable will not be supported in the future, use APPTAINERENV_HISTCONTROL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HOSTNAME as environment variable will not be supported in the future, use APPTAINERENV_HOSTNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEPID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEPID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_DISTRIBUTION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_DISTRIBUTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_ENVIRONMENT as environment variable will not be supported in the future, use APPTAINERENV_ENVIRONMENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_KDEDIRS as environment variable will not be supported in the future, use APPTAINERENV_KDEDIRS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PROCID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PROCID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_GID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_GID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURMD_NODENAME as environment variable will not be supported in the future, use APPTAINERENV_SLURMD_NODENAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_VIRTUAL_ENV as environment variable will not be supported in the future, use APPTAINERENV_VIRTUAL_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_S_COLORS as environment variable will not be supported in the future, use APPTAINERENV_S_COLORS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_SERVICES_XML as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_SERVICES_XML instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_which_declare as environment variable will not be supported in the future, use APPTAINERENV_which_declare instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_TCP_PORT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_TCP_PORT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_SESSION_ID as environment variable will not be supported in the future, use APPTAINERENV_XDG_SESSION_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_PLUGIN as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_PLUGIN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_USER as environment variable will not be supported in the future, use APPTAINERENV_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NNODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NNODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LAUNCH_NODE_IPADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LAUNCH_NODE_IPADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_ROLE_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_ROLE_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GRID_ENV_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GRID_ENV_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PWD as environment variable will not be supported in the future, use APPTAINERENV_PWD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MYPROXY_SERVER as environment variable will not be supported in the future, use APPTAINERENV_MYPROXY_SERVER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CLUSTER_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CLUSTER_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CLIENT as environment variable will not be supported in the future, use APPTAINERENV_SSH_CLIENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_VERSION as environment variable will not be supported in the future, use APPTAINERENV_LMOD_VERSION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NTASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NTASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_UMASK as environment variable will not be supported in the future, use APPTAINERENV_SLURM_UMASK instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_LEVEL_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_LEVEL_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_ENV as environment variable will not be supported in the future, use APPTAINERENV_BASH_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SPIDER_LOGIN_UI as environment variable will not be supported in the future, use APPTAINERENV_SPIDER_LOGIN_UI instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_WORKING_CLUSTER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_WORKING_CLUSTER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SHELL_PRGM as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SHELL_PRGM instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_TMPDIR as environment variable will not be supported in the future, use APPTAINERENV_TMPDIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_sys as environment variable will not be supported in the future, use APPTAINERENV_LMOD_sys instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_LCG_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOBID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOBID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CONF as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CONF instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODE_ALIASES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODE_ALIASES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_ROOT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_QOS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_QOS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR_PATTERN as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR_PATTERN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GT_PROXY_MODE as environment variable will not be supported in the future, use APPTAINERENV_GT_PROXY_MODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPUS_ON_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPUS_ON_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHELL as environment variable will not be supported in the future, use APPTAINERENV_SHELL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_UID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_UID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_X509_USER_PROXY as environment variable will not be supported in the future, use APPTAINERENV_X509_USER_PROXY instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_PARTITION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_PARTITION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_USE_CURRENT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_USE_CURRENT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_LIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_LIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_USER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NPROCS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NPROCS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHLVL as environment variable will not be supported in the future, use APPTAINERENV_SHLVL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ACCOUNT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ACCOUNT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MANPATH as environment variable will not be supported in the future, use APPTAINERENV_MANPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_CWL_SINGULARITY_CACHE as environment variable will not be supported in the future, use APPTAINERENV_CWL_SINGULARITY_CACHE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_LAUNCHER_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_LAUNCHER_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION_VAR as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION_VAR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_GTIDS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_GTIDS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LOGNAME as environment variable will not be supported in the future, use APPTAINERENV_LOGNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DBUS_SESSION_BUS_ADDRESS as environment variable will not be supported in the future, use APPTAINERENV_DBUS_SESSION_BUS_ADDRESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_RUNTIME_DIR as environment variable will not be supported in the future, use APPTAINERENV_XDG_RUNTIME_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH_ROOT as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PATH as environment variable will not be supported in the future, use APPTAINERENV_SING_USER_DEFINED_PATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_TYPE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_TYPE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DEBUGINFOD_URLS as environment variable will not be supported in the future, use APPTAINERENV_DEBUGINFOD_URLS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_TASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_TASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULESHOME as environment variable will not be supported in the future, use APPTAINERENV_MODULESHOME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SETTARG_FULL_SUPPORT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SETTARG_FULL_SUPPORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTSIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTSIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_PKG as environment variable will not be supported in the future, use APPTAINERENV_LMOD_PKG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTFILESIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTFILESIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_CMD as environment variable will not be supported in the future, use APPTAINERENV_LMOD_CMD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LOCALID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LOCALID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_GFAL_INFOSYS as environment variable will not be supported in the future, use APPTAINERENV_LCG_GFAL_INFOSYS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_OMP_NUM_THREADS as environment variable will not be supported in the future, use APPTAINERENV_OMP_NUM_THREADS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LESSOPEN as environment variable will not be supported in the future, use APPTAINERENV_LESSOPEN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_DIR as environment variable will not be supported in the future, use APPTAINERENV_LMOD_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_which%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_which%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_module%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_module%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_ml%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_ml%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PS1 as environment variable will not be supported in the future, use APPTAINERENV_PS1 instead source: /.inject-apptainer-env.sh:209:11: invalid var name (E) Pipeline for the SASid 81045 has probably failed [2022-11-11T18:35:34+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] exited with status: 1 [2022-11-11T18:35:34+0100] [MainThread] [E] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] Job error: Error validating output record. the `output_json` field is not valid because 'Any' type must be non-null in { "output_json": null } [2022-11-11T18:35:34+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] completed permanentFail Traceback (most recent call last): File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/worker.py", line 407, in workerScript job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2406, in _runner returnValues = self._run(jobGraph=None, fileStore=fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2324, in _run return self.run(fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/cwl/cwltoil.py", line 2217, in run raise cwltool.errors.WorkflowException(status) cwltool.errors.WorkflowException: permanentFail [2022-11-11T18:35:34+0100] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host wn-ca-07.spider.surfsara.nl <========= [2022-11-11T18:35:34+0100] [MainThread] [W] [toil.job] Due to failure we are reducing the remaining try count of job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v11 with ID kind-CWLJob/instance-upr5_lv2 to 0 [2022-11-11T18:35:34+0100] [MainThread] [W] [toil.leader] Job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v12 is completely failed [2022-11-11T18:35:52+0100] [MainThread] [I] [toil.leader] Finished toil run with 8 failed jobs. [2022-11-11T18:35:52+0100] [MainThread] [I] [toil.leader] Failed jobs at end of the run: 'CWLScatter' kind-CWLScatter/instance-b2qvthu_ v2 'CWLGather' kind-CWLGather/instance-jt662rjc v3 'CWLWorkflow' kind-CWLWorkflow/instance-pw_l7cs1 v2 'CWLWorkflow' kind-CWLWorkflow/instance-qcz3spu5 v2 'CWLScatter' kind-CWLScatter/instance-nqr59ox9 v3 'CWLJob' bf_remove_double_tgz.cwl.eliminate_double_tgz.double_tgz_elimination.cwl kind-CWLJob/instance-djtbvqku v2 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v12 'CWLJob' bf_remove_double_tgz.cwl.compute_md5sum.computemd5.cwl kind-CWLJob/instance-qan19d1t v3 [2022-11-11T18:35:52+0100] [MainThread] [I] [toil.common] Successfully deleted the job store: FileJobStore(/tmp/tmpjtdgbk5i) Traceback (most recent call last): File "/project/ldv/Software/executor-venv/bin/toil-cwl-runner", line 8, in sys.exit(main()) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/cwl/cwltoil.py", line 3654, in main outobj = toil.start(wf1) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/common.py", line 966, in start return self._runMainLoop(rootJobDescription) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/common.py", line 1386, in _runMainLoop return Leader(config=self.config, File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/leader.py", line 309, in run raise FailedJobsException(self.jobStore, failed_jobs, exit_code=self.recommended_fail_exit_code) toil.leader.FailedJobsException: The job store '/tmp/tmpjtdgbk5i' contains 8 failed jobs: 'CWLScatter' kind-CWLScatter/instance-b2qvthu_ v2, 'CWLGather' kind-CWLGather/instance-jt662rjc v3, 'CWLWorkflow' kind-CWLWorkflow/instance-pw_l7cs1 v2, 'CWLWorkflow' kind-CWLWorkflow/instance-qcz3spu5 v2, 'CWLScatter' kind-CWLScatter/instance-nqr59ox9 v3, 'CWLJob' bf_remove_double_tgz.cwl.eliminate_double_tgz.double_tgz_elimination.cwl kind-CWLJob/instance-djtbvqku v2, 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v12, 'CWLJob' bf_remove_double_tgz.cwl.compute_md5sum.computemd5.cwl kind-CWLJob/instance-qan19d1t v3 Log from job "'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v12" follows: =========> [2022-11-11T18:35:31+0100] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG--- [2022-11-11T18:35:31+0100] [MainThread] [I] [toil] Running Toil version 5.7.1-b5cae9634820d76cb6c13b2a6312895122017d54 on host wn-ca-07.spider.surfsara.nl. [2022-11-11T18:35:31+0100] [MainThread] [I] [toil.worker] Working on job 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v10 [2022-11-11T18:35:32+0100] [MainThread] [I] [toil.worker] Loaded body Job('CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v10) from description 'CWLJob' bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl kind-CWLJob/instance-upr5_lv2 v10 [2022-11-11T18:35:32+0100] [MainThread] [W] [cwltool] You have specified `--preserve-entire-environment` while running a container which will override variables set in the container. This may break the container, be non-portable, and/or affect reproducibility. [2022-11-11T18:35:32+0100] [MainThread] [I] [cwltool] Using local copy of Singularity image found in /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow [2022-11-11T18:35:33+0100] [MainThread] [I] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] /tmp/tmpvb_1uaio$ singularity \ --quiet \ exec \ --contain \ --ipc \ --cleanenv \ --userns \ --home \ /tmp/tmpvb_1uaio:/AYKXMY \ --bind \ /tmp/tmpyxpnlfkb:/tmp \ --bind \ /project/ldv/Data/beamformed/pulp-logs:/var/lib/cwl/stgd2c2c400-1a7c-4b67-9061-813b0b13e8a5/pulp-logs:ro \ --pwd \ /AYKXMY \ --net \ --network \ none \ /project/ldv/Share/run/workflows/bf_remove_double_tgz/workflow/git.astron.nl:5000_ldv_bf_double_tgz:latest.sif \ collect_unspecified_metadata \ 81045 \ LOFAR_PULSAR_ARCHIVE_locus093_L81045_CVplots_ea0c6a37.tar \ --rootdir \ /var/lib/cwl/stgd2c2c400-1a7c-4b67-9061-813b0b13e8a5/pulp-logs WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_MEM_PER_CPU as environment variable will not be supported in the future, use APPTAINERENV_SLURM_MEM_PER_CPU instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SRUN_DEBUG as environment variable will not be supported in the future, use APPTAINERENV_SRUN_DEBUG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODEID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODEID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASK_PID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASK_PID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CONNECTION as environment variable will not be supported in the future, use APPTAINERENV_SSH_CONNECTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PRIO_PROCESS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PRIO_PROCESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_VERBOSE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_VERBOSE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV__ as environment variable will not be supported in the future, use APPTAINERENV__ instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LANG as environment variable will not be supported in the future, use APPTAINERENV_LANG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_DIR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTCONTROL as environment variable will not be supported in the future, use APPTAINERENV_HISTCONTROL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HOSTNAME as environment variable will not be supported in the future, use APPTAINERENV_HOSTNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEPID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEPID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_DISTRIBUTION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_DISTRIBUTION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_ENVIRONMENT as environment variable will not be supported in the future, use APPTAINERENV_ENVIRONMENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_KDEDIRS as environment variable will not be supported in the future, use APPTAINERENV_KDEDIRS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_PROCID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_PROCID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_GID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_GID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURMD_NODENAME as environment variable will not be supported in the future, use APPTAINERENV_SLURMD_NODENAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_VIRTUAL_ENV as environment variable will not be supported in the future, use APPTAINERENV_VIRTUAL_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_S_COLORS as environment variable will not be supported in the future, use APPTAINERENV_S_COLORS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_SERVICES_XML as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_SERVICES_XML instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_which_declare as environment variable will not be supported in the future, use APPTAINERENV_which_declare instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_TCP_PORT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_TCP_PORT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_SESSION_ID as environment variable will not be supported in the future, use APPTAINERENV_XDG_SESSION_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_SD_PLUGIN as environment variable will not be supported in the future, use APPTAINERENV_GLITE_SD_PLUGIN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_USER as environment variable will not be supported in the future, use APPTAINERENV_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NNODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NNODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LAUNCH_NODE_IPADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LAUNCH_NODE_IPADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_TASKS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_TASKS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_ROLE_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_ROLE_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GRID_ENV_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GRID_ENV_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PWD as environment variable will not be supported in the future, use APPTAINERENV_PWD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MYPROXY_SERVER as environment variable will not be supported in the future, use APPTAINERENV_MYPROXY_SERVER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CLUSTER_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CLUSTER_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SSH_CLIENT as environment variable will not be supported in the future, use APPTAINERENV_SSH_CLIENT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_VERSION as environment variable will not be supported in the future, use APPTAINERENV_LMOD_VERSION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NTASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NTASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_UMASK as environment variable will not be supported in the future, use APPTAINERENV_SLURM_UMASK instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_LEVEL_REQUESTED as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_LEVEL_REQUESTED instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_ENV as environment variable will not be supported in the future, use APPTAINERENV_BASH_ENV instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SPIDER_LOGIN_UI as environment variable will not be supported in the future, use APPTAINERENV_SPIDER_LOGIN_UI instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_WORKING_CLUSTER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_WORKING_CLUSTER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SHELL_PRGM as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SHELL_PRGM instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NODELIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NODELIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NAME as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SRUN_COMM_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SRUN_COMM_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_TMPDIR as environment variable will not be supported in the future, use APPTAINERENV_TMPDIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_sys as environment variable will not be supported in the future, use APPTAINERENV_LMOD_sys instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_LCG_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOBID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOBID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CONF as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CONF instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NODE_ALIASES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NODE_ALIASES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_ROOT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_QOS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_QOS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_TOPOLOGY_ADDR_PATTERN as environment variable will not be supported in the future, use APPTAINERENV_SLURM_TOPOLOGY_ADDR_PATTERN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GT_PROXY_MODE as environment variable will not be supported in the future, use APPTAINERENV_GT_PROXY_MODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPUS_ON_NODE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPUS_ON_NODE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHELL as environment variable will not be supported in the future, use APPTAINERENV_SHELL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_UID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_UID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_X509_USER_PROXY as environment variable will not be supported in the future, use APPTAINERENV_X509_USER_PROXY instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_PARTITION as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_PARTITION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SELINUX_USE_CURRENT_RANGE as environment variable will not be supported in the future, use APPTAINERENV_SELINUX_USE_CURRENT_RANGE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_LIST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_LIST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL as environment variable will not be supported in the future, use APPTAINERENV_GLOBUS_GSSAPI_MAX_TLS_PROTOCOL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_USER as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_USER instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_NPROCS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_NPROCS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SHLVL as environment variable will not be supported in the future, use APPTAINERENV_SHLVL instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_SUBMIT_HOST as environment variable will not be supported in the future, use APPTAINERENV_SLURM_SUBMIT_HOST instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ACCOUNT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ACCOUNT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MANPATH as environment variable will not be supported in the future, use APPTAINERENV_MANPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_CWL_SINGULARITY_CACHE as environment variable will not be supported in the future, use APPTAINERENV_CWL_SINGULARITY_CACHE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_LAUNCHER_PORT as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_LAUNCHER_PORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION_VAR as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION_VAR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_GTIDS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_GTIDS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LOGNAME as environment variable will not be supported in the future, use APPTAINERENV_LOGNAME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DBUS_SESSION_BUS_ADDRESS as environment variable will not be supported in the future, use APPTAINERENV_DBUS_SESSION_BUS_ADDRESS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_XDG_RUNTIME_DIR as environment variable will not be supported in the future, use APPTAINERENV_XDG_RUNTIME_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_GLITE_LOCATION as environment variable will not be supported in the future, use APPTAINERENV_GLITE_LOCATION instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULEPATH_ROOT as environment variable will not be supported in the future, use APPTAINERENV_MODULEPATH_ROOT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PATH as environment variable will not be supported in the future, use APPTAINERENV_SING_USER_DEFINED_PATH instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_ID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_ID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_CPU_BIND_TYPE as environment variable will not be supported in the future, use APPTAINERENV_SLURM_CPU_BIND_TYPE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_DEBUGINFOD_URLS as environment variable will not be supported in the future, use APPTAINERENV_DEBUGINFOD_URLS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_TASKS as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_TASKS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_MODULESHOME as environment variable will not be supported in the future, use APPTAINERENV_MODULESHOME instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_SETTARG_FULL_SUPPORT as environment variable will not be supported in the future, use APPTAINERENV_LMOD_SETTARG_FULL_SUPPORT instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTSIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTSIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 as environment variable will not be supported in the future, use APPTAINERENV_SLURM_JOB_CPUS_PER_NODE_PACK_GROUP_0 instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_PKG as environment variable will not be supported in the future, use APPTAINERENV_LMOD_PKG instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_HISTFILESIZE as environment variable will not be supported in the future, use APPTAINERENV_HISTFILESIZE instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_STEP_NUM_NODES as environment variable will not be supported in the future, use APPTAINERENV_SLURM_STEP_NUM_NODES instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_CMD as environment variable will not be supported in the future, use APPTAINERENV_LMOD_CMD instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_SLURM_LOCALID as environment variable will not be supported in the future, use APPTAINERENV_SLURM_LOCALID instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LCG_GFAL_INFOSYS as environment variable will not be supported in the future, use APPTAINERENV_LCG_GFAL_INFOSYS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_OMP_NUM_THREADS as environment variable will not be supported in the future, use APPTAINERENV_OMP_NUM_THREADS instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LESSOPEN as environment variable will not be supported in the future, use APPTAINERENV_LESSOPEN instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_LMOD_DIR as environment variable will not be supported in the future, use APPTAINERENV_LMOD_DIR instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_which%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_which%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_module%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_module%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_BASH_FUNC_ml%% as environment variable will not be supported in the future, use APPTAINERENV_BASH_FUNC_ml%% instead WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_PS1 as environment variable will not be supported in the future, use APPTAINERENV_PS1 instead source: /.inject-apptainer-env.sh:209:11: invalid var name (E) Pipeline for the SASid 81045 has probably failed [2022-11-11T18:35:34+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] exited with status: 1 [2022-11-11T18:35:34+0100] [MainThread] [E] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] Job error: Error validating output record. the `output_json` field is not valid because 'Any' type must be non-null in { "output_json": null } [2022-11-11T18:35:34+0100] [MainThread] [W] [cwltool] [job bf_remove_double_tgz.cwl.extract_metadata.extract_metadata.cwl] completed permanentFail Traceback (most recent call last): File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/worker.py", line 407, in workerScript job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2406, in _runner returnValues = self._run(jobGraph=None, fileStore=fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/job.py", line 2324, in _run return self.run(fileStore) File "/project/ldv/Software/executor-venv/lib/python3.9/site-packages/toil/cwl/cwltoil.py", line 2217, in run raise cwltool.errors.WorkflowException(status) cwltool.errors.WorkflowException: permanentFail [2022-11-11T18:35:34+0100] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host wn-ca-07.spider.surfsara.nl <========= srun: error: wn-ca-07: task 0: Exited with exit code 1