How to Setup CAF at clued0 and Skim the Data

Follow the wiki page setup (clued0 is a tcshell): ( https://plone4.fnal.gov/P1/D0Wiki/caf/caffaq/CompleteStarter)

setup D0RunII p18.03.00
setup d0tools
setup d0cvs
setenv LINK_SHARED yes
newrel -t p18.03.00 work_new
cd work_new
d0setwa
set path=($SRT_PRIVATE_CONTEXT/shbin/$SRT_SUBDIR $path )
gmake
rehash

The work_new directory is at:

/rooms/safe/brazil/mercadan/work_new

an area that Greg gave us with 2 Tera of disk space:

lafex-clued0:/rooms/safe/brazil> df .
Filesystem 1K-blocks Used Available Use% Mounted on
calvus-clued0:/raid2_1/work
2113784984 89057936 2024727048 5% /rooms/safe

In this directory I have run cafe with the following cafe.config to skim the data:

lafex-clued0:/rooms/safe/brazil/mercadan/work> more cafe_101_1.config
cafe.Run: EventFlags(myFlags) Write(MyFile)
cafe.Input: listfile:emhightpt_101_1.lst
MyFile.File: electrons_101_1.root
myFlags.All: HAS_2EM_15
#MySelection.Select: EMscone[0].Pt() > 20. && EMscone[1].Pt() > 20.
MyFile.Disable: TrackCal TrackCalJet_JCCA TrackCalJet_JCCB
cafe.Progress: 1000
#cafe.Events: 4000

I have several cafe_***_.config with the respective file list. In this way each job has only 20 input files that gives 2 gb of output *.root. The complete list of all input files are in

emhightpt_tot.lista (916 files).

However this list has some files that were processed two times and we need to keep only one. The cafe_*_*.lst contains only one copy of each file (895 files).

The list of skimmed files was found in:

https://plone4.fnal.gov/P1/D0Wiki/caf/cafproduction-v1/output
2EMhighpt: /prj_root/1403/caf_write/CAF-2EMhighpt-p18.01.00/

To run in clud0 batch system we need to:

setup caf_tools
runcafe -clued0 -cput=5:00:00 -mem=500mb -- cafe_12_1.config

To see the queue:

qstat -n -u mercadan

The output are is in:

cafe_200511*/

There are about 40 different list of files, each with an output of 2Gb. To transfer it via scp to D0server is a long job.