SUSY Trigger Validation Tutorial

General Idea

The SUSY trigger validation is based on FullSim RelVal samples of signal in the LM1 point (LM1 corresponds to a certain set of values for the 5 free parameters for mSUGRA SUSY). The idea is to evaluate the global efficiency (number of events selected by the trigger over number of total signal events generated) of each trigger path in a given trigger menu. The comparison of one release is done, path by path, with respect to the efficiency of the previous validated release.

Technical Details

The name of the sample is: RelValLM1_sfts

The trigger paths of a given menu, implemented on the CMSSW release, are analyzed using the codes on CVS:

http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/CMSSW/HLTriggerOffline/SUSYBSM/

Specifically, the pieces with greater interest are the following:

The piece of .cc that makes all the relevant histograms:

http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/CMSSW/HLTriggerOffline/SUSYBSM/src/HltSusyExoPostProcessor.cc?hideattic=1&view=log

The piece of .py which is run over the proper GEN-SIM-RECO RelVal dataset:

http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/CMSSW/HLTriggerOffline/SUSYBSM/python/SUSYBSM_postProcessor_cff.py?hideattic=1&view=log

The code can be run locally for testing and developing purposes by checking out the proper package: cvs co HLTriggerOffline/SUSYBSM

The dataset in to be called in the .py file is nowadays (March/2013) located at T1 Fermilab.

For each CMSSW release, this RelVal sample is generated and the trigger analysis code is run centrally. The plots are them acessible on the DQM Offline system at the CMSWeb platform. Links and details on how to access that will be shown in the step-by-step of a validation below.

Validation step-by-step

1. Check the available and the previous validated releases

This can be done by checking the ValDB website:

https://cms-conddb-dev.cern.ch/PdmV/valdb/

That is how the website looks like (March/2013):

To check a list of related releases, write part of the name in the "Select one release" box. For instance, for releases of the family 6_1_X, write 6_1. The website will then warn you how many releases match this search criteria:

The releases to be shown will then appear in the "List of releases" item. To select all, just click on "Show all". All selected releases will then be shown in the list in the right column of the website:

The status of each of the SUSY trigger validation can be seen if HLT -> FullSim is selected in the gray menu in the bottom right of the page. The legend shows what each symbol means. This is where we can see which of the CMSSW releases still need validation (in this screenshot example, 6_1_1), and which is the last previously validated (with OK status) release to which we will compare the release to be validated with (in this screenshot example, 6_1_0).

2. Check plots on DQM GUI

Go to the CMSWEB website (note you need your certificate installed in the browser):

https://cmsweb.cern.ch/

Click on DQM RelVal link. The following page will appear:

Click on the link on Data, in the last line in the top black banner menu. It will look like:

In the light blue banner, under the black banner, type: "Search": LM1 and change "Vary By" to "Any" item.

A list of the CMSSW releases which were produced with the RelVal SUSY LM1 samples will appear. Click on the releases in which you want to get details from. In this example, I am going to click on 6_1_1, which is the release I want to validate, and on 6_1_0, which is the reference release. It will look like:

One thing that can be useful: select and copy in your clipboard the name of the reference release dataset. In the example here, my clipboard will have the following content: /RelValLM1_sfts/CMSSW_6_1_0-START61_V8-v1/DQM . Then click on the link corresponding to the dataset which you want to validate now (in my case, I will click on the /RelValLM1_sfts/CMSSW_6_1_1-START61_V11-v1/DQM ) and the following page will appear:

Now, in the top menu, second column, under the item "Workspace", click on "Summary". A menu with other options will open:

Then, click on "Shift" in the first column. A new page will appear.

Click on the "HLT" folder. Then, on the "SusyExo" folder. Then on the "00 No Selection" folder. Two folders which will be analyzed separately will appear: "L1" and "HLT", as the picture below:

First, click on "L1". The first set of histograms (in the DQM a histogram is called a "Monitor Element"), corresponding to the global efficiencies of each of the L1 triggers, separated by categories, will appear.

You can change some parameters for better visualization: for instance, change the size from "Medium" to "Fill". Now, to compare to the previous validated release, click on "View Details", located in the top of the page, left side, besides the name in which you are logged in. A new window will appear. In the third section, "Reference", change "Show reference" to the parameter "For all". Leave "Position" in "Overlay", and "Stats" and "ErrBars" in "True". In the numberes list, in the numebr 1, leave "Run" empty, and in "Dataset", paste that reference path we have put in the clipboard before. In this example,/RelValLM1_sfts/CMSSW_6_1_0-START61_V8-v1/DQM. The window after all these settings will look like:

To exit this popup window, just click again in "View details". The refence histogram, in blue, will be overaid to the points corresponding to the release we want to validate. Now, we can compare if the black points are above or compatible (inside 2%) the blue lines. The Y axis is percentage/100. An example can be seen below. Notice that most paths in this category are compatible with the reference blue histogram, and that some paths have higher efficiency than the blue histogram. This mean this L1 paths are fine, and validated. Do this for every plot, and report if any drop in efficiency bigger than 2% is observed.

To get a link of this page and settings, to be pasted in the validation report to be written, click on the "Link Me" item in the right of the top light blue bar. A link that can be pasted anywhere appears in a window, as in the figure below:

Afterwards, click again in 00 No Selection, in the top light blue bar. This will take you back to the two folders, "L1" and "HLT".

Now, instead of clicking in "L1", click on "HLT". Now, the comparison and the formatting is ready to look in the HLT trigger paths, because all the setup was done previously for "L1". However, note that the validation is hard to be done by eye, because it is hard to read the name of some trigger paths, the plot is overpopulated. An example can be seen in the next print. Click on "Link Me" to get the link to these plots to paste in your validation summary later.

To make this comparison without needing to rely in the eye, a root macro was written by Flavia Dias. To use it, follow the steps in the next section.

Using ROOT macro to validate HLT paths

The first thing you are going to need is to download the ROOT files with all the histograms which appear on DQM. This can be done in the following link:

https://cmsweb.cern.ch/dqm/relval/data/browse/ROOT/

Under the section RelVal, and inside the folder of the given release family. Note that the file names you need to download will have "LM1" in the name, as well as the CMSSW release corresponding to it.

With the root files downloaded locally (I am going to call them "old_release.root" and "new_release.root"), you need to download the script, which is called TriggerScript.C and is also attached in this Twiki:

https://web.sprace.org.br/twiki/pub/Main/TriggerValidationTutorial/TriggerScript.C

Now, open a root section. Load the script:

.L TriggerScript.C

Now, to run the script, use:

TriggerScript("old_release.root", "new_release.root")

The result will have the following format:

For each of the classifications of the trigger plots, the script compares the name of the trigger paths of both releases (ignoring the version, such as _v1), and when the paths are the same, it compares the efficiency of the old and new release. It then compares the number of paths in each menu in that given category, how many of those paths are in common, and, if some path had a decrease bigger than 2%, it prints which path, and how much was the decrease. This HAS TO BE PASTED in the validation report. An example of a print reporting a problem is shown below:

Now, all the results have to be reported. This is done once more in the ValDB website.

ValDB report

Once more, enter the ValDB website. Click on the square corresponding to the CMSSW release to be validated, in the column of "Susy". A new window will appear:

Click on "Edit Information". Change the "SUSY Validation Status" to the proper value. If nothing is wrong to be reported, "OK". If something is wrong and needs attention, "Failure", and explain why next. In the field "Comments", you say if the trigger menu is the same or if the number of triggers changed, and if it did, how it changed (this is part of the output from the trigger script). It's also where you report trigger paths which decreased. In the field "Links", you paste the two links from the L1 and HLT DQM, which you have obtained before. For the 6_1_1 agains 6_1_0, this window look like:

You click on "Save Information" and an automated email will warn the appropriate mailing lists that this relase has been validated, and what is the result of this validation.

You SUSY trigger validation is now done. Move to the next not validated release, and repeat all the procedure.

-- FlaviaDias - 2013-03-16