Matteo Risoldi - Files & Docs

The AnalysisDistributor program

This program is in perpetuous execution on the condor central machine (pcepcmt19). What it does is checking the analysiscache table to see if any new run has been inserted and not yet analyzed (i.e. started = null).

If this is the case, it creates a condor command file in /home/control/AnalysisFiles/commandfiles, named according to this scheme:

Analysis<run number><date and time>.condor

It will contain all the necessary information for the condor process to be spawned. It then spawns the process.

Important note: condor is very painful about paths and such. To make this work, a symbolic link has to be provided to the AnalysisEngine.class file. This link must be located in the /home/control/AnalysisFiles directory, MUST be named AnalysisEngine.class (like the real file), and must point to the AnalysisEngine.class file in the CVS repository, under Control/Analysis/AnalysisDistributor. Feel free to find a better solution to this, I could not do it in the time I had.

Spawning the process will also create 3 files for that process, under the outputfiles, errorfiles, and logfiles directories located in /home/control/AnalysisFiles. These are named in the following way:

Analysis<run number><date and time>.<run number>[ .error | .log ]

And will contain, respectively, the output, errors and log data of the condor process. In principle, the output file could be used to store the analysis results (although this is not mandatory). A choice will have to be made when the analysis engine will be ready.

If you want to check whether the condor processes are being spawned, run the condor_q command from the central machine (pcepcmt19).

To do:

This program should, in principle, also take care of understanding if one of the processes sent to analysis has hung up, and in that case restart it (or better: set the started field to null for that run, so at the next round it will be rescheduled). It should also do the cleaning up before rescheduling.

For completed runs, it should probably set the completed flag in analysiscache (I still have to understand if this flag has any real uses).

Also, a cleaning up of the analysis cache could be provided at predefined intervals (remove all runs whose analysis is over) to avoid unnecessary growth of the table.