Ganga tutorial

(The Gaudi/Athena and Grid Alliance)

Tuomas Hapola

IPPP, Durham University

users.hepforge.org/~hapola/reveal.js-2.6.1/ganga.html

Introduction
Installation
Proxy certificate
'Hello World'
How to split jobs
Useful commands

The Grid

"The Grid" is a collection of resources

Computing Elements (CE)
Storage Elements (SE)

We are using the LHC Computing Grid (LCG)

Access to Resources

The grid Certificate used to identify the user
A Virtual Organisation (VO) membership allows user to access resources available to VO
Communication with the Grid Security Infrastructure via proxy certificate

Proxy certificate has a finite lifetime
Created from the grid certificate

What is Ganga

A front-end for the configuration, execution and management of computational tasks
A user-centric layer above existing grid middleware
Implemented in python

simple to use
easy to extend

Provides an IPython shell

Certificate

Apply a certificate from UK e-Science Certification Authority
Apply a VO membership http://www.phenogrid.dur.ac.uk/
Export your certificate from Certificate Wizard (.p12)
Go to Grid UI
Create ~/.globus directory and convert the certificate


	      $ mkdir -p .globus
$ openssl pkcs12 -nocerts -in gridcert.p12 -out .globus/userkey.pem
$ openssl pkcs12 -clcerts -nokeys -in gridcert.p12 -out .globus/usercert.pem
$ chmod 444 .globus/usercert.pem
$ chmod 400 .globus/userkey.pem

Installation

http://ganga.web.cern.ch/ganga/
Download an installation script
Execute the script


	       $ wget http://ganga.web.cern.ch/ganga/download/ganga-install
	       $ python ganga-install

Proxy


		$ voms-proxy-init --voms pheno

Valid for 12 hours
Instructions for long term proxy http://egee-uig.web.cern.ch/egee-uig/production_pages/ProxyRenewal.html

Fire up Ganga shell


		$ Ganga/install/6.0.23/bin/ganga

Asks to create a default .gangarc file

used to modify default settings

'Hello World' (local)


		j0 = Job()
		j0.application = Executable(exe=File('/bin/echo'), args=['HelloWorld'])
		j0.submit()

		jobs          #list jobs

'Hello World' (Grid)


		j1 = Job()
		j1.application = Executable(exe=File('/bin/echo'), args=['HelloWorld'])
		j1.backend=LCG()
		j1.submit()

LCG requirements

One can set requirements for CEs:

allowedCEs
excludedCEs
cputime
walltime
memory


		j2 = Job()
		j2.application = Executable(exe=File('/bin/echo'), args=['HelloWorld'])
		j2.backend=LCG()
		j2.backend.requirements.allowedCEs="\.dur\.ac\.uk"
		j2.submit()

Example with a user program

For this example we have 3 files

g++ writeargv.cxx -> a.out
gangaExample.py
submit.py

We use splitter to submit multiple jobs

Argument splitter example

A simple c++ program


	      #include <fstream>
	      #include <iostream> 
	      using namespace std;
		  
              int main(int argc, char* argv[]) { 
	       cout << "argc = " << argc << endl;

	       ofstream a_file ("output.txt");

	       for(int i = 0; i < argc; i++) 
                 a_file << "argv[" << i << "] = " << argv[i] << endl;

	       a_file.close();
               return 0; 
             }

Argument splitter example

python driver script: gangaExample.py


	      #!/usr/bin/env python
	      """Example Ganga job"""
	      __author__ = "Tuomas Hapola (t.a.hapola@durham.ac.uk)"
	      __date__ = "9 April 2014"


	      import sys
	      import os

	      cmd = "./a.out "+sys.argv[1]+" "+sys.argv[2]
	      os.system(cmd)

Argument splitter example

Submission script: submit.py


	      #!/usr/bin/env ganga

#Define arguments for the splitter
arglist = [[1,'fisrt'],[2,'second'],[3,'third']]
argSplit = ArgSplitter(args = arglist)

# Create the job and "split" it
j0 = Job()
j0.application = Executable(exe=File('~/tmp/ganga-tutorial/gangaExample.py'))
j0.backend=LCG()
j0.splitter=argSplit
     
#Define input and output sandboxes 
j0.inputsandbox = [File('~/tmp/ganga-tutorial/a.out')]
j0.outputsandbox = ['*.txt',]

#Submit
j0.submit()

Argument splitter example

Submission using the ganga IPython prompt


	     execfile('./tmp/ganga-tutorial/submit.py')

or using terminal


	     $ Ganga/install/6.0.26/bin/ganga tmp/ganga-tutorial/submit.py

Useful commands

List all your jobs


	     jobs

List all the subjobs


	     jobs(1).subjobs

Print out all the information of a subjob


	     jobs(1).subjobs(0)

Or specific information


	     jobs(1).subjobs(0).backend.actualCE
	     jobs(1).subjobs(0).backend.reason

Useful commands

List output sandbox content


	     jobs(1).subjobs(0).peek()

Open stdout.gz (or stderr.gz) using less or emacs


	     jobs(1).subjobs(0).peek('stdout.gz')
	     jobs(1).subjobs(0).peek('stdout.gz','emacs')

Physically output sandbox files are in


	     /gangadir/workspace/(username)/LocalXML/(jobid)/(subjobid)/output/

Force status failed


	    jobs(1).subjobs(0).force_status('failed', force=True)

Useful commands

Killing jobs


	     jobs(1).subjobs(0).kill()

removing jobs


	     jobs(1).subjobs(0).remove()
	     jobs(1).remove()

Resubmit failed jobs

Submit to any allowed CEs


	     j=jobs(1)
	     failjobs= [js for js in j.subjobs if js.status=="failed"]

	     for js in failjobs:
	         js.resubmit()

or with one line


	      jobs(1).subjobs.select(status='failed').resubmit()

Large files

Sandboxes only for small files -> use Grid storage

Environmental variables


	      export LFC_HOST=lfc.grid.sara.nl
	      export LCG_CATALOG_TYPE=lfc
	      export LFC_HOME=/grid/pheno


	      lfc-mkdir myUsername/output
	      lcg-cr --vo pheno -l lfn:myUsername/output/output.txt  file:$PWD/output.txt

Copy file from the grid storage to local machine


	     $ lcg-cp lfn:myUsername/output/output.txt output.txt

Large files

Input and object files can be send to grid storage and copied to worker node


	    $  lfc-mkdir myUsername/input
	    $  lcg-cr --vo pheno -l lfn:myUsername/input/a.out  file:$PWD/a.out

Include lines to the script to copy files from storage


	     lcg-cp lfn:myUsername/input/a.out a.out

Example

modified gangaExample.py


	      #!/usr/bin/env python

import sys
import os
os.environ["LFC_HOST"]="lfc.grid.sara.nl"
os.environ["LCG_CATALOG_TYPE"]="lfc"
os.environ["LFC_HOME"]="/grid/pheno"

os.system('lcg-cp lfn:myUsername/input/a.out a.out')
os.system('chmod +x a.out')

cmd = "./a.out "+sys.argv[1]+" "+sys.argv[2]
os.system(cmd)

os.system('lfc-mkdir hapola/output')
os.system('lcg-cr --vo pheno -l lfn:hapola/output/output'+sys.argv[1]+'.txt
                file:$PWD/output.txt')

Example

Modified submission script


	      #!/usr/bin/env ganga

#Define arguments for the splitter
arglist = [[1,'fisrt'],[2,'second'],[3,'third']]
argSplit = ArgSplitter(args = arglist)

# Create the job and "split" it
j0 = Job()
j0.application = Executable(exe=File('~/tmp/ganga-tutorial/gangaExample.py'))
j0.backend=LCG()
j0.splitter=argSplit
     
#Define input and output sandboxes 
#j0.inputsandbox = [File('~/tmp/ganga-tutorial/a.out')]
#j0.outputsandbox = ['*.txt',]

#Submit
j0.submit()

More Useful commands

List all the resources


	     $ lcg-infosites --vo pheno all

List SEs


	     $ lcg-infosites --vo pheno se

List directory content (Grid storage)


	     $ lfc-ls /grid/pheno/myUsername

Make replica to another SE


	     $ lcg-rep -v --vo pheno lfn:myUsername/a.out -d se2.ppgrid1.rhul.ac.uk

More Useful commands

Delete replicas of a file


	     $ lcg-del -a lfn:myUsername/a.out

Remove an empty(!) direcroty


	     $ lfc-rm -r /grid/pheno/myUsername

Conclusions

Ganga makes job management easy
Easy to extend if more functionalities are needed

Ganga tutorial

(The Gaudi/Athena and Grid Alliance)

Contents

The Grid

Access to Resources

What is Ganga

Certificate

Installation

Proxy

Fire up Ganga shell

'Hello World' (local)

'Hello World' (Grid)

LCG requirements

Example with a user program

Argument splitter example

Argument splitter example

Argument splitter example

Argument splitter example

Useful commands

Useful commands

Useful commands

Resubmit failed jobs

Large files

Large files

Example

Example

More Useful commands

More Useful commands

Conclusions