Grid

From Support-Applicatif

Jump to: navigation, search

Contents

Useful links

>>> LAPP user support

Your message will be managed by the LAPP computing departement helpdesk (Request Tracker). Please avoid contacting directly a member of IT department team and use the following generic addresses.

LAPP and LAPTh members: LAPP-LAPTh IT support
All other users: MUST user support

>>> LAPP IT Department documentation

Some documentation can also be found inside LAPP IT department documentation

>>> LAPP cluster jobs monitoring

You can check the load of the system on the MUST monitoring Web page

>>> LAPP cluster storage monitoring

Only reachable when connected to LAPP network (or via VPN) :LAPP cluster monitoring

>>> Overview of all grid sites

An overview of all grid sites is available here : http://goc.grid.sinica.edu.tw/gstat/

You will find all information you need about all EGEE sites : error/downtime, installed software releases, OS, ...
LAPP site is here: http://goc.grid.sinica.edu.tw/gstat/IN2P3-LAPP/

>>> EGI web site

The official EGI web site is here: http://www.egi.eu/

>>> NEW : Grid for Newbies

A very useful document if you don't know anything about the Grid: https://forge.in2p3.fr/attachments/download/13593/La_Grille_pour_les_Nuls_v1.0.pdf

>>> NEW : MOOC on Grid Computing

A Massive Open Online Course (MOOC) on Grid Computing is available here : http://doc.grid.surfsara.nl/en/latest/Pages/Tutorials/MOOC/mooc.html#mooc

>>> Operations Portal

Any information about VO can be found here: http://operations-portal.egi.eu/

Grid / EGI user's guide

This guide is an introduction to the WLCG/EGI Grid and to the gLite 3 middleware from a user's point of view: https://edms.cern.ch/ui/file/722398/1.4/gLite-3-UserGuide.pdf

Grid certificate / VO

This page is a brief summary of the grid quick start guide mentioned above

Your first need to get a CNRS certificate @ http://igc.services.cnrs.fr/GRID2-FR.

You'll receive a mail of confirmations and you have to extract it using the same browser you used asking for the certificate, and be careful choosing your passwords !!!

To check if you already have a valid CNRS certificate :

First method  : open the certificate section of your web browser and check the validity date of your certificate.

If you use Chrome [[1]], either Mozilla [[2]], or Internet Explorer [[3]].

Second method : go to the following link https://igc.services.cnrs.fr/GRID2-FR/certificats.html and click on "Tester votre certificat" in the left panel.

CAUTION : the CNRS grid certificate you need to access grid is marked as GRID2-FR certificate not as UMR...

How to get registered in you VO

Once you get a CNRS grid certificate, you need to register it in your VO in order to have access to all the grid environment of the experiment:

Go to one of the following web page depending on your VO:

and follow the procedure.

You will receive a confirmation e-mail, just click on the given link to confirm your registration

How to store your certificate on lappsl, lappuds, ccage, lxplus and all linux machines

Create a certificate file containing your grid certificate data :

Open the certificate section of your web browser
Select your CNRS certificate and export it to your windows environment.

This certificate file has to be copied into your LINUX machine account in a .globus directory :

mkdir $HOME/.globus (if not done)

Be careful if you've exported your certificate from Windows, you then have to

dos2unix usercertkey.p12

Once your certificate in your $HOME/.globus you need to convert it :

chmod 600 usercertkey.p12
openssl pkcs12 -nocerts -in usercertkey.p12 -out userkey.pem
openssl pkcs12 -clcerts -nokeys -in usercertkey.p12 -out usercert.pem
chmod 400 userkey.pem #private key
chmod 444 usercert.pem #public cert

The computing element (CE) from LAPP

Under construction

As LAPP user you have more priority than other people from the same VO. You are encouraged to use official tools developed and maintained by your VO team.

The storage element (SE) from LAPP

Under construction

>>> DPM / DPNS / SRM

>>> LFC catalog

>>> How to browse file directories : dpns commands

>>> Local protocol access : rfcp commands

How to submit MPI jobs

In order to invoke MPI-START you need a wrapper script that sets the environment variables that define your job. This script is generic and should not need to have significant modifications made to it.

#!/bin/bash

# Pull in the arguments.
MY_EXECUTABLE=`pwd`/$1
MPI_FLAVOR=$2

# Convert flavor to lowercase for passing to mpi-start.
MPI_FLAVOR_LOWER=`echo $MPI_FLAVOR | tr '[:upper:]' '[:lower:]'`

# Pull out the correct paths for the requested flavor.
eval MPI_PATH=`printenv MPI_${MPI_FLAVOR}_PATH`

# Ensure the prefix is correctly set.  Don't rely on the defaults.
eval I2G_${MPI_FLAVOR}_PREFIX=$MPI_PATH
export I2G_${MPI_FLAVOR}_PREFIX

# Touch the executable.  It exist must for the shared file system check.
# If it does not, then mpi-start may try to distribute the executable
# when it shouldn't.
touch $MY_EXECUTABLE
chmod +x $MY_EXECUTABLE

# Setup for mpi-start.
export I2G_MPI_APPLICATION=$MY_EXECUTABLE
export I2G_MPI_APPLICATION_ARGS=
export I2G_MPI_TYPE=$MPI_FLAVOR_LOWER
# optional hooks
#export I2G_MPI_PRE_RUN_HOOK=mpi-hooks.sh
#export I2G_MPI_POST_RUN_HOOK=mpi-hooks.sh

# If these are set then you will get more debugging information.
#export I2G_MPI_START_VERBOSE=1
#export I2G_MPI_START_DEBUG=1

# Invoke mpi-start.
$I2G_MPI_START

In your JDL file you should set the jobtype as Normal and then set the NodeNumber to the number of desired nodes. The Executable should be your wrapper script for MPI-START (mpi-start-wrapper.sh in this case) and the Arguments are your MPI binary and the MPI flavour that it uses. MPI-START allows user defined extensions via hooks, check the EGI MPI User Guide and Parallel Computing Support User Guide for examples. Here is an example JDL for the submission of the cpi application using 10 processes:

JobType = "Normal";
VirtualOrganisation = "dteam";
NodeNumber = 10;
Executable = "mpi-start-wrapper.sh";
Arguments = "cpi OPENMPI";
StdOutput = "cpi.out";
StdError = "cpi.err";
InputSandbox = {"cpi", "mpi-start-wrapper.sh"};
OutputSandbox = {"cpi.out", "cpi.err"};
Requirements = Member("MPI-START", other.GlueHostApplicationSoftwareRunTimeEnvironment)
            && Member("OPENMPI-1.2.5",  other.GlueHostApplicationSoftwareRunTimeEnvironment);

Please note that the NodeNumber variable refers to the number of CPUs you are requiring. The new EGEE MPI WG is discussing how to implement a fine-grained selection of the nodes and/or CPUs (i.e. to specify the number of processors per node and the number of nodes, not only the number of CPUs).

Personal tools