Grid

From Support-Applicatif

Jump to: navigation, search

Contents

Useful links

>>> LAPP user support

Your message will be managed by the LAPP computing departement helpdesk (Request Tracker). Please avoid contacting directly a member of IT department team and use the following generic addresses.

LAPP and LAPTh members: LAPP-LAPTh IT support
All other users: MUST user support

>>> LAPP IT Department documentation

Some documentation can also be found inside LAPP IT department documentation (restricted access to LAPP & LAPTh users)

>>> LAPP cluster jobs monitoring

You can check the load of the system on the MUST monitoring Web page

>>> LAPP cluster storage monitoring

Only reachable when connected to LAPP network (or via VPN): LAPP cluster monitoring

>>> Overview of all grid sites

An overview of all grid sites is available here : https://operations-portal.egi.eu/vapor

You will find all information you need about all EGI sites : error/downtime, installed software releases, OS, ...

WLCG REsource, Balance & USage is available at https://gstat-wlcg.cern.ch/apps/topology/

LAPP site WLCG information is here: https://gstat-wlcg.cern.ch/apps/topology/federation/228/

>>> EGI web site

The official EGI web site is https://www.egi.eu/

>>> NEW : Grid for Newbies

A very useful document if you don't know anything about the Grid: https://forge.in2p3.fr/attachments/download/13593/La_Grille_pour_les_Nuls_v1.0.pdf

>>> NEW : MOOC on Grid Computing

A Massive Open Online Course (MOOC) on Grid Computing is available here : http://doc.grid.surfsara.nl/en/latest/Pages/Tutorials/MOOC/mooc.html#mooc

>>> Operations Portal

Any information about VO can be found here: http://operations-portal.egi.eu/

Grid / EGI user's guide

This guide is an introduction to the WLCG/EGI Grid and to the gLite 3 middleware from a user's point of view: https://edms.cern.ch/ui/file/722398/1.4/gLite-3-UserGuide.pdf

Grid certificate / VO

To check if you already have a valid Grid certificate

First method  : open the certificate section of your web browser and check the validity date of your certificate:

  • for Chrome, see [[1]]
  • for Mozilla, see [[2]]
  • for Internet Explorer, see [[3]].

Second method :

How to get a certificate until May 31, 2018

This page is a brief summary of the grid quick start guide mentioned above

Your first need to get a CNRS grid certificate at https://igc.services.cnrs.fr/GRID2-FR.

You'll receive a mail for confirmation.

Do NOT follow the instruction included in the email, but do this instead :

Using the browser used to request the new certificate, go to :

https://igc.services.cnrs.fr/search_user_certificate/?CA=GRID2-FR&lang=fr&body=search_user.html

  • Look for the certificate using your user name
  • Use the option "Sauvegarder dans un fichier" to save it in a file on your computer
  • Go into your browser menu (exemple for firefox )

Preferences > Privacy & Security > Certificates > View Certificates

  • click on "Import", check all possible file extensions
  • Open the previously saved .crt file
  • You are done, certificate is now in your browser.

How to get a certificate after May 31, 2018

Import the certification chain in your browser
  • download the Chaine_GRID-FR.p7b file
  • in your browser, go to the "Advanced options", display the "Certificates", open the "Authorities" tab, "Import" the previously downloaded Chaine_GRID-FR.p7b file, check the 3 options and validate 4 times in a row (it is slightly different each time so do not cancel instead of validate even if thinking it is the same)
Request a certificate
  • Go to https://pub-ee-grid.pncn.education.gouv.fr/EE/ (with FIREFOX, not CHROME)
  • Select "Demander un certificat" (Request a certificate)
  • Select "Personnels" for the category and the certificates management profile
  • Enter your e-mail address
  • Fill-in the form with "High level" selected
  • Once you receive your confirmation e-mail, open the link in the same browser as the one that you used for the request and then click on the link to integrate the certificate into your browser.

CAUTION : keep the received e-mail including your revocation code

New grid users who never got an EGI grid certificate shall have to provide information in order to be registered in the french grid database (name, surname,e-mail address, unit=laboratory, organization=CNRS). All this information will be validated by the IT support team of the laboratory. This registration will be done only once, when the user makes his first grid certificate request.

"Old" grid users, depending on the VO they belong to, might have to add their new certificate to their grid profile selecting add an additional certificate on the relevant VOMS server (it will be validated by the VO Manager). This should NOT happen for CERN VOs such as ATLAS or LHCb.

Recommandations

Be careful choosing your passwords !!!

CAUTION : the Grid certificate you need to access the EGI grid is marked as GRID-FR not as UMR...

How to get registered in a VO (Virtual Organization)

Once you get a Grid certificate, you need to register it in a VO in order to get access to all the grid environment of the associated experiment.

Go to one of the following web page depending on your VO:

and follow the procedure.

You will receive a confirmation e-mail, just click on the given link to confirm your registration.

If the link doesn't work, please change web browsers.

How to store your certificate on lappsl, lappuds, cca, lxplus and all linux machines

Create a certificate file containing your grid certificate data:

Open the certificate section in the options of your web browser
Select your GRID-FR certificate and export it to your windows environment with the PKCS12 format.

This certificate file has to be copied into your LINUX machine account in a .globus directory:

mkdir $HOME/.globus (if not existing)

Be careful if you've exported your certificate from Windows, you then have to convert it first to Linux format:

dos2unix usercertkey.p12

Once your certificate is in your $HOME/.globus, you need to convert it to pem format:

chmod 600 usercertkey.p12
openssl pkcs12 -nocerts -in usercertkey.p12 -out userkey.pem
openssl pkcs12 -clcerts -nokeys -in usercertkey.p12 -out usercert.pem
chmod 400 userkey.pem #private key
chmod 444 usercert.pem #public cert

The computing element (CE) from LAPP

Under construction

As LAPP user you have more priority than other people from the same VO. You are encouraged to use official tools developed and maintained by your VO team.

The storage element (SE) from LAPP

Under construction

>>> DPM / DPNS / SRM

>>> LFC catalog

>>> How to browse file directories : dpns commands

>>> Local protocol access : rfcp commands

How to submit MPI jobs

In order to invoke MPI-START you need a wrapper script that sets the environment variables that define your job. This script is generic and should not need to have significant modifications made to it.

#!/bin/bash

# Pull in the arguments.
MY_EXECUTABLE=`pwd`/$1
MPI_FLAVOR=$2

# Convert flavor to lowercase for passing to mpi-start.
MPI_FLAVOR_LOWER=`echo $MPI_FLAVOR | tr '[:upper:]' '[:lower:]'`

# Pull out the correct paths for the requested flavor.
eval MPI_PATH=`printenv MPI_${MPI_FLAVOR}_PATH`

# Ensure the prefix is correctly set.  Don't rely on the defaults.
eval I2G_${MPI_FLAVOR}_PREFIX=$MPI_PATH
export I2G_${MPI_FLAVOR}_PREFIX

# Touch the executable.  It exist must for the shared file system check.
# If it does not, then mpi-start may try to distribute the executable
# when it shouldn't.
touch $MY_EXECUTABLE
chmod +x $MY_EXECUTABLE

# Setup for mpi-start.
export I2G_MPI_APPLICATION=$MY_EXECUTABLE
export I2G_MPI_APPLICATION_ARGS=
export I2G_MPI_TYPE=$MPI_FLAVOR_LOWER
# optional hooks
#export I2G_MPI_PRE_RUN_HOOK=mpi-hooks.sh
#export I2G_MPI_POST_RUN_HOOK=mpi-hooks.sh

# If these are set then you will get more debugging information.
#export I2G_MPI_START_VERBOSE=1
#export I2G_MPI_START_DEBUG=1

# Invoke mpi-start.
$I2G_MPI_START

In your JDL file you should set the jobtype as Normal and then set the NodeNumber to the number of desired nodes. The Executable should be your wrapper script for MPI-START (mpi-start-wrapper.sh in this case) and the Arguments are your MPI binary and the MPI flavour that it uses. MPI-START allows user defined extensions via hooks, check the EGI MPI User Guide and Parallel Computing Support User Guide for examples. Here is an example JDL for the submission of the cpi application using 10 processes:

JobType = "Normal";
VirtualOrganisation = "dteam";
NodeNumber = 10;
Executable = "mpi-start-wrapper.sh";
Arguments = "cpi OPENMPI";
StdOutput = "cpi.out";
StdError = "cpi.err";
InputSandbox = {"cpi", "mpi-start-wrapper.sh"};
OutputSandbox = {"cpi.out", "cpi.err"};
Requirements = Member("MPI-START", other.GlueHostApplicationSoftwareRunTimeEnvironment)
            && Member("OPENMPI-1.2.5",  other.GlueHostApplicationSoftwareRunTimeEnvironment);

Please note that the NodeNumber variable refers to the number of CPUs you are requiring. The new EGEE MPI WG is discussing how to implement a fine-grained selection of the nodes and/or CPUs (i.e. to specify the number of processors per node and the number of nodes, not only the number of CPUs).

Personal tools