What is it?

The CernVM-File System (CernVM-FS) provides a scalable, reliable and low- maintenance software distribution service. It was developed to assist High Energy Physics (HEP) collaborations to deploy software on the worldwide- distributed computing infrastructure used to run data processing applications. CernVM-FS is implemented as a POSIX read-only file system in user space (a FUSE module). Files and directories are hosted on standard web servers and mounted in the universal namespace /cvmfs.

JLab Specifics

CVMFS is preconfigured on ifarm/farm nodes. Here's the setting used for the local squid caches:

CVMFS_HTTP_PROXY="http://sci-squid901.jlab.org:3128|http://sci-squid902.jlab.org:3128|http://sci-squid903.jlab.org:3128|http://sci-squid904.jlab.org:3128"

We publish our software to CVMFS via OASIS (OSG Application Software Installation Service).

Example usage on ifarm

Use the software stored there just like you would locally installed packages.

% ls /cvmfs/oasis.opensciencegrid.org/
accre  auger    csiu  fermilab  glow   ilc   ligo  nanohub  osg           sbgrid
atlas  cmssoft  enmr  geant4    gluex  jlab  mis   nova     osg-software  snoplussnolabca

For usage of Scicomp provided software, see: Environment Modules

Getting data into OASIS

Data written to the source areas listed below will be rsynced and then published to OASIS for use via CVMFS.

src: /scigroup/cvmfs/<project>
dst: /cvmfs/oasis.opensciencegrid.org/jlab/<project>
log: /scigroup/cvmfs/logs/<project>.log

rate: about every 4 hours via cronjob

The individual project directories at the top-level will be rsynced within the scheduled cronjob. This requires an update to the cronjob script for new directories. Submit ServiceNow incident to request updates or changes.

Adding and removing files from the src directory will be reflected at roughly at the cronjob interval. Once all rsyncs are completed, if file changes were made, an update job is scheduled within the OSG.

Log files

Each project will have an associated log file. These provide a great resource to verify successful rsyncs (ex. no permissions errors). They will be rotated and purged.

Rsync Excludes

By default, each top-level project directory will be given an exclude file (oasis_exclude.txt). This file is passed to rsync's `--exclude-from` option (see the man page for details). Things added to this file after they have already been published, will be deleted.

Need help?

Submit a ServiceNow incident for help regarding adding a new project or any issues getting your files updated.

CVMFS at JLab