Harnessing HPC resources for CMS jobs using a Virtual Private Network

The processing needs for the High Luminosity (HL) upgrade for the LHC require the CMS collaboration to harness the computational power available on non-CMS resources, such as High-Performance Computing centers (HPCs). These sites often limit the external network connectivity of their computational n...

Full description

Saved in:
Bibliographic Details
Published in:EPJ Web of Conferences 2021, Vol.251, p.2032
Main Authors: Tovar, Benjamin, Bockelman, Brian, Hildreth, Michael, Lannon, Kevin, Thain, Douglas
Format: Article
Language:English
Subjects:
Large Hadron Collider
Luminosity
Nodes
Servers
Service introduction
Solenoids
Virtual networks
Virtual private networks
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The processing needs for the High Luminosity (HL) upgrade for the LHC require the CMS collaboration to harness the computational power available on non-CMS resources, such as High-Performance Computing centers (HPCs). These sites often limit the external network connectivity of their computational nodes. In this paper we describe a strategy in which all network connections of CMS jobs inside a facility are routed to a single point of external network connectivity using a Virtual Private Network (VPN) server by creating virtual network interfaces in the computational nodes. We show that when the computational nodes and the host running the VPN server have the namespaces capability enabled, the setup can run entirely on user space with no other root permissions required. The VPN server host may be a privileged node inside the facility configured for outside network access, or an external service that the nodes are allowed to contact. When namespaces are not enabled at the client side, then the setup falls back to using a SOCKS server instead of virtual network interfaces. We demonstrate the strategy by executing CMS Monte Carlo production requests on opportunistic non-CMS resources at the University of Notre Dame. For these jobs, cvmfs support is tested via fusermount (cvmfsexec), and the native fuse module.
ISSN:2100-014X
2101-6275
2100-014X
DOI:10.1051/epjconf/202125102032