Connecting to Farm and QCD Interactive Nodes


Table of Contents

 

Login Nodes

In order to meet updated security requirements, the interactive nodes (ifarm/qcdi) now sit behind 2-factor login nodes.  The login nodes can be reached from on- and off-site.  Acceptable login nodes include login.jlab.org. and acclogin.jlab.org. (and any of the several aliases which actually also land you on login).

NOTE:  direct connections to ifarm/qcdi nodes are expected to be BLOCKED from all user VLANs.  Connections must be made directly from a 2-factor login node.  Actions like ssh login.jlab.org. => mydesktop => ifarm will NOT be allowed, as opposed to

See article on configuring ProxyJump in your .ssh/config to streamline this workflow.

QCD Users:  references to ifarm/Farm within the article should be treated as examples.  Substitute your specific qcdi hostname as appropriate.

 

Prerequisites

  1. Multi-Factor Authentication (MFA), one of
  2. account access, one of

 

Connecting to JLab Interactive Nodes

Please ensure all prerequisites in the list above have been met.  Then, to access the Farm using SSH you can

ssh <user>@login.jlab.org.
Password: <PIN followed by token, no spaces>
ssh ifarm
Password: <CUE password>

Alternatively, one can use the ProxyJump directive (-J).  Just remember the first login is 2-factor and the second is CUE password, and that mistyped credentials can result in additional, confusingly-unlabled password prompts, depending on your OpenSSH version.

ssh -J <user>@login.jlab.org. <user>@ifarm
Password: <PIN followed by token, no spaces>
Password: <CUE password>

See below if your version of OpenSSH does not support -J.  The second authentication can also be via SSH key instead of CUE password, to your personal account or group account, but the first authentication must be two-factor and using your personal account.

 

X-forwarding

To run an X client application, you may need to specify the appropriate flag (if X-forwarding isn't configured in the SSH system or user client configuration). For example,

ssh -X -J <user>@login.jlab.org <user>@ifarm

An easy test to make sure it's working is to try running xclock:

xclock

If it's working, you will see the xclock graphical clock open on your local display. If not, retry swapping -X with the -Y flag.  See below if your version of OpenSSH does not support -J.

 

Transferring Files

Files can be transferred with scp, sftp, or rsync. For simplicity, we'll show scp using ProxyJump (-J):

scp -J <user>@login.jlab.org. file_to_upload <user>@ifarm:/dest/path/
login.jlab.org.'s Password: <PIN+token, no spaces or +>
ifarm's Password: <CUE password>
scp -J <user>@login.jlab.org. <user>@ifarm:/file/to/download ./
login.jlab.org.'s Password: <PIN+token, no spaces or +>
ifarm's Password: <CUE password>

Add -r (before -J, with a space in between, or combined with it, -rJ) to copy whole directories. Note that if you try to scp, sftp, rsync, or otherwise transfer files to and from login.jlab.org. directly, you will not have access to the same filesystems that are available on ifarm or qcdi.  See below if your version of OpenSSH does not support -J.

 

ProxyJump Configuration (Advanced)

Instead of using -J in the above commands or figuring out the ssh options to pass rsync, refer to our article on configuring ProxyJump. It tries to include many possible cases, but should be fairly easy to copy/paste.

After that, your individual commands will be greatly simplified. Examples:

ssh ifarm
scp testfile ifarm:/dest/path/
rsync testdir ifarm:/dest/path/

 

Issues

Generally

To simplify things, you can bypass your .ssh/config.  If the following works, the problem is likely with your config file.

# If off-site, you need to include the domain name
ssh -F /dev/null <username>@login.jlab.org

You can also try ssh -F ~/.ssh/config ifarm which if works means specifically that the problem with your config file is that OpenSSH is ignoring it, and if does not work will give a much more obvious error if, for example, you created the config file in the wrong location and ~/.ssh/config therefore does not exist.

If you're able to log in but some other functionality is broken, it may behoove you to disable any shell initialization files on the destination, e.g. by moving them

[user@ifarm2402 ~]$ mv .cshrc .cshrc.disabled

to see if something in them is causing the breakage.

 

ssh/scp/sftp commands have no -J option

unknown option -- J
usage: scp [-12346BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]
           [-l limit] [-o ssh_option] [-P port] [-S program]
           [[user@]host1:]file1 ... [[user@]host2:]file2

ProxyJump and the corresponding -J flag were added to OpenSSH in version 7.3, and the -J flag was extended to the scp and sftp commands in version 8.0.  So if your ssh command supports -J, but your scp and sftp commands do not, you can use

-o ProxyJump='%r@login.jlab.org.'

instead, and if neither your ssh nor scp/sftp commands support -J, you can use

-o ProxyCommand='ssh -W %h:%p %r@login.jlab.org.'

Alternatively, you can avoid needing to specify these options at all by configuring ProxyJump (or ProxyCommand) in your .ssh/config.

If your OpenSSH doesn't support %r, complaining percent_expand: unknown key %r, replace it with your JLab username.

If your OpenSSH doesn't support -W, that suggests it is a version older than 5.4, and, while there exist solutions involving the use of programs like netcat, versions of OpenSSH older than 5.4 were last shipped in Enterprise Linux 6 and Ubuntu 10.04 "Lucid," which reached end-of-life November 2020 and April 2015, respectively, so it is likely, whatever distribution you are using, that you are no longer receiving security updates (unless you are paying for them) and need to upgrade to a new version rather than attempt to employ such a solution.

 

"Received message too long"

Fortunately, OpenSSH tells you pretty directly what to do.

scp: Received message too long 1734439522
scp: Ensure the remote shell produces no output for non-interactive sessions.

You have something in your .cshrc or .bashrc (or some other initialization script or something being run by those scripts) that emits output.  If the culprit is not apparent when you inspect your initialization scripts, you can try progressively commenting parts out and SSHing again until the output is no longer present.

I (lsh) personally like a clean, fast log-in, but if you insist on uncommenting the culprit afterward, strategies for allowing it to coexist with SCP include

 

SCP/ssh host command... hangs

I've seen this where a user was trying to have their .tcshrc run another shell.  The latter shell then waits indefinitely for input, so the login shell never finishes its initialization script, and the non-interactive command never runs.  Please just file a ticket to have your login shell changed.

 

"no hostkey alg"

EL6 has this problem connecting to EL9.  You can add -o HostKeyAlgorithms=ecdsa-sha2-nistp256 (also to the ProxyCommand ssh if applicable), but you will then find, if Proxying, that RHEL6's OpenSSH sends an invalid originator port when attempting to set up the channel.  You can avoid that by using nc instead of ssh -W, or preferably, with a newer OpenSSH!

 

When reporting to the CST Division Help Desk

Please include the following in your incident report: