I have a cluster of 7 nodes (All Centos 7 OS). Master node is maercher5 and the rest are slave nodes. I need to setup passwordless ssh on the master node to the slave nodes to run MPI programs. The home directory is shared by NFS from the master node to all the slave nodes. I followed this tutorial to do a passwordless ssh from master node to slave nodes. I have the same UID and GID on all machines. Since there is only 1 ssh folder shared on all nodes. Permissions for ssh folder is:
$ ls -al  $HOME/.ssh
total 28
drwx------.  2 sarah sarah    76 Apr 16 21:17 .
drwx------. 17 sarah sarah  4096 Apr 17 13:51 ..
-rw-------.  1 sarah sarah 11895 Apr 16 21:17 authorized_keys
-rw-------.  1 sarah sarah  1679 Apr  3 00:55 id_rsa
-rw-r--r--.  1 sarah sarah   411 Apr 10 14:24 id_rsa.pub
-rw-------.  1 sarah sarah  2265 Apr 10 13:58 known_hosts

Nodes can ping each other well. Marcher5 is the master node.

[sarah@marcher5]$ cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.250.15  marcher5.cs.txstate.edu marcher5
192.168.250.17  marcher7.cs.txstate.edu marcher7
192.168.250.18  marcher8.cs.txstate.edu marcher8
192.168.250.19  marcher9.cs.txstate.edu marcher9
192.168.250.20  marcher10.cs.txstate.edu marcher10
192.168.250.21  marcher11.cs.txstate.edu marcher11
192.168.250.22  marcher12.cs.txstate.edu marcher12

On all slave nodes, NFS Mount is as follows:

[sarah@marcher11 ~]$ cat /etc/fstab

/dev/mapper/centos-root /                       xfs     defaults        1 1
UUID=79c2716b-9099-4731-82cc-094ca26eb837 /boot                   xfs     defaults        1 2
#/dev/mapper/centos-home /home                   xfs     defaults        1 2
/dev/mapper/centos-swap swap                    swap    defaults        0 0
marcher5:/home/sge_users /home/sge_users nfs soft,intr,bg,nosuid,timeo=20,retrans=10,async,wsize=8192,rsize=8192  0 0

[sarah@marcher11 ~]$ mount |grep home
    /dev/mapper/centos-home on /home type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
    marcher5:/home/sge_users on /home/sge_users type nfs4 (rw,nosuid,relatime,vers=4.0,rsize=8192,wsize=8192,namlen=255,soft,proto=tcp,port=0,timeo=20,retrans=10,sec=sys,clientaddr=192.168.250.21,local_lock=none,addr=192.168.250.15)

The problem is that passwordless ssh does not work.

[sarah@marcher5 mpi2007]$ ssh -v marcher11
OpenSSH_6.6.1, OpenSSL 1.0.1e-fips 11 Feb 2013
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 56: Applying options for *
debug1: Connecting to marcher11 [192.168.250.21] port 22.
debug1: Connection established.
debug1: identity file /home/sge_users/sarah/.ssh/id_rsa type 1
debug1: identity file /home/sge_users/sarah/.ssh/id_rsa-cert type -1
debug1: identity file /home/sge_users/sarah/.ssh/id_dsa type -1
debug1: identity file /home/sge_users/sarah/.ssh/id_dsa-cert type -1
debug1: identity file /home/sge_users/sarah/.ssh/id_ecdsa type -1
debug1: identity file /home/sge_users/sarah/.ssh/id_ecdsa-cert type -1                                                           [29/1894]
debug1: identity file /home/sge_users/sarah/.ssh/id_ed25519 type -1
debug1: identity file /home/sge_users/sarah/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1
debug1: match: OpenSSH_6.6.1 pat OpenSSH_6.6.1* compat 0x04000000
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5-etm@openssh.com none
debug1: kex: client->server aes128-ctr hmac-md5-etm@openssh.com none
debug1: kex: curve25519-sha256@libssh.org need=16 dh_need=16
debug1: kex: curve25519-sha256@libssh.org need=16 dh_need=16
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ECDSA 80:81:97:62:dd:9b:fc:e2:76:bc:13:ce:30:07:79:49
debug1: Host 'marcher11' is known and matches the ECDSA host key.
debug1: Found key in /home/sge_users/sarah/.ssh/known_hosts:5
debug1: ssh_ecdsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
debug1: Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available

debug1: Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available

debug1: Unspecified GSS failure.  Minor code may provide more information


debug1: Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available

debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/sge_users/sarah/.ssh/id_rsa
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Trying private key: /home/sge_users/sarah/.ssh/id_dsa
debug1: Trying private key: /home/sge_users/sarah/.ssh/id_ecdsa
debug1: Trying private key: /home/sge_users/sarah/.ssh/id_ed25519
debug1: Next authentication method: password
sarah@marcher11's password:
debug1: Authentication succeeded (password).
Authenticated to marcher11 ([192.168.250.21]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env LC_ALL = C
debug1: Sending env LANG = en_US.UTF-8

I've been gated in this issue for over a month, any help would be appreciated. I tried to do this from root@master to root@slave and it works.