Troubleshooting XDMCP Connections (Attachmate Technical Note 1229)

This technical note describes how to troubleshoot problems connecting with XDMCP (X Display Manager Control Protocol), such as timeout or "cannot open display" errors. The troubleshooting options in this note apply to connections made to the most common UNIX and Linux hosts.

For information specific to Red Hat Linux, see Technical Note 1680.

XDMCP Connections

The XDMCP protocol is used to communicate between the X Display Manager (XDM) running on a host machine and the X server (Reflection X) running on your workstation (computer). The X Display Manager provides a user friendly interface for starting X clients and opening X windows.

Troubleshooting XDM Connection Problems

There are several reasons why an XDMCP connection might fail. The most common reasons are:

• The XDM daemon is not running on the host, or the host is unavailable.
• Port 177 (UDP) or port 6000 (TCP) is blocked.
• The workstation is not correctly configured in the network DNS table.

To determine the cause of the problem and resolve it, start with section A. XDMCP Error Codes, and proceed through each subsequent section as directed.

Note: The troubleshooting steps in this note help you ensure that the XDM daemon is running and that Reflection X can successfully connect and run sessions using this daemon. An alternate workaround for starting sessions is to manually start X sessions. This procedure is described below under "To manually start an X session without using the XDM Daemon."

A. XDMCP Error Codes

The most common error codes and their definitions are:

• RX2102 - "Your XDMCP connection timed out - make sure that the hosts on your network are running XDM programs. (RX2102)"

This is a generic XDMCP error code. To determine the cause of the error and resolve it, proceed to section B. Examine the Network Environment.

• RX2112 - "XDM session to host failed (Session XXX failed for display <fully qualified PC name>:0: Cannot open display). (RX2112)"

XDM is running on the host and has accepted the connection request; however, XDM is unable to successfully reply to the Reflection X server. Refer to sections F. Verify that Reflection X is Detecting the Correct IP Address and G. Ping the Workstation from the Host.

B. Examine the Network Environment

Before you begin the troubleshooting steps, be sure that your network environment is compatible with XDMCP. XDMCP does not work with, or needs to be specifically configured for, the following network environments:

• Secure Shell (SSH): XDMCP connections cannot be created through an SSH tunnel, because SSH does not support UDP transmissions. See Technical Note 1818 for more information.
• Network Address Translation (NAT): If you are using Reflection X in a network environment configured for NAT (sometimes referred to as IP masquerading, transparent proxying, or IP address overloading), read Technical Note 1513 before proceeding.
• Virtual Private Networks (VPN): If you are connecting to the network through a VPN tunnel, read Technical Note 1580 before proceeding.
• Some hosts include security features that prevent Reflection X (or other X servers) from establishing XDMCP connections. In some cases, this feature is enabled by default. Verify that your host is configured to permit XDMCP connectivity.

For details regarding this feature and Red Hat Linux 8.0, see technical note 1680.

Once you determine that your environment is compatible with XDMCP, proceed to C. Increase the Connection Timeout.

C. Increase the Connection Timeout

Increasing the connection timeout value may resolve connection problems, particularly when working in a low bandwidth environment. The connection timeout defines the number of seconds that Reflection X will try to make an XDMCP connection. By default, the value is set to 15 seconds.

Follow these steps to configure the connection timeout value.

1. In the Reflection X Manager window, click File > New Client File. Then, in the New Connection window, click XDMCP Connection; click OK.
2. Click Advanced to open the Advanced XDMCP Settings dialog box.
3. In the Connection timeout field, enter the number of seconds that Reflection X should wait before timing out, and then click OK.
4. Try to connect.

Did this work?

Yes. Save the .RXC file with the increased timeout setting for future use.

No. Proceed to D. Make a Direct XDMCP Connection.

D. Make a Direct XDMCP Connection

By default, Reflection X makes XDMCP connections using broadcast UDP packets. Some routers and firewalls are configured to block UDP or broadcast packets. Follow these steps to make a non-broadcast XDMCP connection:

1. In the Reflection X Manager window, click File > New Client File. Then, in the New Connection window, click XDMCP Connection; click OK.
2. In the Method drop-down list, select Direct.
3. In the Host name field, enter your IP address or fully-qualified host name (including the domain name). For example, unixhost.mycompany.com.
4. Click Connect.

Note: Depending on the configuration of your network, it may be possible to connect to your host using an unqualified host name; however, because this does not work in all network configurations, using a fully-qualified name is recommended.

Did this work?

Yes. Your router or firewall is most likely configured to block UDP broadcast packets. To workaround this problem, either use the Direct XDMCP connection; configure your workstation's firewall to permit UDP broadcasts, or contact your network administrator about configuring your router or firewall to permit UDP broadcasts.

No. Perform these steps:

1. Follow the steps provided in the previous section, C. Increase the Connection Timeout, and increase the timeout setting for your Direct XDMCP connection. Try the connection again. If it still fails, proceed to step b.
2. If connecting using the fully-qualified host name did not work, try using the host IP address. If the connection still fails, proceed to E. Ping the Host from the Workstation.

E. Ping the Host from the Workstation

Follow the steps below to determine if the host is available.

1. Click Start > Run.
2. In the Open field, enter command, and then click OK.
3. At the command prompt, enter:

For example:

Did the host respond?

Yes. Proceed to F. Verify that Reflection X is Detecting the Correct IP Address.

No. Contact your system administrator to verify that the host is running and that you have the correct name or IP address for the host.

F. Verify that Reflection X is Detecting the Correct IP Address

Reflection X sends the workstation's IP address to the XDM daemon on the host. To verify that the workstation IP address is correct, follow the steps below.

1. Click Start > Run.
2. In the Open field, enter command, and then click OK.
3. At the command prompt, enter the following command and note the IP address displayed:

4. Note the IP address of the interface you want to use.
5. In Reflection X Manager 14.1 SP1 or higher:
   1. Click Settings > View Settings.
   2. In the Search box, type “Detected IP”.
   3. Click the appropriate option for IPv4 or IPv6.
   4. Compare the IP address referenced under “Settings details” with the one you identified above.

In Reflection X Manager 14.0 - 14.1:

1. Click Settings > Network.
2. Compare the IP address referenced under "Autodetect network interface" with the one you identified above.

Do the IP addresses match?

Yes. Proceed to G. Ping the Workstation from the Host.

No. If the IP address automatically detected by Reflection X is not correct, manually enter the correct address under Settings > Network and return to D. Make a Direct XDMCP Connection.

G. Ping the Workstation from the Host

Verify that the host can access the workstation and correctly resolve the workstation's fully-qualified name. Incorrect resolution of the workstation name to the workstation IP address may cause XDMCP connections to fail.

An example of a fully-qualified workstation name is "lindaj.mycompany.com" (not "Lindaj"). Depending on the configuration of your network, it may be possible to connect to your host using an unqualified host name; however, because this does not work in all network configurations, using a fully-qualified name is recommended.

Linux: ping -c 3 <fully-qualified workstation name>

Solaris: ping -a <fully-qualified workstation name>

HP-UX: ping <fully-qualified workstation name> -n 3

Note: You may need to navigate to the /etc directory before issuing the ping command. To do so, enter cd /etc. If you are unable to navigate to /etc, or are still unable to use the ping command from that location, contact your system administrator for assistance.

A successful ping looks similar to this:

Did this work?

Yes. If you can ping the workstation by name, and the IP address returned is correct, proceed to H. Confirm that the Host XDM Daemon is Running.

If you can ping, but the IP address returned by ping is incorrect, you may have identified a DNS problem and should contact your network administrator for assistance. This problem must be resolved before proceeding.

Note: A temporary workaround for a DNS error is for the system administrator to add the correct workstation IP address and fully-qualified workstation name to the host's /etc/hosts file. Once this has been done, pinging from the host to the workstation should resolve properly. This method should only be used as a temporary workaround, especially in a DHCP network environment, because the /etc/hosts file would need to be updated each time the workstation obtains a new DHCP lease.

No. If you are unable to successfully ping using the workstation's name, try using the workstation's IP address in place of <fully-qualified workstation name>. If this works, proceed to the next section, "Reconfigure the Windows DNS suffix setting."

If a ping to the workstation IP address fails, contact your network administrator for assistance.

Note: Personal firewalls (running on the workstation) can be configured to disallow incoming echo requests (such as ping requests). If you have a personal firewall on your workstation, ensure that it is configured to allow incoming echo requests while you are testing with the ping command.

Reconfigure the Windows DNS suffix setting:

1. Open the Local Area Connection Properties dialog box:

2. Select Internet Protocol (TCP/IP). If multiple protocol versions are present, select the appropriate protocol version (IPv4 or IPv6).
3. Click Advanced.
4. On the DNS tab, check "Use the connection's DNS suffix in DNS registration," click OK, and exit Local Area Connection Properties.
5. Re-try your host connection.

If changing this setting does not resolve the issue, clear the "Use this connection's DNS suffix in DNS registration" check box, and save the setting again. Proceed to H. Confirm that the Host XDM Daemon is Running.

H. Confirm that the Host XDM Daemon is Running

Follow these steps to confirm that the XDM daemon is running on your host.

1. Make a connection to your host using either Reflection for UNIX and OpenVMS or Reflection Workspace, as in step G.
2. At the command prompt, enter the appropriate command for your host system:

Host	Command
UNIX host running CDE	ps -eaf | grep dtlogin
UNIX - Generic	ps -eaf | grep login
Linux host running KDM	ps -eaf | grep kdm
Linux host running GDM	ps -eaf | grep gdm
Linux host running XDM	ps -eaf | grep xdm

A typical UNIX host response looks similar to the following; the path will vary depending on the host environment.

In this example, the 'root' line confirms that the XDM daemon, entitled dtlogin, is running on the host, and is owned by the user 'root.'

A typical Linux host response looks similar to the following:

In this example, the 'root' line confirms that the XDM daemon, entitled kdm, is running on the host, and is owned by the user 'root.' (The '-nodaemon' parameter does not indicate that the daemon is unavailable.)

Note the following:

• The 'guest' line should be disregarded; it shows the status of the 'grep' command, not the status of the XDM daemon.
• If you do not get a response after entering the appropriate command for your host, the XDM daemon is not currently running. Is the Daemon running?

Yes. Proceed to I. Check Port 177.

No. Contact your system administrator about the daemon. For more information on the XDM daemon, refer to the host Man pages.

Note: Whether the XDM Daemon is running or not, you can follow the steps below to manually start your X sessions. This workaround may allow you to use Reflection X while you are troubleshooting the XDMCP connection problem.

To manually start an X session without using the XDM Daemon

You can use the following steps to start an X desktop session if the XDM Daemon is not running.

1. Start Reflection X (it can be minimized).
2. Connect to your host using Reflection for UNIX and OpenVMS or Reflection Workspace.
3. At the host command prompt, enter:

For UNIX:

For Linux:

To use either of these last two Linux commands, the display variable has to be set first as neither supports the -display switch. (For example: "export SET DISPLAY=<workstation IP or name>:0")

Note: When you manually start an X session, some windows may stay open when you exit. To close these windows, reset your Reflection X server from the Action menu in the Reflection X Manager.

I. Check Port 177

To make an XDMCP connection, the network must be configured to allow the host and X server to communicate using UDP packets over port 177.

To verify that port 177 is open, contact your network administrator for assistance or use the Microsoft Portqry.exe command-line utility to determine if port 177 is available. For information about this utility, see Microsoft article 310099 at http://support.microsoft.com/default.aspx?scid=kb;en-us;310099.

Is Port 177 open on the host?

Yes. Proceed to J. Check Port 6000.

No. Work with your network administrator to identify why port 177 is unavailable. The port can be blocked by a network router and/or firewall. Connections using XDMCP are not possible until UDP port 177 is open.

J. Check Port 6000

X protocol typically uses port 6000 for communication between the host and Reflection X; therefore, if port 6000 is blocked between the host and the workstation, you will probably be unable to connect using Reflection X (one exception is when using Secure Shell). Follow the steps below to determine if port 6000 is available for the X protocol.

1. Start Reflection X and minimize the Reflection X Manager window. When Reflection X is started, it attempts to open port 6000 on the workstation.
2. Make a connection to the host using Reflection for UNIX and OpenVMS or Reflection Workspace.
3. At the command prompt, type the following:

For example: telnet lindaj.mycompany.com 6000

Port 6000 is open between the host and the workstation if the response looks similar to the following:

To exit the telnet connection, enter Ctrl+], and then enter quit.

Port 6000 is not open between the host and the workstation if the response is similar to one of the following strings:

Is port 6000 open?

Yes. For further troubleshooting assistance, contact Attachmate Technical Support: http://support.attachmate.com/contact/.

No. If this is the case, check the following.

Firewalls:

• Ensure that there is not a personal firewall blocking port 6000. See K. Check the Windows Firewall.
• Some versions of the Cisco VPN Client contain a built-in firewall called Stateful Firewall. If this is running on your workstation, configure it to allow traffic on port 6000. For further information regarding this product, contact Cisco support or your company’s IT department.
• Some versions of the Checkpoint VPN client contain a built-in firewall. If this is running on your workstation, configure the client to allow traffic over port 6000. For further information regarding this product, contact Checkpoint support or your company's IT department.
• Talk to your network administrator to determine if a router or network firewall is blocking port 6000.

Note: It is not always obvious that a firewall is running, because it might be running in the background as a service. You may be able to determine if a firewall is running in this manner by looking in the Task Manager under the Application and Processes tabs, or by looking in the Services list. (Click Start > Control Panel > Administrative Tools > Services.)

Other Applications:

Determine if port 6000 is currently in use by an application other than Reflection X by entering netstat -a at the Windows Command Prompt (Programs > Accessories > Command Prompt).

• If port 6000 is being used, you will see a line similar to the following:

• If port 6000 is being used, and Reflection X is not running, another application is using port 6000 (and possibly disallowing Reflection X access to the port). Determine what application is using the port and disable it.
• Depending on which operating system you are using, you may be able to use a freeware utility, such as Active Ports by SmartLine Inc, to determine what application is using this port. This utility can be obtained by doing a web search for "active ports smartline".

If you are successful in opening port 6000, try to connect using XDMCP again. If still unable to connect, proceed to Contact Attachmate.

K. Check the Windows Firewall

Follow the steps below to view and modify the Windows Firewall settings

1. Click Start > Control Panel > Windows Firewall.

Is the firewall enabled?

Yes. Before you can make an XDMCP connection, you must include Reflection X in the list of allowed programs. (You may also choose to disable the firewall temporarily for testing, or permanently if this is appropriate in your environment.)

Add Reflection X to the list of allowed programs:

On Windows 7:

1. Click Start > Control Panel > Windows Firewall.
2. From the left pane, click Allow a program or feature through Windows Firewall.
3. Click Allow another program.
4. Browse to select the Reflection X program file. (The default location is either C:\Program Files\Attachmate\Rx.exe, or C:\Program Files\Attachmate\R14\Rx.exe.)

On Windows XP:

1. Click Start > Control Panel > Windows Firewall.
2. Click Exceptions.
3. Click Add Program.
4. Browse to select the Reflection X program file. (The default location is either C:\Program Files\Attachmate\Rx.exe, or C:\Program Files\Attachmate\R14\Rx.exe.)

No. Return to J. Check Port 6000 and proceed through the firewalls list. Or, if you have already done this, proceed to Contact Attachmate.

L. Record and Review a Network Trace

If you are familiar with network traces, take a network trace of your XDMCP connection attempt and refer to the following Reflection XDMCP connection details while reviewing the trace.

Note: If you do not have a network trace utility, you may want to try one of the free online utilities, such as Wireshark, a multi-platform network protocol analyzer available at http://www.wireshark.org/.

In the network trace, you should see something similar to the following, showing Reflection X and the host negotiating and maintaining the connection. (Exactly how the information is presented depends on the trace utility you are using.)

The first set of entries show Reflection X connecting to port 177 on the host using XDMCP protocol over UDP. Communication is established between a random port number on the PC and port 177 on the host. During this process Reflection X (on the PC) provides the host with information about which port number it is configured to use for X11 protocol. Typically, this is port 6000.

Note: If there is a firewall between the PC and host, the firewall must be configured to allow UDP packets from the PC to the host on port 177.

The second set of entries shows the host sending a SYN request to the PC on the port that Reflection X specified for X11 protocol (typically port 6000), using TCP. The following SYN and ACK communications are made between a random host port and the port Reflection X specified for X11 protocol.

Note: If there is a firewall between the PC and host, the firewall must be configured to allow TCP packets from the host to the PC on port 6000 (or whichever port Reflection X is listening on for X11 traffic).

Finally, the host and Reflection X begin communicating using X11 protocol over TCP on the ports established during the SYN-ACK process above. No additional ports need be opened in the firewall.

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March 12, 2019