Linux Networking Configuration

The first step in configuring Linux TCP-IP stack is configuring the network card interface.  Different flavors of Linux has different configuration files and slightly different approach to the configuration

• Redhat Network Configuration
  • RHEL NTP configuration
  • Sendmail on RHEL
  • How to disable SELinux
  • RHEL Tips
• Suse network configuration
  • Pure-FTPd installation and configuration
  • Setting up static routing in SUSE
  • Setting Up an SSH Server on Suse
  • Suse /etc/sysconfig/network/route file
  • Suse NFS service
  • Suse Tips

Outdated but still useful free ebook on the topic is Linux Network Administrator's Guide, 2nd Edition By Olaf Kirch & Terry Dawson (June 2000 )

Linux networking encompass a wide variety of protocols and applications. Among them the most important is LAMP stack. LAMP is an acronym for a solution stack of free, open source software, referring to the first letters of Linux (operating system), Apache HTTP Server, MySQL (database software) and PHP (or sometimes Perl or Python), principal components to build a viable general purpose web server. See

• Apache Webserver
• HTTP Protocol
• HTTP Return Codes
• HTTP Servers Log Analyses
• MySql

NEWS CONTENTS

• 20200613 : Configure Networking on CentOS 8 with nmcli Command (20 Examples) ( Jun 13, 2020 , www.howtoforge.com )
• 20190726 : How to check open ports in Linux using the CLI> by Vivek Gite ( Jul 26, 2019 , www.cyberciti.biz )
• 20190130 : TCP - IP communication of RHEL 7 is extremely slow compared with RHEL 6 and earlier. - Red Hat Customer Portal ( Jan 30, 2019 , access.redhat.com )
• 20181027 : Linux Kernel /etc/sysctl.conf Security Hardening ( Oct 23, 2018 , www.cyberciti.biz )
• 20101130 : Life As A Sys Admin Best Networking Tweaks for Linux by Michael Adams ( Nov 29, 2010 , Network World )
• 20101130 : IBM Redbooks Linux Performance and Tuning Guidelines ( IBM Redbooks Linux Performance and Tuning Guidelines, )
• 20090225 : How to troubleshoot RHEL performance bottlenecks by Ken Milberg ( How to troubleshoot RHEL performance bottlenecks, Feb 25, 2009 )
• 20090223 : Deployment_Guide ( Deployment_Guide, Feb 23, 2009 )

Old News ;-)

[Jun 13, 2020] Configure Networking on CentOS 8 with nmcli Command (20 Examples)

Jun 13, 2020 | www.howtoforge.com

1. Check NetworkManager Status

To check whether the Network Manager is running or not using the following command:

nmcli -t -f RUNNING general

You should see the following output:

running

To get a general status, run the following command:

nmcli general

You should see the following output:

STATE      CONNECTIVITY  WIFI-HW  WIFI     WWAN-HW  WWAN    
connected  full          enabled  enabled  enabled  enabled

2. Check All Available Device

You can display all available device in your system, run the following command:

nmcli dev status

You should see the following output:

DEVICE  TYPE      STATE      CONNECTION  
eth0    ethernet  connected  System eth0 
eth1    ethernet  connected  System eth1 
lo      loopback  unmanaged  --

3. Check Active Connection

To list all active connection in your system, run the following command:

nmcli con show

You should get the following output:

NAME         UUID                                  TYPE      DEVICE 
System eth0  5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03  ethernet  eth0   
System eth1  9c92fad9-6ecb-3e6c-eb4d-8a47c6f50c04  ethernet  eth1

4. Display Information About Ethernet Connection

You can display the brief information about the ethernet connection using the following command:

nmcli con show "System eth0"

You should get the following output:

connection.id:                          System eth0
connection.uuid:                        5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03
connection.stable-id:                   --
connection.type:                        802-3-ethernet
connection.interface-name:              eth0
connection.autoconnect:                 yes
connection.autoconnect-priority:        0
connection.autoconnect-retries:         -1 (default)
connection.multi-connect:               0 (default)
connection.auth-retries:                -1
connection.timestamp:                   1588217245
connection.read-only:                   no
connection.permissions:                 --
connection.zone:                        --
connection.master:                      --
connection.slave-type:                  --
connection.autoconnect-slaves:          -1 (default)
connection.secondaries:                 --
connection.gateway-ping-timeout:        0
connection.metered:                     unknown
cat /etc/sysconfig/network-scripts/ifcfg-eth1 | grep BOOTPROTO
BOOTPROTO=dhcp

5. Change "Automatically Connect" Directive

By default all ethernet connection will connect automatically. You can disable it with the following command:

nmcli con mod "System eth1" connection.autoconnect no

You can verify the changes with the following command:

cat /etc/sysconfig/network-scripts/ifcfg-eth1 | grep ONBOOT

You should see the following output:

ONBOOT=no

6. Change Network Connection BOOTPROTO Directive

You can change the ethernet connection BOOTPROTO directive from static to DHCP using the following command:

nmcli con mod "System eth1" ipv4.method auto

You can now verify the changes with the following command:

cat /etc/sysconfig/network-scripts/ifcfg-eth1 | grep BOOTPROTO

You should see the following output:

BOOTPROTO=dhcp

To change the ethernet connection BOOTPROTO directive static to DHCP to static using the following command:

nmcli con mod "System eth1" ipv4.method manual ipv4.address 192.168.0.10/24 ipv4.gateway 192.168.0.1

7. Disable IPv6 Address with nmcli

By default, both IPv6 and IPv4 connection is enabled in CentOS 8. You can disable the IPv6 connection wiht the following command: Advertisements

nmcli con mod "System eth1" ipv6.method ignore

8. Add DNS Server to Existing Connection

To add a new DNS server to an existing connection with the following command:

nmcli con mod "System eth1" ipv4.dns 8.8.4.4

You can verify the changes with the following command:

cat /etc/sysconfig/network-scripts/ifcfg-eth1 | grep DNS

Output:

DNS1=8.8.4.4

You can also append a new DNS server using the +ipv4.dns option:

nmcli con mod "System eth1" +ipv4.dns 4.4.4.4

9. Remove DNS Server from Existing Connection

To remove the single DNS server from the connection, run the following command: Advertisements

nmcli con mod "System eth1" -ipv4.dns 8.8.4.4

To remove the multiple DNS servers from the connection, run the following command:

nmcli con mod "System eth1" -ipv4.dns 8.8.4.4,8.8.2.2

10. Add/Edit Connection Interactively

You can also create a new connection or edit an existing connection using an interactive editor.

For example, edit an existing connection, run the following command:

nmcli con edit "System eth1"

You should see the following output:

===| nmcli interactive connection editor |===

Editing existing '802-3-ethernet' connection: 'System eth1'

Type 'help' or '?' for available commands.
Type 'print' to show all the connection properties.
Type 'describe [.]' for detailed property description.

You may edit the following settings: connection, 802-3-ethernet (ethernet), 802-1x, dcb, sriov, ethtool, match, ipv4, ipv6, tc, proxy
nmcli>

Now, display an existing IP address, run the following command:

nmcli> print ipv4.address

Output:

ipv4.addresses: 192.168.0.10/32

To set a new IP address, run the following command:

nmcli> set ipv4.address 192.168.0.11

You can verify and save the connection with the following command:

nmcli> verify

Output:

Verify connection: OK

nmcli> save

Output: Advertisements

Connection 'System eth1' (9c92fad9-6ecb-3e6c-eb4d-8a47c6f50c04) successfully updated.

You can now verify the saved connection with the following command:

cat /etc/sysconfig/network-scripts/ifcfg-eth1 | grep IPADDR

You should see the following output:

IPADDR=192.168.0.10
IPADDR1=192.168.0.11

11. Monitor Connection Activity

You can also monitor NetworkManager activity using nmcli like, changes in connection state, profiles, devices, etc.

After modifying the ethernet connection, run the following command to monitor it:

nmcli con monitor "System eth1"

12. Create a New Connection with Static IP

You can also create a new static ethernet connection with nmcli. For example, create a new ethernet connection named eth2, IP 192.168.0.12/24, Gateway 192.168.0.1, "onboot=yes" by running the following command:

nmcli con add con-name eth2 type ethernet ifname eth2 ipv4.method manual ipv4.address 192.168.0.15/24 ipv4.gateway 192.168.0.1

You should see the following output:

Connection 'eth2' (cefb3f7d-424c-42f8-b4e8-ed54e7dcb880) successfully added.

Now, verify the connection with the following command:

nmcli con

Output:

NAME         UUID                                  TYPE      DEVICE 
System eth0  5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03  ethernet  eth0   
System eth1  9c92fad9-6ecb-3e6c-eb4d-8a47c6f50c04  ethernet  eth1   
eth2         cefb3f7d-424c-42f8-b4e8-ed54e7dcb880  ethernet  eth2

13. Create a New Connection with DHCP

You can also create a new DHCP connection with nmcli. For example, create a new DHCP ethernet connection named eth3 with the following command:

nmcli con add con-name eth3 type ethernet ifname eth3 ipv4.method auto

You should see the following output:

Connection 'eth3' (ff54dbd6-255d-4935-abc8-73773bef5b55) successfully added.

14. Activate a New Connection

To activate the new ethernet connection eth2, run the following command:

nmcli con up eth2

You should see the following output:

Connection successfully activated

You can now verify the active connection with the following command:

nmcli con show --active

You should see the following output:

Output:

NAME         UUID                                  TYPE      DEVICE 
System eth0  5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03  ethernet  eth0   
System eth1  9c92fad9-6ecb-3e6c-eb4d-8a47c6f50c04  ethernet  eth1   
eth2         cefb3f7d-424c-42f8-b4e8-ed54e7dcb880  ethernet  eth2

15. Deactivate a Connection

To deactivate the connection eth2, run the following command:

nmcli con down eth2

16. Delete a Connection

You can also delete a specific ethernet connection with nmcli.

For example, to delete a connection eth2, run the following command:

nmcli con del eth2

You should see the following output:

Connection 'eth2' (cefb3f7d-424c-42f8-b4e8-ed54e7dcb880) successfully deleted.

17. Change Hostname with nmcli

To find the current hostname of your system, run the following command:

nmcli general hostname

You should see the following output:

centos8

Next, change the hostname from centos8 to linux using the following command: Advertisements

nmcli general hostname linux

Next, verify the hostname with the following command:

nmcli general hostname

You should see the following output:

linux

18. Change the DEFROUTE Directive

The DEFROUTE directive is used to disable and enable the default gateway of your ethernet connection.

To enable the DEFROUTE directove for eth2 run the following command:

nmcli con mod "System eth2" ipv4.never-default yes

19. Restart Ethernet Connection

You can restart or reload your ethernet connection with the following command:

nmcli con reload

20. nmcli help

To get more information about nmcli command, run the following command:

nmcli --help

You should see the following output:

Usage: nmcli [OPTIONS] OBJECT { COMMAND | help }

OPTIONS
  -a, --ask                                ask for missing parameters
  -c, --colors auto|yes|no                 whether to use colors in output
  -e, --escape yes|no                      escape columns separators in values
  -f, --fields <field,...>|all|common      specify fields to output
  -g, --get-values <field,...>|all|common  shortcut for -m tabular -t -f
  -h, --help                               print this help
  -m, --mode tabular|multiline             output mode
  -o, --overview                           overview mode
  -p, --pretty                             pretty output
  -s, --show-secrets                       allow displaying passwords
  -t, --terse                              terse output
  -v, --version                            show program version
  -w, --wait                      set timeout waiting for finishing operations

OBJECT
  g[eneral]       NetworkManager's general status and operations
  n[etworking]    overall networking control
  r[adio]         NetworkManager radio switches
  c[onnection]    NetworkManager's connections
  d[evice]        devices managed by NetworkManager
  a[gent]         NetworkManager secret agent or polkit agent
  m[onitor]       monitor NetworkManager changes

Conclusion

In the above guide, we learned how to use nmcli to manage and control ethernet connection in CentOS 8. I hope you can now easily add, edit or create a new connection with nmcli. Feel free to ask me if you have any questions.

About Hitesh Jethva

Over 8 years of experience as a Linux system administrator. My skills include a depth knowledge of Redhat/Centos, Ubuntu Nginx and Apache, Mysql, Subversion, Linux, Ubuntu, web hosting, web server, Squid proxy, NFS, FTP, DNS, Samba, LDAP, OpenVPN, Haproxy, Amazon web services, WHMCS, OpenStack Cloud, Postfix Mail Server, Security etc.

[Jul 26, 2019] How to check open ports in Linux using the CLI> by Vivek Gite

Jul 26, 2019 | www.cyberciti.biz

Using netstat to list open ports

Type the following netstat command
sudo netstat -tulpn | grep LISTEN

... ... ...

For example, TCP port 631 opened by cupsd process and cupsd only listing on the loopback address (127.0.0.1). Similarly, TCP port 22 opened by sshd process and sshd listing on all IP address for ssh connections:

Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode      PID/Program name 
tcp   0      0      127.0.0.1:631           0.0.0.0:*               LISTEN      0          43385      1821/cupsd  
tcp   0      0      0.0.0.0:22              0.0.0.0:*               LISTEN      0          44064      1823/sshd

Where,

• -t : All TCP ports
• -u : All UDP ports
• -l : Display listening server sockets
• -p : Show the PID and name of the program to which each socket belongs
• -n : Don't resolve names
• | grep LISTEN : Only display open ports by applying grep command filter.

Use ss to list open ports

The ss command is used to dump socket statistics. It allows showing information similar to netstat. It can display more TCP and state information than other tools. The syntax is:
sudo ss -tulpn

... ... ...

Vivek Gite is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.

[Jan 30, 2019] TCP - IP communication of RHEL 7 is extremely slow compared with RHEL 6 and earlier. - Red Hat Customer Portal

Jan 30, 2019 | access.redhat.com

TCP / IP communication of RHEL 7 is extremely slow compared with RHEL 6 and earlier.
My application transferd from RHEL 5 to RHEL 7.
That application makes TCP / IP communication with another application on the same server.
That communication speed is about twice slower.

Apart from that, we compared it with "ping localhost". RHEL 7 averaged 0.04 ms, RHEL 6 and RHEL 5 average 0.02 ms. RHEL 7 is twice as slow as RHEL 6 or earlier.

The environment is the minimum installation, stop firewalld and postfix, then do "ping localhost".

Why was communication delayed like this?
Or, what is going on late?
Is not it worth it?

RED HAT GURU 7422 Points

---

24 January 2019 11:09 PM Jamie Bainbridge

Guesses: differences in process scheduling, memory fragmentation, other CPU workload, timing inaccuracy, incorrect test method, firewall behaviour, system performance differences, code difference like the security vulnerability mentioned above, probably much more that I have not thought of.

A good troubleshooting path forward is to identify:

• the specific behaviour in your application which is different
• what you expect performance to be
• what performance measurement you are currently getting

And then look into possible causes. I would start with perf collection during an application run, and possibly strace of the application although that can negatively affect performance too.

There are some more questions to give ideas at Initial investigation for any performance issue .

I see you have "L3 support" through your hardware vendor, possibly you bought RHEL pre-installed on your system, so the hardware vendor's tech support would be the first place to ask. The vendor will contact us if they identify a bug in RHEL.

25 January 2019 12:18 AM R. Hinton. Community Leader

One side note, make sure you really have your dns resolver /etc/resolv.conf set properly. The suggestions above are of course indeed good, but if your dns is not set properly, you'll have another round of slowness. Remember that /etc/resolv.conf is populated generally from the "DNSx" and "DOMAIN" directives found in the active '/etc/sysconfig/netowrk-scripts/ifcfg-XYZ" file. You can find what interfaces are actually active by using ip -o -4 a s which will reveal all IPV4 active interfaces with the interface name in the results at the very far left.

There are instances where if you have a system that is doing a lot of actions that rely on dns, you could make a dns caching server at your location that would assist with lookups and cache relevant things for your system.

Again, the other answers above are very useful, on spot, but if your /etc/resolv.conf is off, or not optimal, it could cause issue.

Another thing to review, and yes, it is exhaustive, the Red Hat tuning guide would be a good reference to double-check.

One method to test network bandwidth and latency performance is here .

I have not fully vetted this article where someone did some additional tuning and it would be good to validate what is in that article for legitimacy, and make backups of any configurations before making changes.

One last thing, using the rpm iftop can give you an idea of what systems are hitting your server, or visa versa.

Regards,

RJ

25 January 2019 2:05 AM Jamie Bainbridge

and yes, it is exhaustive, the Red Hat tuning guide

For reference, the Network Performance Tuning Guide PDF is only the original publish. We have updated the knowledgebase article a couple of times since then:

• https://access.redhat.com/articles/1391433

I have not fully vetted this article

Using tuned is a good idea... if the tuning profile matches your use case. Users are encouraged to think of the shipped profiles as just a starting point and develop their own more customised tuning profile.

An overview of the default profiles is at: https://access.redhat.com/solutions/369093

[Oct 27, 2018] Linux Kernel /etc/sysctl.conf Security Hardening

Oct 23, 2018 | www.cyberciti.biz

... ... ...

sysctl is an interface that allows you to make changes to a running Linux kernel. With /etc/sysctl.conf you can configure various Linux networking and system settings such as:

1. Limit network-transmitted configuration for IPv4
2. Limit network-transmitted configuration for IPv6
3. Turn on execshield protection
4. Prevent against the common 'syn flood attack'
5. Turn on source IP address verification
6. Prevents a cracker from using a spoofing attack against the IP address of the server.
7. Logs several types of suspicious packets, such as spoofed packets, source-routed packets, and redirects.

Linux Kernel /etc/sysctl.conf Security Hardening with sysctl

The sysctl command is used to modify kernel parameters at runtime. /etc/sysctl.conf is a text file containing sysctl values to be read in and set by sysct at boot time. To view current values, enter:
# sysctl -a
# sysctl -A
# sysctl mib
# sysctl net.ipv4.conf.all.rp_filter
# sysctl -a --pattern 'net.ipv4.conf.(eth|wlan)0.arp'
To load settings, enter:
# sysctl -p

Sample /etc/sysctl.conf for Linux server hardening

Edit /etc/sysctl.conf or /etc/sysctl.d/99-custom.conf and update it as follows. The file is documented with comments. However, I recommend reading the official Linux kernel sysctl tuning help file (see below):

# The following is suitable for dedicated web server, mail, ftp server etc. 
# ---------------------------------------
# BOOLEAN Values:
# a) 0 (zero) - disabled / no / false
# b) Non zero - enabled / yes / true
# --------------------------------------
# Controls IP packet forwarding
net.ipv4.ip_forward = 0
 
# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0
 
# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0
 
# Controls whether core dumps will append the PID to the core filename
# Useful for debugging multi-threaded applications
kernel.core_uses_pid = 1
 
# Controls the use of TCP syncookies
# Turn on SYN-flood protections
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_synack_retries = 5
 
########## IPv4 networking start ##############
# Send redirects, if router, but this is just server
# So no routing allowed 
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
 
# Accept packets with SRR option? No
net.ipv4.conf.all.accept_source_route = 0
 
# Accept Redirects? No, this is not router
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.secure_redirects = 0
 
# Log packets with impossible addresses to kernel log? yes
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.default.secure_redirects = 0
 
# Ignore all ICMP ECHO and TIMESTAMP requests sent to it via broadcast/multicast
net.ipv4.icmp_echo_ignore_broadcasts = 1
 
# Prevent against the common 'syn flood attack'
net.ipv4.tcp_syncookies = 1
 
# Enable source validation by reversed path, as specified in RFC1812
net.ipv4.conf.all.rp_filter = 1
 
# Controls source route verification
net.ipv4.conf.default.rp_filter = 1 
 
########## IPv6 networking start ##############
# Number of Router Solicitations to send until assuming no routers are present.
# This is host and not router
net.ipv6.conf.default.router_solicitations = 0
 
# Accept Router Preference in RA?
net.ipv6.conf.default.accept_ra_rtr_pref = 0
 
# Learn Prefix Information in Router Advertisement
net.ipv6.conf.default.accept_ra_pinfo = 0
 
# Setting controls whether the system will accept Hop Limit settings from a router advertisement
net.ipv6.conf.default.accept_ra_defrtr = 0
 
#router advertisements can cause the system to assign a global unicast address to an interface
net.ipv6.conf.default.autoconf = 0
 
#how many neighbor solicitations to send out per address?
net.ipv6.conf.default.dad_transmits = 0
 
# How many global unicast IPv6 addresses can be assigned to each interface?
net.ipv6.conf.default.max_addresses = 1
 
########## IPv6 networking ends ##############
 
#Enable ExecShield protection
#Set value to 1 or 2 (recommended) 
#kernel.exec-shield = 2
#kernel.randomize_va_space=2
 
# TCP and memory optimization 
# increase TCP max buffer size setable using setsockopt()
#net.ipv4.tcp_rmem = 4096 87380 8388608
#net.ipv4.tcp_wmem = 4096 87380 8388608
 
# increase Linux auto tuning TCP buffer limits
#net.core.rmem_max = 8388608
#net.core.wmem_max = 8388608
#net.core.netdev_max_backlog = 5000
#net.ipv4.tcp_window_scaling = 1
 
# increase system file descriptor limit    
fs.file-max = 65535
 
#Allow for more PIDs 
kernel.pid_max = 65536
 
#Increase system IP port limits
net.ipv4.ip_local_port_range = 2000 65000
 
# RFC 1337 fix
net.ipv4.tcp_rfc1337=1

Reboot the machine soon after a kernel panic

kernel.panic=10

Addresses of mmap base, heap, stack and VDSO page are randomized

kernel.randomize_va_space=2

Ignore bad ICMP errors

net.ipv4.icmp_ignore_bogus_error_responses=1

Protects against creating or following links under certain conditions

fs.protected_hardlinks=1
fs.protected_symlinks=1

How do I tune Linux VM subsystem?

• See FAQ: Linux Tuning The VM (memory) Subsystem

How do I tune Linux network stack?

• See FAQ: Linux Tune Network Stack (Buffers Size) To Increase Networking Performance

Other Linux security tips

• Linux Server Hardening Security Tips

[Nov 30, 2010] Life As A Sys Admin Best Networking Tweaks for Linux By Michael Adams

Nov 29, 2010 | Network World

A Linux system can be tweaked to a degree Windows users may envy (or fear) especially for networking. Tweaking a Linux box for networking is a bit more mundane than other platforms: there are specific driver settings one can work with but its best flexibility comes from a mix of OS-level modifications and adherence to different RFCs.

ifconfig (interface) txqueuelen #

Software buffers for network adapters on Linux start off at a conservative 1000 packets. Network researchers and scientists have mucked around with this, and figured out that we should be using 10,000 for anything decent on a LAN; more if you're running GB or 10GE stuff. Slow interfaces, such as modems and WAN links, can default to 0-100, but don't be afraid to bump it up towards 1000 and see if your performance improves. Bumping up this setting does use memory, so be careful if you're using an embedded router or something (I've used 10,000 on 16MB RAM OpenWRT units, no prob).

You can edit /etc/rc.local, add an "up" command to /etc/networking/interfaces, or whatever your distribution suggests and it's best to put a command like this at startup.

/etc/sysctl.conf

This file governs default behavior for many network and file operation settings on Linux and other *nix-based systems. If you deploy Ubuntu or Fedora systems, you'll notice they will add their own tweaks (usually security or file-oriented) to the file: don't delete those, unless you read up on them, or see any that are contradicted by the suggested additions here...

net.ipv4.tcp_rfc1337=1
net.ipv4.tcp_window_scaling=1
net.ipv4.tcp_workaround_signed_windows=1
net.ipv4.tcp_sack=1
net.ipv4.tcp_fack=1
net.ipv4.tcp_low_latency=1
net.ipv4.ip_no_pmtu_disc=0
net.ipv4.tcp_mtu_probing=1
net.ipv4.tcp_frto=2
net.ipv4.tcp_frto_response=2
net.ipv4.tcp_congestion_control=illinois

1. RFC 1337, TIME-WAIT Assassination Hazards in TCP, a fix written in 1992 for some theoretically-possible failure modes for TCP connections. To this day this RFC still has people confused if it negatively impacts performance or not or is supported by any decent router. Murphy's Law is that the only router that it would even have trouble with, is most likely your own.

2. TCP window scaling tries to avoid getting the network adapter saturated with incoming packets.

3. TCP SACK and FACK refer to options found in RFC 2018 and are also documented back to Linux Kernel 2.6.17 with an experimental "TCP-Peach" set of functions. These are meant to get you your data without excessive losses.

4. The latency setting is 1 if you prefer more packets vs bandwidth, or 0 if you prefer bandwidth. More packets are ideal for things like Remote Desktop and VOIP: less for bulk downloading.

5. I found RFC 2923, which is a good review of PMTU. IPv6 uses PMTU by default to avoid segmenting packets at the router level, but its optional for IPv4. PMTU is meant to inform routers of the best packet sizes to use between links, but its a common admin practice to block ICMP ports that allow pinging, thus breaking this mechanism. Linux tries to use it, and so do I: if you have problems, you have a problem router, and can change the "no" setting to 1. "MTU probing" is also a part of this: 1 means try, and 0 means don't.

6. FRTO is a mechanism in newer Linux kernels to optimize for wireless hosts: use it if you have them; delete the setting, or set to 0, if you don't.

For further study, there's a great IBM article regarding network optimizations: it was my source for some of these settings, as well as following numerous articles on tweaking Linux networking over the years (SpeedGuide has one from 2003).

TCP Congestion Controls

Windows Vista and newer gained Compound TCP as an alternative to standard TCP Reno. Linux Kernel 2.6 has had numerous mechanisms available to it for some time: 2.6.19 defaulted to CUBIC which was supposed to work well over "long links." My two personal favorites: TCP Westwood + and TCP Illinois. But you can dig in, look at different research papers online, and see what works best for your environment.

1. Make sure your kernel has the correct module: in my example, I use TCP Illinois, which has been compiled with any standard Ubuntu kernel since 2008, and is found as tcp_illinois.

2. Add said kernel module to /etc/modules

3. Change /etc/sysctl.conf to use the non "tcp_" part of your selection.

There you have it -- some of my favorite Linux tweaks for networking. I'm interested in hearing how these worked for you. If you have some of your own, please post a comment and share them with other readers.

IBM Redbooks Linux Performance and Tuning Guidelines

Abstract

Over the past few years, Linux has made its way into the data centers of many corporations all over the globe. The Linux operating system has become accepted by both the scientific and enterprise user population. Today, Linux is by far the most versatile operating system. You can find Linux on embedded devices such as firewalls and cell phones and mainframes. Naturally, performance of the Linux operating system has become a hot topic for both scientific and enterprise users. However, calculating a global weather forecast and hosting a database impose different requirements on the operating system. Linux has to accommodate all possible usage scenarios with the most optimal performance. The consequence of this challenge is that most Linux distributions contain general tuning parameters to accommodate all users.

IBM® has embraced Linux, and it is recognized as an operating system suitable for enterprise-level applications running on IBM systems. Most enterprise applications are now available on Linux, including file and print servers, database servers, Web servers, and collaboration and mail servers.

With use of Linux in an enterprise-class server comes the need to monitor performance and, when necessary, tune the server to remove bottlenecks that affect users. This IBM Redpaper describes the methods you can use to tune Linux, tools that you can use to monitor and analyze server performance, and key tuning parameters for specific server applications. The purpose of this redpaper is to understand, analyze, and tune the Linux operating system to yield superior performance for any type of application you plan to run on these systems.

The tuning parameters, benchmark results, and monitoring tools used in our test environment were executed on Red Hat and Novell SUSE Linux kernel 2.6 systems running on IBM System x servers and IBM System z servers. However, the information in this redpaper should be helpful for all Linux hardware platforms.

Update 4/2008: Typos corrected

[Feb 25, 2009] How to troubleshoot RHEL performance bottlenecks by Ken Milberg

09.30.2008

You've just had your first cup of coffee and have received that dreaded phone call. The system is slow. What are you going to do? This article will discuss performance bottlenecks and optimization in Red Hat Enterprise Linux (RHEL5).

Before getting into any monitoring or tuning specifics, you should always use some kind of tuning methodology. This is one which I've used successfully through the years:

1. Baseline – The first thing you must do is establish a baseline, which is a snapshot of how the system appears when it's performing well. This baseline should not only compile data, but also document your system's configuration (RAM, CPU and I/O). This is necessary because you need to know what a well-performing system looks like prior to fixing it.

2. Stress testing and monitoring – This is the part where you monitor and stress your systems at peak workloads. It's the monitoring which is key here – as you cannot effectively tune anything without some historic trending data.

3. Bottleneck identification – This is where you come up with the diagnosis for what is ailing your system. The primary objective of section 2 is to determine the bottleneck. I like to use several monitoring tools here. This allows me to cross-reference my data for accuracy.

4. Tune – Only after you've identified the bottleneck can you tune it.

5. Repeat – Once you've tuned it, you can start the cycle again – but this time start from step 2 (monitoring) – as you already have your baseline.

It's important to note that you should only make one change at a time. Otherwise, you'll never know exactly what impacted any changes which might have occurred. It is only by repeating your tests and consistently monitoring your systems that you can determine if your tuning is making an impact.

RHEL monitoring tools

Before we can begin to improve the performance of our system, we need to use the monitoring tools available to us to baseline. Here are some monitoring tools you should consider using:

Oprofile

This tool (made available in RHEL5) utilizes the processor to retrieve kernel system information about system executables. It allows one to collect samples of performance data every time a counter detects an interrupt. I like the tool also because it carries little overhead – which is very important because you don't want monitoring tools to be causing system bottlenecks. One important limitation is that the tool is very much geared towards finding problems with CPU limited processes. It does not identify processes which are sleeping or waiting on I/O.

The steps used to start up Oprofile include setting up the profiler, starting it and then dumping the data.

First we'll set up the profile. This option assumes that one wants to monitor the kernel.

# opcontrol --setup –vmlinux=/usr/lib/debug/lib/modules/'uname -r'/vmlinux

Then we can start it up.

# opcontrol --start

Finally, we'll dump the data.

# opcontrol --stop/--shutdown/--dump

SystemTap

This tool (introduced in RHEL5) collects data by analyzing the running kernel. It really helps one come up with a correct diagnosis of a performance problem and is tailor-made for developers. SystemTap eliminates the need for the developer to go through the recompile and reinstallation process to collect data.

Frysk

This is another tool which was introduced by Red Hat in RHEL5. What does it do for you? It allows both developers and system administrators to monitor running processes and threads. Frysk differs from Oprofile in that it uses 100% reliable information (similar to SystemTap) - not just a sampling of data. It also runs in user mode and does not require kernel modules or elevated privileges. Allowing one to stop or start running threads or processes is also a very useful feature.

Some more general Linux tools include top and vmstat. While these are considered more basic, often I find them much more useful than more complex tools. Certainly they are easier to use and can help provide information in a much quicker fashion.

Top provides a quick snapshot of what is going on in your system – in a friendly character-based display.

It also provides information on CPU, Memory and Swap Space.

Let's look at vmstat – one of the oldest but more important Unix/Linux tools ever created. Vmstat allows one to get a valuable snapshot of process, memory, sway I/O and overall CPU utilization.

Now let's define some of the fields:

Memory
swpd – The amount of virtual memory
free – The amount of free memory
buff – Amount of memory used for buffers
cache – Amount of memory used as page cache

Process
r – number of run-able processes
b – number or processes sleeping. Make sure this number does not exceed the amount of run-able processes, because when this condition occurs it usually signifies that there are performance problems.

Swap
si – the amount of memory swapped in from disk
so – the amount of memory swapped out.
This is another important field you should be monitoring – if you are swapping out data, you will likely be having performance problems with virtual memory.

CPU
us – The % of time spent in user-level code.
It is preferable for you to have processes which spend more time in user code rather than system code. Time spent in system level code usually means that the process is tied up in the kernel rather than processing real data.
sy – the time spent in system level code
id – the amount of time the CPU is idle wa – The amount of time the system is spending waiting for I/O. If your system is waiting on I/O – everything tends to come to a halt. I start to get worried when this is > 10. There is also:

Free – This tool provides memory information, giving you data around the total amount of free and used physical and swap memory.

Now that we've analyzed our systems – lets look at what we can do to optimize and tune our systems.

CPU Overhead – Shutting Running Processes
Linux starts up all sorts of processes which are usually not required. This includes processes such as autofs, cups, xfs, nfslock and sendmail. As a general rule, shut down anything that isn't explicitly required. How do you do this? The best method is to use the chkconfig command.

Here's how we can shut these processes down.
[root ((Content component not found.)) _29_140_234 ~]# chkconfig --del xfs

You can also use the GUI - /usr/bin/system-config-services to shut down daemon process.

Tuning the kernel
To tune your kernel for optimal performance, start with:

sysctl – This is the command we use for changing kernel parameters. The parameters themselves are found in /proc/sys/kernel

Let's change some of the parameters. We'll start with the msgmax parameter. This parameter specifies the maximum allowable size of a single message in an IPC message queue. Let's view how it currently looks.

[root ((Content component not found.)) _29_139_52 ~]# sysctl kernel.msgmax
kernel.msgmax = 65536
[root ((Content component not found.)) _29_139_52 ~]#

There are three ways to make these kinds of kernel changes. One way is to change this using the echo command.

[root ((Content component not found.)) _29_139_52 ~]# echo 131072 >/proc/sys/kernel/msgmax
[root ((Content component not found.)) _29_139_52 ~]# sysctl kernel.msgmax
kernel.msgmax = 131072
[root ((Content component not found.)) _29_139_52 ~]#

Another parameter that is changed quite frequently is SHMMAX, which is used to define the maximum size (in bytes) for a shared memory segment. In Oracle this should be set large enough for the largest SGA size. Let's look at the default parameter:

# sysctl kernel.shmmax
kernel.shmmax = 268435456

This is in bytes – which translates to 256 MG. Let's change this to 512 MG, using the -w flag.

[root ((Content component not found.)) _29_139_52 ~]# sysctl -w kernel.shmmax=5368709132
kernel.shmmax = 5368709132
[root ((Content component not found.)) _29_139_52 ~]#

The final method for making changes is to use a text editor such as vi – directly editing the /etc/sysctl.conf file to manually make our changes.

To allow the parameter to take affect dynamically without a reboot, issue the sysctl command with the -p parameter.

Obviously, there is more to performance tuning and optimization than we can discuss in the context of this small article – entire books have been written on Linux performance tuning. For those of you first getting your hands dirty with tuning, I suggest you tread lightly and spend time working on development, test and/or sandbox environments prior to deploying any changes into production. Ensure that you monitor the effects of any changes that you make immediately; it's imperative to know the effect of your change. Be prepared for the possibility that fixing your bottleneck has created another one. This is actually not a bad thing in itself, as long as your overall performance has improved and you understand fully what is happening.

Performance monitoring and tuning is a dynamic process which does not stop after you have fixed a problem. All you've done is established a new baseline. Don't rest on your laurels, and understand that performance monitoring must be a routine part of your role as a systems administrator.

About the author: Ken Milberg is a systems consultant with two decades of experience working with Unix and Linux systems. He is a SearchEnterpriseLinux.com Ask the Experts advisor and columnist.

[Feb 23, 2009] Deployment_Guide/Gathering System Information

Before you learn how to configure your system, you should learn how to gather essential system> information. For example, you should know how to find the amount of free memory, the amount of available hard drive space, how your hard drive is partitioned, and what processes are running. This chapter discusses how to retrieve this type of information from your Red Hat Enterprise Linux system using simple commands and a few simple programs.

1. System Processes

The ps ax command displays a list of current system processes, including processes owned by other users. To display the owner alongside each process, use the ps aux command. This list is a static list; in other words, it is a snapshot of what was running when you invoked the command. If you want a constantly updated list of running processes, use top as described below. The ps output can be long. To prevent it from scrolling off the screen, you can pipe it through less:

ps aux | less

You can use the ps command in combination with the grep command to see if a process is running. For example, to determine if Emacs is running, use the following command:

ps ax | grep emacs

The top command displays currently running processes and important information about them including their memory and CPU usage. The list is both real-time and interactive. An example of output from the top command is provided as follows:

To exit top press the q key. Useful interactive commands that you can use:

• Space Immediately refresh the display

• h Display a help screen

• k Kill a process. You are prompted for the

• process ID and the signal to send to it.

• n Change the number of processes displayed.

• You are prompted to enter the number.

• u Sort by user.

• M Sort by memory usage.

• P Sort by CPU usage.

For more information, refer to the top(1) manual page.

Recommended Links

Linux Network Administrator's Guide, 2nd Edition June 2000 By Olaf Kirch & Terry Dawson

IBM Redbooks Linux Performance and Tuning Guidelines (June 05, 2007)

Over the past few years, Linux has made its way into the data centers of many corporations all over the globe. The Linux operating system has become accepted by both the scientific and enterprise user population. Today, Linux is by far the most versatile operating system. You can find Linux on embedded devices such as firewalls and cell phones and mainframes. Naturally, performance of the Linux operating system has become a hot topic for both scientific and enterprise users. However, calculating a global weather forecast and hosting a database impose different requirements on the operating system. Linux has to accommodate all possible usage scenarios with the most optimal performance. The consequence of this challenge is that most Linux distributions contain general tuning parameters to accommodate all users.

IBM® has embraced Linux, and it is recognized as an operating system suitable for enterprise-level applications running on IBM systems. Most enterprise applications are now available on Linux, including file and print servers, database servers, Web servers, and collaboration and mail servers.

With use of Linux in an enterprise-class server comes the need to monitor performance and, when necessary, tune the server to remove bottlenecks that affect users. This IBM Redpaper describes the methods you can use to tune Linux, tools that you can use to monitor and analyze server performance, and key tuning parameters for specific server applications. The purpose of this redpaper is to understand, analyze, and tune the Linux operating system to yield superior performance for any type of application you plan to run on these systems.

The tuning parameters, benchmark results, and monitoring tools used in our test environment were executed on Red Hat and Novell SUSE Linux kernel 2.6 systems running on IBM System x servers and IBM System z servers. However, the information in this redpaper should be helpful for all Linux hardware platforms. >

dkftpbench

http://www.kegel.com/

Check out the "c10k problem" page in particular, but the entire site has _lots_ of useful tuning info.

http://linuxperf.nl.linux.org/

Site organized by Rik Van Riel and a few other folks. Probabaly the best linux specific system tuning page.

http://www.citi.umich.edu/projects/citi-netscape/

Linux Scalibity Project at Umich.

NFS Performance Tunging

Info on tuning linux kernel NFS in particular, and linux network and disk io in general

http://home.att.net/~jageorge/performance.html

Linux Performance Tuning Checklist. Some useful content.

http://www.linux.com/tuneup/

Miscelaneous performace tuning tips at linux.com

http://www.psc.edu/networking/perf_tune.html#Linux

Summary of tcp tuning info

Some simple utilities that come in handy when doing performance tuning.

dkftpbench

Need to stress out an ftp server, or measure how many users it can support? dkftpbench can do it.

Want to write your own highly efficient networking software, but annoyed by having to support very different code for Linux, FreeBSD, and Solaris? libPoller can help.

dklimits

a simple util to check the actually number of file descriptors available, ephemeral ports available, and poll()-able sockets. Handy. Be warned that it can take a while to run if there are a large number of fd's available, as it will try to open that many files, and then unlinkt them.

This is part of the dkftpbench package.

fd-limit

a tiny util for determining the number of file descriptors available.

fd-limit.c

thread-limit

A util for determining the number of pthreads a system can use. This and fd-count are both from the system tuning page for Volano chat, a multithread java based chat server.

thread-limit.c

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: June 13, 2020