Problem:

My PuTTY connection becomes inactive, thereby requiring me to reconnect to the server and login again.

Background & Analysis:

How to identify an inactive PuTTY connection? When you cannot type anything in your PuTTY window and you see "(inactive)" in your PuTTY window’s title bar, it means that your connection is inactive (has been dropped by the server).

Why does a PuTTY connection become inactive? Connections use resources like cpu and memory on the client and server. When a server determines that your session is idle (you have not used it for a pre-defined interval), it will drop your connection. This is done to protect the server (and network components) against a build-up of several redundant connections that could affect performance.

Solution:

In order to prevent the server from dropping your PuTTY connection and making it inactive, your PuTTY client must send packets to the server even when you’re not using it. Given below are steps to do this:

STEP 1: Modify Connection options

Go to PuTTY Configuration –> Connection and set the "seconds between keepalives" to a non-zero value. This ensures that your PuTTY client sends NULL packets to the server regularly. If you do not know the idle session timeout defined on your server, then choose a small value for the "seconds between keepalives". The value you select must be less than the idle session timeout of your server for PuTTY to keep your connection active. The default value is 0 which tells PuTTY not to bother about keeping the connection alive. Refer the picture below:

STEP 2: Save your PuTTY configuration

Saving your PuTTY session is not the most obvious feature of PuTTY as there’s no SAVE button on every configuration window. So, after completing STEP 1, you must go to the Session window and click the "Save" button. refer the picture below:

Root Cause:

My PuTTy connection became inactive because my PuTTY client was not sending "keepalive" packets to the server to prevent the connection from becoming idle and consequently, the server dropped the connection.

NOTE:

(1) The solution above describes a successful problem-solving experience and may not be applicable to other problems with similar symptoms.

(2) Your rating of this post will be much appreciated. Also, feel free to leave comments.

Heard this song in my friend’s car and can understand why this song was written…we all have this “moment” wherein we wanna rant, don’t we? Check out the video below – country song with some amusing images.

Like this song, especially the refrain “I got soul but I’m not a soldier”

This song is a product of Glee, an American musical aired on Fox in the US.

Problem:

A Solaris 9 mountpoint was 100% utilized (as per “df”) and no new files could be added.

df output:

cybergavin@myhost:/dashboard> df –h /dashboard
Filesystem             size   used  avail capacity  Mounted on
/dev/vx/dsk/A19278-S01-7uitx-dg/dashboard
                        16G    16G   2.1M   100%    /dashboard

du output:

cybergavin@myhost:/dashboard> du –sk /dashboard
1789259 /dashboard

Background & Analysis:

As you can see above, both “du” and “df” provide significantly different metrics for the utilization of /dashboard. The “df” output tells me that I have very little free space (~ 2.1 MB) whereas the “du” output indicates that I have around 14 GB free space.

Well, first and foremost, df and du intend to give you disk usage stats, but they do not work in the same way. Refer this article to understand the differences between df and du.

Secondly, the mountpoint /dashboard was mounted on a VxFS. The dmesg output showed the following:

Feb  1 09:29:00 myhost vxfs: [ID 702911 kern.notice] NOTICE: msgcnt 112748 mesg 001: V-2-1: vx_nospace -  /dev/vx/dsk/A19278-S01-7uitx-dg/dashboard file system full (1 block extent)

An explanation for the above (quite obvious) message is given in this Symantec article.

I found a runaway background process (iostat –x 2) running for the past 2 months. It was a process launched by a shell script. The shell script exited, but the process wasn’t killed. The process was redirecting its output to a file and that file was also deleted. Consequently, the process’ stdout file descriptor (1) was not closed and the process was still writing to stdout. This caused the space occupied by the stdout to be hidden. To determine how much space is actually being used by the process when writing to stdout, try the following command (<pid> = process id):

ls -l /proc/<pid>/fd/1

Solution:

Killed the runaway process and the mountpoint utilization dropped significantly to 14%. Further, df and du outputs correlated.

Root Cause:

A runaway process was consuming most of the disk space and this disk space consumption was “hidden” because the file to which the process’ stdout was being redirected, was deleted.

NOTE:

(1) The solution above describes a successful problem-solving experience and may not be applicable to other problems with similar symptoms.

(2) Your rating of this post will be much appreciated. Also, feel free to leave comments.