Ahoy there! This is my personal blog which I use as my memory extension and a medium to share stuff that could be useful to others.

Linux Archives

RHN registration Introspect error

Problem:

When trying to register a Linux client with an RHN Satellite server using the rhnreg_ks utility, the following error is displayed:

Introspect error: The name com.redhat.SubscriptionManager was not provided by any .service files

 

Background & Analysis:

  • The Linux client was on RHEL 5.8
  • The rhnreg_ks is a utility used to register a Linux client with RHN and is part of the rhn-setup package (rpm).
  • Even though the error was displayed, the Linux client was successfully registered with RHN.

 

Solution:

I obtained the following solution from this Red Hat Bug Listing. So, here’s what I implemented:

 

STEP 1: Determine location of rhnreg.py and back it up

 sudo grep 'com.redhat.SubscriptionManager' `rpm -ql rhn-client-tools rhn-setup` 

For my Linux client, rhnreg.py was located in /usr/share/rhn/up2date_client/

Take a backup of rhnreg.py

 

STEP 2: Modify rhnreg.py

Comment out the following:

#validity_obj = bus.get_object('com.redhat.SubscriptionManager', 
#'/EntitlementStatus')

Add the following just below the commented lines:

 validity_obj = bus.ProxyObjectClass(bus,'com.redhat.SubscriptionManager',
'/EntitlementStatus', introspect=False) 

NOTE: Since you are modifying Python code, ensure that you maintain the program indentation.

 

Root Cause:

The RHN registration client writes an exception to stderr. The functionality isn’t affected and so this change only has aesthetic value. Perhaps, that’s why I did not find an update in the RHN RHEL 5.8 repository.  Anyway, for those of you who find that exception annoying, you now know how to get rid of it.

 

(1) The solution above describes a successful problem-solving experience and may not be applicable to other problems with similar symptoms.

(2) Your rating of this post will be much appreciated as it gives me and others who read this article, an indication of whether this solution has worked for people other than me. Also, feel free to leave comments.

 

VN:F [1.9.22_1171]
Rating: +2 (from 2 votes)

Sendmail is slow

Problem:

The sendmail service takes a while (more than a minute) to start and emails sent via sendmail take a couple of minutes to get delivered:

 

Background & Analysis:

Sendmail uses DNS for the following:

  • At startup, to obtain the canonical name for the local host.
  • To obtain the canonical name of a remote host that connects to the local host.
  • To obtain the address of the the SMTP Relay to which sendmail connects.
  • When sendmail expands $[ and $] in the RHS of a rule.

Refer this article for more details.

Solution:

STEP 1: Ensure that the local host’s canonical name may be looked up by adding the following as the first entry in /etc/hosts:

127.0.0.1       localhost.localdomain <hostname>

where, <hostname> should be replaced by the local host’s hostname.

Restart the sendmail service as shown below:

 sudo service sendmail restart 

Implementation of this step enabled my sendmail process start up very quickly.

 

STEP 2: Ensure that the SMTP Mail Relay is configured to not look up DNS by surrounding the Mail Relay’s address by square brackets in /etc/mail/sendmail.cf as shown in the example below:

DS[10.1.1.10]

Restart the sendmail service as shown below:

 sudo service sendmail restart 

Implementation of this step enabled my sendmail process to deliver emails quickly.

 

Root Cause:

The sendmail process takes a while to do lookups in DNS.

 

(1) The solution above describes a successful problem-solving experience and may not be applicable to other problems with similar symptoms.

(2) Your rating of this post will be much appreciated as it gives me and others who read this article, an indication of whether this solution has worked for people other than me. Also, feel free to leave comments.

 

VN:F [1.9.22_1171]
Rating: +1 (from 1 vote)

Linux LVM Example

Requirement: Wipe away /old and add the space recovered (5 GB) to /new for the following disk layout:

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/VG01-LV01_app 5G 2.5G 2.5G 50% /old

/dev/mapper/VG02-LV02_app 5G 2.5G 2.5G 50% /new

  • /old is mounted on Logical Volume LV01 in Volume Group VG01
  • Volume VG01 comprises 1 Logical Volume /dev/mapper/VG01-LV01_app and1 Physical Volume /dev/sdb1
  • /new is mounted on Logical Volume LV02 in Volume Group VG02
  • System and other partitions (/, /boot, /home, etc.) are mounted on Logical Volumes in Volume Group VG00 (not considered in example).

Implementation: Here’s how I met the above requirement on RHEL 5.

STEP 1: Backup the partition to be extended

Backup /new before extending it with space recovered from wiping out /old.

 

STEP 2: Unmount the partitions

sudo umount /old
sudo umount /new

NOTE: You cannot unmount the partitions if they’re busy. You may use lsof to determine which files are using the partition and stop the associated services/processes.

 

STEP 3: Determine the number of extents used by the Logical Volume associated with /old

sudo lvdisplay /dev/mapper/VG01-LV01_app

 

STEP 4: Remove Volumes associated with /old

sudo lvremove /dev/mapper/VG01-LV01_app
sudo vgremove VG01
sudo pvremove /dev/sdb1

 

STEP 5: Extend Volumes associated with /new

sudo pvcreate /dev/sdb1
sudo vgextend VG02 /dev/sdb1
sudo lvextend -l+1599 /dev/mapper/VG02-LV02_app /dev/sdb1

Note that the above assumes that the number of extents associated with /dev/mapper/VG01-LV01_app is 1599 (determined in STEP 2).

 

STEP 6: Check and resize the filesystem

Now that the Volume Group and Logical Volume associated with /new has been extended, the filesystem for the Logical Volume must be checked and resized as follows:

sudo e2fsck -f /dev/mapper/VG02-LV02_app
sudo resize2fs /dev/mapper/VG02-LV02_app

 

STEP 7: Mount partitions

Mount the /new partition as follows:

sudo mount /new

You should see the following disk layout for /new:

Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VG02-LV02_app 10G 2.5G 7.5G 25% /new

VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Failure in yum update

Problem:

On RHEL 6 (Santiago), a “yum update” fails with the following error:

Error: failed to retrieve repodata/filelists.xml.gz from rhel-x86_64-server-6
       error was [Errno -1] Metadata file does not match checksum
You could try using –skip-broken to work around the problem
You could try running: rpm -Va –nofiles –nodigest

And trying the suggestions above did not help.

Background & Analysis:

Yum uses a cache directory to speed up its operations. Sometimes, the cache could become corrupted.

Solution:

1.  Clean the cache directory as shown below:

sudo yum clean all

2.  Rebuild the cache directory as shown below: 

sudo yum makecache

3.  Perform a yum update as shown below: 

sudo yum update

Root Cause:

Corrupted yum cache directory

(1) The solution above describes a successful problem-solving experience and may not be applicable to other problems with similar symptoms.

(2) Your rating of this post will be much appreciated as it gives me and others who read this article, an indication of whether this solution has worked for people other than me. Also, feel free to leave comments.

 

VN:F [1.9.22_1171]
Rating: +7 (from 7 votes)

Under normal working conditions, when SNMP Managers query SNMP agents (snmpd) on RHEL 6, several lines of information similar to the following are logged into syslog:

 May 27 17:39:14 MyLinuxHost snmpd[1521]: Connection from UDP: [192.168.100.200]:54907->[172.23.10.10] 

As snmpd is typically queried frequently, your syslog (e.g. /var/log/messages) will be filled with several such informational lines as this can lead to “noise” and is truly not required.

SNMP Logging Levels are given below:

LOG LEVEL DESCRIPTION
0 Emergencies – System is unusable
1 Alerts – Immediate action needed
2 Critical – Critical conditions
3 Errors – Error conditions
4 Warnings – Warning conditions
5 Notifications – Informational messages
6 Informational – Normal but significant conditions
7 Debugging – Debugging messages

By default, SNMP on RHEL 6 has logging levels 0-6 enabled. The redundant information in the logs is logged at level 6. Given below are steps to disable these informational messages for SNMP on RHEL 6:

STEP 1:Modify the SNMP Logging Level

Edit /etc/init.d/snmpd and modify the OPTIONS variable to reflect logging levels 0-5 as shown below:

 OPTIONS="-LS0-5d -Lf /dev/null -p /var/run/snmpd.pid" 

STEP 2:Restart the SNMP service

Restart the SNMP service for the changes to take effect:

 sudo service snmpd restart 
VN:F [1.9.22_1171]
Rating: +17 (from 23 votes)

Logrotate: High CPU utilization and failure

Problem:

The logrotate process caused high CPU utilization, ran for about 5-6 hours and finally exited with a non-zero (failure) exit code.

Background & Analysis:

Observed this problem on one of our RHEL 3 hosts. A new daily job was added and this job specified log rotation for all files in a directory (rather than a file or specific patterns to match files) and used the compress option. Consequently, logrotate compressed files that were already compressed and so on and files in the directory started having very long filenames (e.g. test.dmp.gz.gz.1.gz.1.gz…) and a huge number of files were present in the directory, making logrotate’s job more and more difficult, thereby causing logrotate to consume high CPU (system cpu utilization %) and take longer to complete. Also, logrotate’s state file (/var/lib/logrotate.status) ended up being about 58 MB.

Solution:

  • Modify the daily job to rotate only files matching a specific pattern (in my case, I removed the daily job as it was redundant. The files were created with timestamp prefixes and so rotation wasn’t required).
  • Remove logrotate’s state file (/var/lib/logrotate.status). When done, logrotate will generate a new state file.

NOTE: The removal of logrotate’s state file is akin to running logrotate for the first time. So, you can modify the dates in the state file to reflect the previous day or the previous week before running a daily or weekly job to ensure logrotate rotates the files on its first run. ALternatively, use the logrotate –f option.

Root Cause:

Improper configuration of logrotate for a daily job.

(1) The solution above describes a successful problem-solving experience and may not be applicable to other problems with similar symptoms.

(2) Your rating of this post will be much appreciated as it gives me and others who read this article, an indication of whether this solution has worked for people other than me. Also, feel free to leave comments.

 

VN:F [1.9.22_1171]
Rating: +2 (from 2 votes)

SSH Authentication is slow

Problem:

When trying to connect to a server via SSH, the authentication is slow (takes a few seconds), but the server seems normally responsive after authentication.

Background & Analysis:

I’ve experienced this problem when trying to connect to Linux hosts using SSH. Som of my Linux hosts use OpenSSH and some use Reflection for Secure IT Server.

When authenticating via SSH, the SSH server will perform a reverse domain lookup of the client’s IP address and this delays the authentication process.

SSH servers have options in their configurations to disable the reverse domain lookup of clients’ IP addresses.

Solution:

Disable reverse domain lookup of the client’s IP address in the SSH Server Configuration. Given below are examples of how this is done in two SSH servers:

OpenSSH: Edit /etc/ssh/sshd_config (or wherever your config file is) and set

UseDNS=no

RSIT: Edit /etc/ssh2/sshd2_config and set

ResolveClientHostname=no

Root Cause:

Slow Reverse Domain Lookup of clients’ IP addresses.

 

(1) The solution above describes a successful problem-solving experience and may not be applicable to other problems with similar symptoms.

(2) Your rating of this post will be much appreciated as it gives me and others who read this article, an indication of whether this solution has worked for people other than me. Also, feel free to leave comments.

 

VN:F [1.9.22_1171]
Rating: +5 (from 5 votes)

Unknown groups in SLES

Problem:

On SLES 11 SP 1, while trying to add users with the useradd command, the following is displayed:

useradd: Unknown group `video’.
useradd: Unknown group `dialout’.

Background & Analysis:

The “video” and “dialout” groups are default groups in SLES. This is configured in /etc/default/useradd.

These groups were removed from /etc/group using the groupadd command although the useradd command uses them as default groups.

Solution:

STEP 1: Modify /etc/default/useradd

Edit /etc/default/useradd and remove video and dialout from the GROUPS directive. After doing so, my /etc/default/useradd looks like this (I have removed the GROUPS directive as I do not want default groups):

GROUP=100
HOME=/home
INACTIVE=-1
EXPIRE=
SHELL=/bin/bash
SKEL=/etc/skel
CREATE_MAIL_SPOOL=yes

 

 

VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

How to Install PHP with FreeTDS on Linux

There are PHP applications which use MSSQL as the back-end database and such applications require FreeTDS to enable PHP code interface with MSSQL. This article describes how to install PHP and FreeTDS on Linux hosts.

To compile or not?: Typically, it is recommended to use package managers like yum to install software on Linux platforms. Using package managers facilitates installation and administration (e.g. updates) of the software. However, when the software requires special options to be set or modules/extensions to be enabled, it may be difficult to obtain software built to suit those requirements. In such cases, it will be required to compile the software from its source. If compiling, then it will be prudent to organize all compiled software in standard locations on the host.

Given below are the implementation steps that were used for installing PHP 5.3.3 and FreeTDS 0.91 on RHEL 6.2.

NOTE: All commands in the examples below must be executed with root privileges, unless otherwise stated.

STEP 1: Create Installation Directory Structure

  • Use the following commands to create an appropriate directory structure for compiled software:
mkdir /opt/src
  • Use a standard location (e.g. /opt) to install all software compiled from source to facilitate administration (re-compilation, removal, etc.)

STEP 2: Download and Unpack Software Source

  • Download software sources (typically *.tar.gz files) and place them in /opt/src
  • Unpack source software (*.tar.gz) as per the following examples:
tar xfz php-5.3.3.tar.gz
tar xfz freetds-0.91.tar.gz

The above commands will create directories /opt/src/php-5.3.3 and /opt/src/freetds-0.91

STEP 3: Compile and Build FreeTDS

Compile and build FreeTDS as per the example below:

cd /opt/src/freetds-0.91
./configure --prefix=/opt/freetds-0.91
make
make install

NOTE: In order to facilitate administration, you may create a soft link as follows:

cd /opt
ln -s freetds-0.91 freetds

STEP 4: Compile and Build PHP

Compile and build PHP as per the example below:

cd /opt/src/php-5.3.3
./configure --prefix=/opt/php-5.3.3 --with-config-file-path=/opt/php-5.3.3
make
make install

NOTE: In order to facilitate administration, you may create a soft link as follows:

cd /opt
ln -s php-5.3.3 php

STEP 5: Compile and Build the PHP Sybase Extension

PHP requires the sybase_ct extension to allow PHP code to interface with MSSQL. You may compile and build the sybase_ct extension as follows:

cd /opt/src/php-5.3.3/ext/sybase_ct
sh ../../scripts/phpize
./configure --prefix=/opt/php --with-php-config=/opt/php/bin/php-config --with-sybase-ct=/opt/freetds
make
make install

STEP 6: Enable the PHP Sybase Extension

Enable the sybase_ct extension by adding the following line to /opt/php/php.ini

extension=sybase_ct.so

STEP 7: Verify the PHP (with FreeTDS) Installation

If PHP and the sybase_ct extension have been successfully installed, you should be able to view the sybase_ct module when displaying the PHP configuration information as shown below:

Execute the following command (any user):

php -i | grep sybase_ct

If you see "sybase_ct" in the output, then it means that PHP and the sybase_ct extension have been successfully installed.

NOTE: Since PHP and FreeTDS have been compiled from source and installed in non-standard locations, you must add /opt/php/bin:/opt/freetds/bin to a user’s PATH environment variable.

VN:F [1.9.22_1171]
Rating: +2 (from 2 votes)

Unable to load module osad

Problem:

The osad service does not start (RHEL 5.7) and throws the following error:

Starting osad: Unable to load module osad

Background & Analysis:

The osad service is required on all Red Hat Network Satellite clients to receive pushed actions from the RHN Satellite Server. For more details on the osad and related services, please refer this Spacewalk (open-source version of RHN Satellite) article.

Solution:

The osad service depends on the python-hashlibs package.

STEP 1: Install python-hashlibs

Install python-hashlibs as follows:

sudo rpm -ivh <package>

where <package>= the name of the python-hashlibs rpm

Refer the screenshot below:

osad-python-haslib-dep

NOTE: You may also use a "yum install" if python-haslibs is available in any of your your yum repositories.

(1) The solution above describes a successful problem-solving experience and may not be applicable to other problems with similar symptoms.

(2) Your rating of this post will be much appreciated as it gives me and others who read this article, an indication of whether this solution has worked for people other than me. Also, feel free to leave comments.

 

VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

How to install Tomcat 6 on RHEL 6

Installing software on RHEL platforms using yum is straightforward. However, based on your environment, there could be a few more steps to get there. So, here’s what I did to install Tomcat 6 on RHEL 6.2:

Environment:

OS: Red Hat Enterprise Linux Server release 6.2 (Santiago)

Yum Repos: Red Hat Enterprise Linux Server (v. 6 for 64-bit x86_64), RHN Tools for RHEL (v. 6 for 64-bit x86_64)

Implementation:

STEP 1: Install the Tomcat6 Web Servlet container

sudo yum groupinstall web-servlet

STEP 2: Enable the Tomcat6 service

sudo chkconfig tomcat6 on

STEP 3: Change ownership for the Tomcat6 resources

When tomcat6 is installed via STEP 1, a user and group with the same name (tomcat) is created. For security, the user is created without an interactive login shell (/sbin/nologin). So, in order to ensure that the application support individuals don’t require root privileges, you must do the following:

 

sudo chown -R tomcat:tomcat /usr/share/tomcat6
sudo chown -R tomcat:tomcat /etc/tomcat6/*

 

NOTE: By default, the tomcat user is created with umask 022 and so individual accounts will require sudo privileges to modify the resources owned by tomcat. This also ensures that all operations on tomcat resources are audited.

 

STEP 4: Test the Tomcat6 service

After doing STEPS 1-3, I started/stopped tomcat6 using the following commands:

sudo service tomcat6 start
sudo service tomcat6 stop

Tomcat6 started and stopped successfully (and http://localhost:8080 was accessible), but the following two messages in catalina.out bugged me:

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib

SEVERE: destroyMBeans: Throwable javax.management.MalformedObjectNameException: Cannot create object name for org.apache.catalina.connector.Connector@290fd7f6 at org.apache.catalina.mbeans.MBeanUtils.createObjectName(MBeanUtils.java:764) at org.apache.catalina.mbeans.MBeanUtils.destroyMBean(MBeanUtils.java:1416)

.

The first INFO message was logged whenever tomcat6 was started and the SEVERE message was logged whenever tomcat6 was stopped.

Getting rid of the INFO message requires installing the Tomcat Native Library (see STEP 5) and it’s recommended that you do this for optimal performance (native code faster than Java bytecode).

Regarding the SEVERE message, it seems to have been fixed in Tomcat 6.0.25 ( refer Tomcat6 Bug ) and the version I installed using the above steps was 6.0.24. As this error is harmless, I’d wait for 6.0.25.

 

STEP 5: Install the Tomcat Native Library

Unfortunately, the standard RHEL yum repos used in our company (see Environment above) did not contain packages for the Tomcat Native Library. So, here’s what I did to install the library:

  • Install the pre-requisite packages
sudo yum install apr apr-devel java-1.6.0-openjdk-devel.x86_64 openssl-devel.x86_64

NOTE: I required only the above packages, but your requirement may vary based on your existing OS installation.

  • Download the tomcat native library from here
  • Execute the following commands:
# Extract downloaded tar
tar xvzf tomcat-native-1.1.22-src.tar.gz

# Configure
cd tomcat-native-1.1.22-src/jni/native
sudo ./configure \
--with-apr=/usr/bin/apr-1-config \
--with-java-home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64 \
--with-ssl=/usr/include/openssl \
--prefix=/usr/lib64

# Make
sudo make

#Make Install
sudo make install

# The steps above installed the library in /usr/lib64/lib. As the default LD_LIBRARY_PATH 
#includes /usr/lib64, you may either change the path or set up links as shown below:

cd /usr/lib64
sudo ln -s lib/libtcnative-1.so.0.1.22 libtcnative-1.so
sudo ln -s lib/libtcnative-1.so.0.1.22 libtcnative-1.so.0

Successful installation of the Tomcat Native library will show something similar to the following in catalina.out when you start the tomcat6 service:

INFO: Loaded APR based Apache Tomcat Native library 1.1.22.

I observed that the Tomcat Native Library made quite an improvement to the tomcat6 server start time. Prior to installation, tomcat 6 started in about 144ms and after installation, it took only around 77ms!

VN:F [1.9.22_1171]
Rating: +23 (from 35 votes)

Red Hat Enterprise Linux (RHEL) cluster configuration file is /etc/cluster/cluster.conf

NOTE: Red Hat discourages direct editing of the cluster configuration file and recommends using the system-config-cluster GUI.

However, if you need to edit the file, here’s a method that works:

STEP 1: Edit the  Cluster Configuration file on any one node in the cluster

  • Open /etc/cluster/cluster.conf using your favourite editor amd make your required changes.
  • Ensure that you increment the config_version (in line 2) by 1. For example, if the config_version is 45, then make it 46.
  • Save and close the file

STEP 2: Update the cluster

Execute the following command (with root privileges) on the same node used in STEP 1:

ccs_tool update /etc/cluster/cluster.conf

The above command will display output similar to the following:

Config file updated from version 45 to 46

Update complete.

VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

At times, to meet performance requirements, you would want to disable file system journaling. Given below are steps to do so for an ext4 file system (e.g. /dev/sda1). These steps have been tested on RHEL 5.7). All commands are to be executed with root privileges:

STEP 1: Unmount the file system partition whose journaling you wish to disable

Use the following command to unmount the partition on /dev/sda1 (let’s say it’s /opt):

umount /opt

NOTE: The command used above is umount and not unmount.

STEP 2: Disable journaling for the file system

Use the following command to disable journaling for an ext4 file system:

tune4fs -O ^has_journal /dev/sda1

 

STEP 3: Perform a file system check

Use the following command to perform a file system check. This is not strictly required, but is recommended for checking file system integrity after disabling journaling:

e4fsck –f /dev/sda1

 

STEP 4: Reboot

You may use the following command to reboot the Linux OS:

shutdown –r now

 

STEP 5: Verify that the file system has journaling disabled and the partition is mounted

After the host has rebooted, you may use the following commands to check if journaling is disabled for the filesystem and the partition is mounted:

dmesg | grep EXT4

Expected output similar to: EXT4-fs (dm-3): mounted filesystem without journal

df -h

 

In order to re-enable journaling, repeat all the STEPS above, but without the ‘^’ in STEP 2.

VN:F [1.9.22_1171]
Rating: +17 (from 23 votes)

How PAM works

Pluggable Authentication Modules (PAM) is a framework used for authentication. Typically, most Linux distros come with PAM installed by default. PAM can be powerful if used well and it’s important to understand how PAM works. PAM has its criticisms, but is quite adequate for most purposes.

Refer this LINUX FORMAT article for a good introduction to PAM.

For easy reference, I’ve stitched together an image of important PAM concepts (shown below) taken from the LINUX FORMAT article.

PAM

 

                 First published in lxf

VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Problem:

When changes are made to /etc/security/limits.conf to apply limits for resources (e.g. file descriptors, processes), the changes are not visible in my Shell (SSH Session). However, when using programs such as su, the changes are visible.

Background:

/etc/security/limits.conf is the configuration file for the pam_limits PAM module. By default, the pam_limits module is used in ssh policies in /etc/pam.d. However, the SSH server must be configured to use PAM.

Solution:

  • Make your SSH server PAM-aware by setting one or both of the following in the SSH configuration file:
  • UsePamSessions=yes
    
          or
    
    UsePAM=yes
    
  • Restart the SSH server

Root Cause:

The SSH server was not configured to use PAM.

 

NOTE:

(1) The solution above describes a successful problem-solving experience and may not be applicable to other problems with similar symptoms.

(2) Your rating of this post will be much appreciated. Also, feel free to leave comments.

 

VN:F [1.9.22_1171]
Rating: +6 (from 6 votes)
 Page 1 of 2  1  2 »