Blog

Feb
25

Shell History

AIX (in common with other Unix versions) logs a lot of information including a list of all the commands executed in each shell. Unfortunately you first have to correctly configure it. The most important files are controlled by the following variables:

HISTSIZE = Maximum size of the history file
HISTFILE = Location/filename used to record commands executed.
Smit commands are also recorded
LOG = Smit menus accessed (default = $HOME/smit.log)
SCRIPT = Commands/scripts run by smit (default = $HOME/smit.script)

Creating separate use history files

This is a sample “.profile” to create separate histories for each user that switches user to root:

[[ ! -d $HOME/histories ]] && mkdir $HOME/histories
alias free=”/home/root/scripts/free”
alias prompt=”. /home/root/scripts/prompt”
# Set Putty Window Title
H=$(hostname) ; N=$(host $H | awk ‘{print $NF}’)
echo “33]0;$H – $N07”
#———————————————-

 

[[ ! -d $HOME/histories ]] && mkdir $HOME/histories
HISTSIZE=65536
HISTFILE=$HOME/histories/.”$(who am i | nawk ‘{print $1}’)_history”
LOG=$HOME/histories/smit.log.$(logname)
SCRIPT=$HOME/histories/smit.script.$(logname)
alias smit=’smit -l $LOG -s $SCRIPT’
alias smitty=’smit -l $LOG -s $SCRIPT’
# add timestamps to history
export HISTTIMEFORMAT=”%Y-%m-%d %H:%M:%S %z”

 

Extended History for each user

Separate history files for each day by changing “/etc/profile”:
# Change the location and size of the users history
FILEDATE=`date +%m%d%Y` HISTFILE=/var/adm/shell/history/$LOGIN.sh.history.$ FILEDATE HISTSIZE=65536
export HISTFILE HISTSIZE
 

Time stamp your entries in the shell history by changing “/etc/environment”

# Added for extended shell history EXTENDED_HISTORY=ON
cat .kshrc
TTY=$(tty|cut -f3-4 -d/)
export HISTFILE=$HOME/.sh_hist$(echo ${TTY} | tr -d ‘/’)
Time stamp your entries in the shell history by changing “/etc/environment”: # Added for extended shell history EXTENDED_HISTORY=ON
Use: history -t to get a timestamp with every command that is executed.

Blog

Feb
13

Migrating to Linux and/or the cloud? (Don’t dismiss IBM just yet)!

Many people are either planning to, or are in the process of moving to open-source and/or cloud-based solutions in order to reduce costs and reliance on local IT resources.

The problem is that whilst the destination maybe clear the journey still has to be carefully planned!

If you currently run a complex set of applications and have data on a pSeries or iSeries environment you cannot simply copy everything into the cloud, or onto a Linux server and expect it to work out of the box.

You first need to measure your current system(s) performance and create a map of all the data you hold, and how it flows in and out of your organisation, and then ensure that your new systems can provide a suitable replacement. You also need to plan for a period of parallel-running and testing, and maintain your legacy system(s) until such a time that you can cut-over. Licencing and compliance can also be massive challenges as you may need more Linux (virtual) servers and local regulations may prohibit or restrict your options.

Fortunately AIX has a lot of tools that can make this migration a lot less difficult! Since version 5.1 AIX has included what IBM calls “Linux Afinity” and this means that you can install many open-source tools such as the GNU compiler, and use that to compile a great number of open-source products such Nagios or wget, and there are also sites that provide ready-made RPMs e.g. Perzl or Bull.

AIX can work with both its native LPP (Licenced Program Product) packages, or the open-source style RPM (Redhat Package Manager) format. IBM also provides a large number of Linux RPMs as part of its Linux Toolkit.

The other great advantage here is that you can measure the performance of an open-source application such as PostgreSQL whilst both on Power and Intel or ARM based systems. You may even find that they perform better on your pSeries server and you actually need more x-based systems, or cloud-capacity to maintain your service level.

Another option is to create Power-Linux partitions on your existing hardware. IBM currently supports both Redhat and Suse distributions and you can also install other versions such as YellowDog Linux which means you could actually run a complete Linux environment without having to ditch your pSeries or iSeries servers.

You should also not overlook the fact that IBM has done a huge amount of work creating hardware and software based solutions such as shared-processor-pooling and active-memory sharing and expansion which, regardless of the OS, can result in huge cost savings.

The other “Elephant in the room” is the IT resource required for both the planning and execution phases of your project as they will need both AIX and Linux/cloud knowledge and be able to work comfortably in both environments.

Blog

Feb
13
Making Notes from Book

More Logging

You can never have too much logging, in principle at least, however you need to carefully manage logs so they don’t fill your filesystem(s), and so you don’t get blinded by too much information.
There are several logs provided by AIX that are very useful, but often overlooked:

Shutdown Logs

A log can happen during shutdown that gets missed. Fortunately there is a simple remedy! Always use the “-l” (Logging) flag e.g. shutdown -lF, as this creates the file “/etc/shutdown.log” and this can yield a lot of interesting information and aid fault resolution.

SNMP Logging

Start by checking with version of SNMPD you are using:
# ls -l /usr/sbin/snmpd
lrwxrwxrwx    1 root     system           19 Apr 17 2012  /usr/sbin/snmpd -> /usr/sbin/snmpdv3ne
The example above shows that this system is using version-3.

SNMPD Version 1.

If you are using traditional SNMPD (not recommended as it has lots of security issues)
The “/etc/snmpd.conf” file controls the behaviour of this daemon and the default is not to log anything. To start logging see the following entries:
logging         file=/usr/tmp/snmpd.log         enabled logging         size=100000                     level=0
Ensure that the file “??.log” exists and that the “level” is either 1, 2, or 3, then restart the daemon. You should then

SNMPD Version3.

There are two types of version-3, unencrypted and encrypted. You can see which one you are using from the symbolic-link (e.g.  /usr/sbin/snmpdv3ne is un-encrypted). In either event you must configure it using “/etc/snmpdv3.conf”. The logging entry is almost exactly the same as version-1, except that you can log up to level 4.

13-02-2014

Blog

Feb
05

Using syslog in a chroot jail

If your system uses chroot jails you should enable separate logging within them in order to catch important messages such as break-in attempts.

1. Setup the chroot jail.
The jail requires a dedicated filesystem (in this example “/var/local/chroot”), and a small selection of executables and libraries. These should be copied directly from the installed OS, and updated following each major OS patch or update (There are many scripts available on the web that automate the building of a jail so I won’t list all the steps).

mknod /var/local/chroot/dev/null c 2 2
chmod 666 /var/local/chroot/dev/null

cp -p /usr/bin/ksh /var/local/chroot/usr/bin/
cp -p /usr/lib/libcrypt.a /var/local/chroot/usr/lib/
cp -p /usr/lib/libc.a /var/local/chroot/usr/lib/
cp -p /usr/lib/libodm.a /var/local/chroot/usr/lib/
cp -p /usr/lib/libsrc.a /var/local/chroot/usr/lib/

cp -p /usr/sbin/sftp-server /var/local/chroot/usr/sbin
cp -p /usr/sbin/syslogd /var/local/chroot/usr/sbin

2. A chroot’ed syslog daemon is required for each jail in order to enable jailed events to be logged to the global syslog. These processes should be started AFTER the main syslog daemon and thus placed at the end of the “/etc/inittab” as follows:

syslogJAIL:23456789:once:chroot /var/local/chroot /usr/sbin/syslogd -f /etc/chroot.conf -p /dev/log

Note:    The “chroot*.conf” exists within the jailed environment and can be used to send alerts to a remote/central log file e.g.

/var/local/chroot/etc:
*.debug;local3.none                 /var/adm/sftp.log               rotate size 1m files 21 compress

3. The sshd daemon runs from within each jail and thus requires its own “sshd_config” file:

/var/local/chroot/etc/sshd_config:
Protocol 2
PermitRootLogin no
IgnoreUserKnownHosts yes
PermitEmptyPasswords no
HostbasedAuthentication yes
LogLevel VERBOSE
SyslogFacility AUTH
UsePrivilegeSeparation yes
PasswordAuthentication no
ChallengeResponseAuthentication no
AllowGroups netadm netsadm  sftpgrp
Subsystem      sftp    /usr/sbin/sftp-server -f AUTH -l VERBOSE
#AuthorizedKeysFile     %h/.ssh/authorized_keys

Match User jailed_user
PasswordAuthentication yes

A login file prevents the jailed users from spawning a shell e.g.

“/var/local/chroot/./home/jaileduser/.profile”:

trap “” 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
echo “This account may not be used for shell login!”;exit 0
trap 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

AIX normally knows that user is jailed because they have a “.” within there home-directory path in “/etc/passwd”. This can create a problem when trying to SSH or SFTP to the parent system as the jailed user. In this case you need to find the AIX chroot patch for your version of SSH, and then recompile it.