Centralized Linux Bash History
If you would like to monitor the real-time activity of system users on a centralized platform, take a look at this article.
Join the DZone community and get the full member experience.
Join For FreeMost of the time, a user interacts with the Linux system through shell commands. The default installation of Linux provides some level of storing this information. But these are standalone and cannot predict the anomalies associated with not having session correlation in it. Additionally, a common user can override and hide his own activities.
Approach 1
By default, Linux records all the commands executed in a .bash_history file.
Advantages
- You can’t remove the file
- You can’t change the file permissions
[user1@rhel-host1 ~]$ chmod 700 .bash_history
chmod: changing permissions of ‘.bash_history’: Operation not permitted
[user1@rhel-host1 ~]$ rm .bash_history
rm: cannot remove ‘.bash_history’: Operation not permitted
[user1@rhel-host1 ~]$
Drawbacks
- Not centralized; each user gets their own .bash_history file.
- There is no user session correlation.
- The user can override the default behavior and commands will not be saved.
[user1@rhel-host1 ~]$ export HISTCONTROL=ignorespace ;
[user1@rhel-host1 ~]$ <SPACE> ls -ltr ## add <SPACE in front of the command >
Approach 2
At a regular interval, ship the Bash history to a centralized server and look through all the commands.
Advantages
- Better than the default and there is no need to look at multiple files
Drawbacks
- You need an agent like Filebeat or Logstash to forward the commands to a centralized system.
- Still provides no user session correlation
Approach 3
Use the user command prompt variable (PROMPT_COMMAND
) to send the last executed command to a centralized system.
Advantages
- No agent is required
Drawbacks
A user can reset the variable; this will eventually stop sending the commands to the centralized system
Approach 4
Enable the trap command and capture the user command to a centralized system.
Advantages
- No Agent required
Drawbacks
A user can disable the trap.
Preferred Approach
Building Blocks
trap-trap (Enhanced):
Enable the trap
command at the user session start time (/etc/bashrc) and start logging the commands to our centralized platform.
Profile functions are enhanced to check user commands. if they try to override the expected behavior, the script will overwrite the command. Also, auto-heals /re-enable the trap to some extent.
It traps the remotely executed command and adds a tag to differentiate the regular commands. Also injects user’s session details to the logger.
More information on script documentation and the Docker file of building blocks can be found here.
function log2syslog
{
## set the Hostname of present system (Client)
HOSTNAME=$(hostname)
## Set the Timestamp of command Execution with Date RFC-5424
CMD_EXEC_TIME=$(date '+%Y-%m-%dT%H:%M:%S%:z')
## set exec the Date
CMD_EXEC_DATE=$(date '+%Y-%m-%d')
## Gets the Time
CMD_EXEC_TIME_ALONE=$(date '+%H:%M')
## Checks for user’s origin IP is Empty and set equivalent mock data
[[ -n ${SSH_CLIENT} ]] || SSH_CLIENT="noData noData noData"
declare WHO_RUNS
## Variable declared for getting logged-in user details
WHO_RUNS=$(who -m | sed 's/[()]//g' | sed -r 's/ +/ /g')
## Gets the Logged-in User ID, IP & Login Time
[[ -n ${WHO_RUNS} ]] || WHO_RUNS="noData noData ${CMD_EXEC_DATE} ${CMD_EXEC_TIME_ALONE} noData"
## Checks for user’s origin details are Empty and setting string equivalent mock data
declare COMMAND
## Variable declared for getting logged-in user executed Command
COMMAND=$(fc -ln -0 | sed -r 's/ //g' | sed -r 's/ +/ /g')
## Fetch the executed Command
[[ -z ${BASH_EXECUTION_STRING} ]] || COMMAND="#000 RemotelyExecCommand ${BASH_EXECUTION_STRING}"
## Checks for Remotely (via ssh) Executed Commands and set the COMMAND variable
[[ -n ${COMMAND} ]] || COMMAND="#000 CommandIsEmpty_or_ssh_login"
## Checking for executed Command and setting string equivalent mock data if empty
## Checks if user has tried to Disable the Trap; then SigQuit the PID ; this will reenable the logger trap
if [[ ${PREVIOUS_COMMAND} != *" CommandIsEmpty_or_ssh_login" && -n ${PREVIOUS_COMMAND} && ${COMMAND} == *"trap"*"-"*"DEBUG"* ]];
then
kill -SIGQUIT $$
fi
## Checks whether HISCONTROL variable was modified; then reset it
if [[ ${PREVIOUS_COMMAND} == *"HISTCONTROL"*"="* ]];
then
export HISTCONTROL="ignoredups"
fi
## Checks if User has tried to remove the Trap; then enable it back
if [[ ${PREVIOUS_COMMAND} == *"trap"*"-"*"SIGQUIT"* ]];
then
trap enabletrap SIGQUIT
fi
## Check for deduplicate and remove for redundant commands being logged; also hard-code the hostname in the logger
if [[ -z ${PREVIOUS_COMMAND} || "${COMMAND}" != "${PREVIOUS_COMMAND}" ]] ;
then
logger -n xyz.abc.com -t cmdhist -i -- "${HOSTNAME} ${CMD_EXEC_TIME} ${SSH_CLIENT} ${WHO_RUNS} ${USER} ${COMMAND}"
export PREVIOUS_COMMAND=${COMMAND}
fi
}
## enable the trap in Debug mode
trap log2syslog DEBUG
function enabletrap {
trap log2syslog DEBUG
}
## Enabling the Trap
trap enabletrap SIGQUIT
Key Features
|
|
|
|
|
|
Benefits
|
|
|
|
|
|
Opinions expressed by DZone contributors are their own.
Comments