Friday, 11 September 2015

cPanel/WHM Locations of Logs

cPanel Logs

Access logs and user actions                                /usr/local/cpanel/logs/access_log

Account transfers and misc. logs                         /var/cpanel/logs

Auditing log (account creations, deletions, etc) /var/cpanel/accounting.log

Backup logs                                                         /usr/local/cpanel/logs/cpbackup

Brute force protection (cphulkd) log                 /usr/local/cpanel/logs/cphulkd.log

Cpanel dnsadmin dns clustering daemon                /usr/local/cpanel/logs/dnsadmin_log

Cpanel taskqueue processing daemon               /usr/local/cpanel/logs/queueprocd.log

DBmapping                                                      /usr/local/cpanel/logs/setupdbmap_log

EasyApache build logs                                     /usr/local/cpanel/logs/easy/apache/

Error log                                                           /usr/local/cpanel/logs/error_log

Installation log                                                    /var/log/cpanel

License updates and errors                            /usr/local/cpanel/logs/license_log

Locale database modifications                           /usr/local/cpanel/logs/build_locale_database_log

Login errors (CPSRVD)                                   /usr/local/cpanel/logs/login_log

Horde                                                           /var/cpanel/horde/log/

RoundCube                                                   /var/cpanel/roundcube/log/

SquirrelMail                                                   /var/cpanel/squirrelmail/

Panic log                                                          /usr/local/cpanel/logs/panic_log

Per account bandwidth history (Cached)         /var/cpanel/bandwidth.cache/{USERNAME}

Per account bandwidth history (Human Readable)    /var/cpanel/bandwidth/{USERNAME}

Service status logs                                           /var/log/chkservd.log

Tailwatch driver tailwatchd log                  /usr/local/cpanel/logs/tailwatch_log

Update analysis reporting                     /usr/local/cpanel/logs/updated_analysis/{TIMESTAMP}.log

Update (UPCP) log                                         /var/cpanel/updatelogs/updated.{TIMESTAMP}.log

WebDisk (CPDAVD)                                 /usr/local/cpanel/logs/cpdavd_error_log

Website statistics log                                /usr/local/cpanel/logs/stats_log

cPanel Access Log


Access logs and user actions                /usr/local/cpanel/logs/access_log

cPanel Apache Log

Apache restarts done through cPanel and WHM          /usr/local/cpanel/logs/safeapcherestart_log

Domain access logs                                                    /usr/local/apache/domlogs/{DOMAIN}

Processing of log splitting                                          /usr/local/cpanel/logs/splitlogs_log

suPHP audit log                                                          /usr/local/apache/logs/suphp_log

Web server and CGI application error log                  /usr/local/apache/logs/error_log

cPanel Backup Log

Backups                                               /usr/local/cpanel/logs/cpbackup/{##########}.log

Backup Transporter                               /usr/local/cpanel/logs/cpbackup_transporter.log

cPanel Email Log

Delivery and receipt log                            /var/log/exim_mainlog

Incoming mail queue                           /var/spool/exim/input/

Log of messages rejected based on ACLS or other policies /var/log/exim_rejectlog

Unexpected/Fatal error log                   /var/log/exim_paniclog

IMAP, POP login attempts, transactions, fatal errors and spam scoring /var/log/maillog & /var/log/messages

Mailman                                                   /usr/local/cpanel/3rdparty/mailmain/logs

MySQL Log

MySQL error log                                                 /var/lib/mysql/{SERVER_NAME}.err

MySQL slow query log (if enabled in my.cnf) /var/log/slowqueries

How do I enable remote access to MySQL database server on Plesk?

Since Plesk 8 MySQL users and databases are created with permissions that allow access to the database from outside. In previous Plesk versions MySQL users and databases are created with permissions that allow to access the database from localhost only. However, sometimes you need to provide the remote access.

To assign the full privileges to admin user on all the databases on server running plesk.

1. To go to mysql prompt use command

# mysql -u admin -p`cat /etc/psa/.psa.shadow`

2. To assign the admin user on all the databases on server running plesk

mysql > GRANT ALL PRIVILEGES ON *.* TO 'admin'@'1.1.1.1' identified by 'PASSWORD';

Where,

*.           => For all the databases on the server
admin    => User Name to be used with full privileges
1.1.1.1   => The Ip address of the machine from which you wish to access all the db’s ( Remote Host )

To verify The connectivity of the DB from remote host run the following command.

mysql -h 2.2.2.2 -u admin -p yourpassword

Where,

2.2.2.2   => Is the mysql server IP Address

Domain has exceeded the max defers and failures per hour (5/5 (50%)) allowed. Message discarded.

There is a cPanel feature intended to limit the ability of exploited or hacked sites to send out spam has been implemented. A few customers who send out mass mailings have been triggering this feature, due to the number of bad/undeliverable email addresses on their lists.

If you encounter this problem, you'll receive a bounce message with an error similar to the following:

==========================================================
Domain  has exceeded the max defers and failures per hour (5/5 (71%)) allowed. Message discarded.
==========================================================

cPanel will regularly monitor the emails sent through all email accounts on your domain, and if, over the past hour, more than 25% of the attempted deliveries have failed, outbound email will temporarily be limited.

The "(5/5)" portion of the error indicates that the measurement of bounces kicked in once 5 bounces were detected during the hour. In other words, if you have 4 bounces in an hour during which you sent 16 emails, even though 50% of your emails have bounced, nothing will happen because you are under 5 bounces. Once you reach five bounces in an hour, the bounce percentage measurement is taken and a sending restriction is enforced if you're over the bounce percentage limit.

Because there are clients who fail to keep their PHP scripts updated with newest versions, some sites are vulnerable to exploits by spammers. Sites get hacked and mail accounts hijacked for spam purposes. Because spam campaigns have a high bounce rate, this feature can quickly limit the damage that can be done by these exploits, and prevent our servers from ending up on dns blacklists, and then being unable to deliver legitimate email properly.

If you send mass mail and haven't kept your mailing list clean by removing invalid email addresses, you may generate enough bounces to have your mailings limited. The only way to work around this is to clean up your list and remove any invalid email addresses.

f you're not sure exactly what is causing this, you can probably figure it out by using the Email Trace icon in your hosting control panel. When you click the Email Trace icon, you'll see a field where you can enter a recipient's email address and then click a "Run Report" button to get information about email sent to that recipient. If you enter nothing for the recipient email address, you'll get back data for all email traffic, and as you look through it you should see groups of bounced messages which can help you determine what sender caused the problem, and why.

Thursday, 10 September 2015

Rootkit Hunter (rkhunter) Installation on CentOS

Rootkit Hunter (rkhunter) is a Unix-based tool that scans for rootkits, backdoors and possible local exploits.

Rootkits are self-hiding toolkits secretly installed by a malicious intruder to allow that user to gain access to the server. Rootkit Hunter offers protection by comparing SHA-1 hashes of important files with known good ones in a online database as well as:

*MD5 hash compare.
*Look for default files used by rootkits.
*Wrong file permissions for binaries.
*Look for suspected strings in LKM and KLD modules.
*Look for hidden files.
*Optional scan within plaintext and binary files.

1. Installation

Download Rootkit Hunter

Begin by downloading the latest stable version of Rkhunter by using the wget command. The /usr/local/src folder is where you should put any programs (source or binary) you've downloaded before installing them.

Make sure to check for the latest available version here, and append the instructions below accordingly.

cd /usr/local/src
wget http://dfn.dl.sourceforge.net/sourceforge/rkhunter/rkhunter-1.4.2.tar.gz
wget http://dfn.dl.sourceforge.net/sourceforge/rkhunter/rkhunter-1.4.2.tar.gz.sha1.txt
sha1sum -c rkhunter-1.4.2.tar.gz.sha1.txt

2. Installation Rootkit Hunter.

Once you have downloaded the latest version of Rootkit Hunter, issue the following commands as root to start the installation routine.

tar -zxvf rkhunter-1.4.2.tar.gz
cd rkhunter-1.4.2
./installer.sh --layout default --install
/usr/local/bin/rkhunter --update
/usr/local/bin/rkhunter --propupd
rm -Rf /usr/local/src/rkhunter*

3. Automate Rootkit Hunter.

Rkhunter can be setup to run checks every day so that we always have up-to-date information about intrusions. This can be accomplished by creating a cronjob.

4. Rootkit Hunter configuration.

The configuration file for rkhunter can be found at:

/etc/rkhunter.conf

5. SSHD Root Logon.

The parameter ALLOW_SSH_ROOT_USER tells rkhunter whether or not the root user is allowed to ssh into the system. This is unset by default in the rkhunter.conf file. Rkhunter will complain about this on every run. If you have disabled root login, you should set this parameter to "no".

ALLOW_SSH_ROOT_USER=no

If you need root login over SSH, you should change this parameter to "yes" so that rkhunter can check this and will mark this setting as valid:

ALLOW_SSH_ROOT_USER=yes

Security practices recommend disabling root login.

6. Update rkhunter.

To check the currently installed version enter the following:

# /usr/local/bin/rkhunter --versioncheck

Run the updater by issuing the following command:

# /usr/local/bin/rkhunter --update

With our database files refreshed, we need to tell rkhunter to check the current values and store them as known-good values:

# /usr/local/bin/rkhunter --propupd

6. Manual Scan.

You can initiate a manual scan by issuing the following command:

/usr/local/bin/rkhunter -c

Which runs rkhunter in interactive mode. In other words, when it gets to the end of a particular scan, you need to press 'enter' to continue. If you want to "auto skip" interactive mode, add the -sk option at the end:

/usr/local/bin/rkhunter -c -sk

To scan the entire file system enter:

rkhunter --check

Your scan results should look as follows:

---------------------------- Scan results ----------------------------
MD5 scan
Scanned files: 0
Incorrect MD5 checksums: 0

File scan
Scanned files: 412
Possible infected files: 0

Application scan
Vulnerable applications: 0

Scanning took 39 seconds
-----------------------------------------------------------------------

For more information and options please run the following command.

# rkhunter --help

CSF: Config Server Firewall Installation

The ConfigServer Security & Firewall is a popular open source Stateful Packet Inspection (SPI) firewall, Login/Intrusion Detection and Security application, compatible with most Linux servers.

CSF can be fully configured to block/restrict ports you don't want open. CSF includes the Login Failure Daemon (LFD), which will scan log files and monitor failed login attempts, such as login attempts for FTP and E-Mail accounts, and it will block the IP according to the rules you have setup. CSF also offers Connection Limiting, Real Time Block Lists and Port Scan tracking and much more.

CSF can be easily managed from within its GUI, which is fully compatible with DirectAdmin, CPanel, and WebMin/Virtualmin.

It is important to remove older firewalls or any other firewalls setup to protect the server. This is because the conflict of Firewalls can lead to failures or inaccessibility. You should also not install any other iptables firewall and if it already exists, then it has to be removed at this stage. Most of the systems is likely to have APF+BFD firewalls and has to be removed. So use the following command to detect and remove them if they exist.
------------------------------------------------
sh /usr/local/csf/bin/remove_apf_bfd.sh
------------------------------------------------
1)Install CSF Firewall.

Download the CSF archive to the /tmp folder of your server by using wget, unpack the archive by issuing the TAR command and finally install CSF by starting the ./install.sh setup script.
------------------------------------------------------------------------------------------------
cd /tmp
wget  http://www.configserver.com/free/csf.tgz
tar zxvf csf.tgz
cd  csf
./install.sh
------------------------------------------------------------------------------------------------
The plugins for DirectAdmin or cPanel are installed automatically.

2)Test IPtables configuration.

This test is recommended to double check that the correct iptables modules are installed. The test can be invoked by issuing the command below, or by going to the "test iptables" section, which can be found at the bottom of the CSF Graphic interface. If not all modules are installed, you need to work on getting them installed.
------------------------------------------------------------------------------------------------
$ /etc/csf/csftest.pl
Testing ip_tables/iptable_filter...OK
Testing ipt_LOG...OK
Testing ipt_multiport/xt_multiport...OK
Testing ipt_REJECT...OK
Testing ipt_state/xt_state...OK
Testing ipt_limit/xt_limit...OK
Testing ipt_recent...OK
Testing xt_connlimit...OK
Testing ipt_owner/xt_owner...OK
Testing iptable_nat/ipt_REDIRECT...OK
Testing iptable_nat/ipt_DNAT...OK
RESULT: csf should function on this server
------------------------------------------------------------------------------------------------
3)Configuration.

The configuration of your CSF Firewall installation can be maintained by editing the various config files CSF ships with. On Red Hat Enterprise Linux (RHEL) based distributions these can be found in the following location:/etc/csf/

The configuration files include:

csf.conf - the main configuration file, it has helpful comments explaining what each option does.
csf.allow - a list of IP's and CIDR addresses that should always be allowed through the firewall.
csf.deny - a list of IP's and CIDR addresses that should never be allowed through the firewall.
csf.ignore - a list of IP's and CIDR addresses that lfd should ignore and not block if detected.
csf.*ignore - various ignore files that list files, users, IP's that lfd should ignore. See each file for their specific purpose.

If you modify any of the files listed above, you will need to restart csf to have them take effect. If you use the command line options to add or deny IP addresses, then csf automatically does this for you.

However, for the average user it is far quicker to make use of its Graphic Interface (GUI), which can be accessed from within your DirectAdmin, CPanel or Webmin/Virtualmin Control Panel.

4)Enabling CSF Firewall.

The CSF firewall can be fully enabled by setting:

TESTING = 0

This can be done by accessing the GUI, or by editing the main configuration file, found at /etc/csf/csf.conf.

Please ensure your configuration is correct. The wrong settings may lock you out of your server permanently!

5)TCP_IN and TCP_OUT / UDP_IN and UDP_OUT

Below you will find a basic explanation for the recommended opened TCP_IN ports for INCOMING connections to your Linux/cPanel based web server. These ports can be opened from the GUI or the csf.conf file.

20,21       FTP access 
22            SSH Access
25, 587    SMTP for EXIM to receive e-mail
53            DNS (Named), The port for your nameservers. Both TCP and UDP ports should be opened here.
80, 443     Apache traffic, http and https
110, 993   POP e-mail access
143, 995   IMAP email access
3306         MySQL. You should not open this port if you don't want to allow remote MySQL access, as most MySQL scripts are accessed locally
2222         DirectAdmin Access
2083         cPanel Access over an encrypted SSL connection
2082         cPanel Access over an unencrypted connection
2087         cPanel WHM Access over an encrypted SSL connection
2086         cPanel WHM Access over an unencrypted connection
10000       Webmin Access

* FTP requires a random high port number if the client is in PORT mode. When using ProFTP you may need to add a port range into your /etc/proftpd.conf file to allow ftp connections, eg: PassivePorts 35000 35999 and then open that port range in your CSF firewall. Ranges can be defined in CSF by using a colon eg: 35000:35999

TCP_IN and TCP_OUT / UDP_IN and UDP_OUT is a comma separated list of:

# Allow incoming TCP ports
TCP_IN = "20,21,22,25,53,80,110,143,443,465,587,993,995,2222,35000:35999"
# Allow outgoing TCP ports
TCP_OUT = "20,21,22,25,53,80,110,113,443"
# Allow incoming UDP ports
UDP_IN = "20,21,53"
# Allow outgoing UDP ports
UDP_OUT = "20,21,53,113,123"

6)ICMP_IN and ICMP_OUT

Allowing ping is usually a good option for diagnostic purposes.

Set ICMP_IN to 1 to allow incoming ping requests to your server. Set to 0 refuses such requests. If you are hosting any public services, it is recommended to allow ICMP requests, as these can be used to determine whether or not your service is available. ICMP_IN_LIMIT Sets the number of ICMP (ping) requests allowed from one IP address within a specified amount of time. There is usually no need to change the default value (1/s)

Set ICMP_OUT to 1 to allow outgoing ping from your server. Set to 0 refuses such requests. ICMP_OUT_LIMIT Sets the number of outgoing ICMP (ping) requests within a specified amount of time. There is usually no need to change the default value (0)

7)Port flood protection

This setting provides protection against port flood attacks, such as denial of service (DoS) attacks. You may specify the amount of allowed connections on each port within time period of your liking. Enabling this feature is recommended, as it may possibly prevent an attacker forcing your services down. You should pay attention to what limits you set, as too restrictive settings will drop connections from normal clients. Then again, too permissive settings may allow an attacker to succeed in a flood attack.

PORTFLOOD is a comma separated list of:

port;protocol;hit count*;interval seconds

So, a setting of PORTFLOOD = "22;tcp;5;300,80;tcp;20;5" means:

1. If more than 5 connections to tcp port 22 within 300 seconds, then block that IP address from port 22 for at least 300 seconds after the last packet is seen, i.e. there must be a "quiet" period of 300 seconds before the block is lifted.

2. If more than 20 connections to tcp port 80 within 5 seconds, then block that IP address from port 80 for at least 5 seconds after the last packet is seen, i.e. there must be a "quiet" period of 5 seconds before the block is lifted.

You may add more ports by separating them by commas like described as follows: 

port1;protocol1;connection_count1;time1,port2;protocol2;connection_count2;time2

8)Port knocking.

Port knocking allows clients to establish connections a server with no ports open. The server allows clients connect to the main ports only after a successful port knock sequence. You may find this useful if you offer services which are available to only limited audience.

The feature requires that you list a random selection of unused ports (at least 3) with a timeout. The ports you choose must not be in use and not appear in TCP_IN (UDP_IN for udp packets). The port to be opened must also not appear in TCP_IN (UDP_IN for udp packets).

PORTKNOCKING is a comma separated list of:

openport;protocol;timeout;kport1;kport2;kport3[...;kportN]

So, a setting of PORTKNOCKING = "22;TCP;20;100;200;300;400" means:

Open Port 22 TCP for 20 seconds to the connecting IP address to new connections once ports 100, 200, 300 and 400 have been accessed (i.e . knocked with a SYN packet) each knock being less than 20 seconds apart.

Access to port 22 remains active after 20 seconds until the connection is dropped, however new connections will not be allowed.

9)Syslog and RESTRICT_SYSLOG.

When enabled, this option logs lfd (Login Failure Daemon) messages to syslog as well as to /var/log/lfd.log.

Unfortunately, it is trivial for end-users and scripts run by end-users to spoof log lines that appear identical to any log line reported in logs maintained by syslog/rsyslog. You can identify these logs by looking in /etc/syslog.conf or etc/rsyslog.conf

This means that anyone on the server can maliciously trigger applications that monitor these logs, such as lfd does for the following options:
------------------------------------------------------------------------------------------------------------
LF_SSHD LF_FTPD LF_IMAPD LF_POP3D LF_BIND LF_SUHOSIN LF_SSH_EMAIL_ALERT LF_SU_EMAIL_ALERT LF_CONSOLE_EMAIL_ALERT LF_DISTATTACK LF_DISTFTP LT_POP3D LT_IMAPD PS_INTERVAL UID_INTERVAL WEBMIN_LOG LF_WEBMIN_EMAIL_ALERT PORTKNOCKING_ALERT ST_ENABLE SYSLOG_CHECK LOGSCANNER CUSTOM*_LOG
------------------------------------------------------------------------------------------------------------
A malicious user could use this issue to trigger confusing emails regarding both successful and failed login attempts, kernel log lines (including iptables log lines) etc. Unfortunately, there is very little that can be done about this as syslog/rsyslog has no security framework. Some attempt was made in newer versions of rsyslog, but this version is not available in the current versions used by RedHat/CentOS v6. It also has to be enabled and can have adverse effects on utilities that expect a certain format for the log lines.

To mitigate spoofing attempts we recommend the following, if you are willing to accept the consequences of spoofed log lines:

1. Go through the options above ensuring that only those that you need are enabled.
2. Ensure that DENY_IP_LIMIT and DENY_TEMP_IP_LIMIT are set reasonably low (for example, 200). This will limit attempts to block large numbers of IP addresses.
3. Ensure that administrator/support IP addresses are listed in /etc/csf/csf.allow and perhaps /etc/csf/csf.ignore. This will prevent malicious blocking from denying you access to the server.
4. To confirm successful logins to SSH, use the "last" utility from the root shell, e.g.: last -da
5. Regularly check the server and user data for exploits, old vulnerable applications and out of date OS applications.
6. Consider carefully any application that you use that centralises actions and syslog/rsyslog logs and the implications of spoofed log lines.
7. Consider the implications of this overall issue on applications and scripts other than csf/lfd that use the affected log files.
8. Ultimately, you could consider restricting access to all configured syslog/rsyslog unix sockets. This can be used via file permissions and ownership of the sockets (e.g. /dev/log) but there are several caveats: file permissions and ownership have to be reapplied whenever syslog/rsyslog is restarted; restricting logging will break/limit some applications ability to log to syslog/rsyslog, for example crond.
9. Do not enable syslog/rsyslog reception via UDP/TCP ports.

10)Connection limit protection CONNLIMIT

This feature can be used to limit the number concurrent of active connections from an IP address to each port. When properly configured, this may prevent abuses on the server, such as DoS attacks.

CONNLIMIT is a comma separated list of:

port;limit
So, a setting of CONNLIMIT = "22;5,80;20" means:

Only allow up to 5 concurrent new connections to port 22 per IP address.
Only allow up to 20 concurrent new connections to port 80 per IP address.

11)Port/IP address redirection.

CSF can be configured to redirect connections to an IP/port to another IP/port. Note: After redirection, the source address of the client will be the server's IP address.

Requirements:

nat tables
  ipt_DNAT iptables module
  ipt_SNAT iptables module
  ipt_REDIRECT iptables module

The following are the allowed redirection formats:

DNAT (redirect from one IP address to a different one):

IPx|*|IPy|*|tcp/udp          - To IPx redirects to IPy
IPx|portA|IPy|portB|tcp/udp  - To IPx to portA redirects to IPy portB

DNAT examples:

192.168.254.62|*|10.0.0.1|*|tcp
192.168.254.62|666|10.0.0.1|25|tcp

REDIRECT (redirect from port to a different one):

IPx|portA|*|portB|tcp/udp    - To IPx to portA redirects to portB
*|portA|*|portB|tcp/udp      - To portA redirects to portB

REDIRECT examples:

*|666|*|25|tcp
192.168.254.60|666|*|25|tcp
192.168.254.4|666|*|25|tcp
Where a port is specified it cannot be a range, only a single port.

All redirections to another IP address will always appear on the destination server with the source of this server, not the originating IP address.

This feature is not intended to be used for routing, NAT, VPN, etc tasks.

12)SYNFLOOD, SYNFLOOD_RATE and SYNFLOOD_BURST.

Offers protection against SYN flood attacks. This slows down the initialization of every connection, so you should enable this only if you know that your server is under attack.

SYNFLOOD = "0"
SYNFLOOD_RATE = "100/s"
SYNFLOOD_BURST = "150"

13)Login Failure Daemon (LFD). 

To complement the ConfigServer Firewall, a daemon process that runs all the time and periodically (every X seconds) scans the latest log file entries for login attempts against your server that continually fail within a short period of time. Such attempts are often called "Brute-force attacks" and the daemon process responds very quickly to such patterns and blocks offending IP's quickly.

There are an array of extensive checks that lfd can perform to help alert the server administrator of changes to the server, potential problems and possible compromises.

lfd can monitor the most commonly abused protocols, SSHD, POP3, IMAP, FTP and HTTP password protection. Unlike other applications, lfd is a daemon process that monitors logs continuously and so can react within seconds of detecting such attempts. It also monitors across protocols, so if attempts are made on different protocols in a short space of time, all those attempts will be counted against the threshold.

If you want to add some spam protection, CSF can help. Look in the configuration for the following:

LF_SCRIPT_ALERT = 0 change this to 1. This will send an email alert to the system administrator when the limit configured below is reached within an hour.

LF_SCRIPT_LIMIT = 100 change this to 250. This will alert you when any scripts sends out 250 email messages in an hour.

Define email address to which you need to get alerts and define email address to which you want to get.
LF_ALERT_TO = “test1@google.com”
LF_ALERT_FROM = “test2@google.com”

14)Uninstallation.

Removing csf and lfd is even more simple:

cd /etc/csf
sh uninstall.sh

Here are the most common commands you will be using:

csf -u Update CSF
csf -d IPADDRESS will deny an IP.
csf -a IPADDRESS will allow an IP.
csf -r will reload all rules.
-dr, –denyrm ip    Remove and unblock an IP address in /etc/csf.deny.
-t, –temp          Displays the current list of temporary IP bans and their TTL.
-tr, –temprm ip    Remove an IP address from the temporary IP ban list.

For a complete overview of all command line options enter csf or csf -h on the command line and you will receive a list with all available options.

You can also visit the CSF home page where you can find further documentation and FAQs.

http://configserver.com/cp/csf.html
http://download.configserver.com/csf/install.txt

lfd on VPS server: Excessive resource usage alerts

On a Linux VPS with cPanel you may be alerts along the lines of the following:

Time: <some specific time>
Account: <Your cPanel user account>
Resource: Virtual Memory Size
Exceeded: 218 > 200 (MB)
Executable: /usr/bin/php
Command Line: /usr/bin/php /home/your_site/public_html/index.php
PID: <some_number>
Killed: No

This alert is an informational message. How you deal with these are a matter of personal preference and needs of your organization.

There is a daemon on your VPS, the lfd daemon, that will monitor processes and if a process' memory usage exceeds a defined threshold (in this case of the example above, 200MB), it sends an alert. That is all that is happening here. The server has no way of knowing if this is expected behavior or not, so it just sends an alert so the owner can decide whether to investigate.

So, if the threshold is set to 200MB and a script or process uses more than 200MB memory, you will get an alert containing the name of the script, the time it occurred, and the amount of memory that it is consuming.

Here are some options you may want to consider for how to deal with these. This is not an exhaustive list but should give a good starting point.

*When you get the alerts, glance at them and see what script or process is triggering the alert. If your VPS is running particularly sluggish and you are getting alerts about a PHP script consuming large amounts of RAM, you can consider killing the script to return the server to normal responsiveness. Likewise, if you write some script and start getting alerts about the amount of RAM it is consuming, that might be a heads up to you that your script has problems.

*Increase the memory threshold for the alerts from 200MB to say 300MB. That will allow scripts and processes to consume more RAM before they trigger the alert. If you want to increase it much higher than 300MB, then you might be better off just turning this feature off alltogether (see the next option).

*Turn off this feature entirely. If you are not interested in knowing when a process exceeds a defined threshold, then there is no need to get the alert and it can be turned off.

To change the threshold that LFD alerts on:

Log into WHM of your VPS at https://<your VPS IP address>:2087 with the root account.
Go to ConfigServer Security&Firewall --> Firewall Configuration
Change the value for PT_USERMEM to the threshold you want to report on.

To turn off this feature, set the PT_USERMEM paramter to 0.

Fix WHM/cPanel cPHulk Brute Force Protection Lock Out Via SSH

If you or a client is getting the following error:

=============================================
Brute Force Protection
This account is currently locked out because a brute force attempt was detected. Please wait 10 minutes and try again. Attempting to login again will only increase this delay. If you frequently experience this problem, we recommend having your username changed to something less generic.
=============================================

But have been trying to log in a bit later and still receiving the message then you made need to take some further action to resolve the issue.

cPHulk stores all of its information in a database called cphulkd. There are two tables of interest: logins and brutes. The logins table stores login authentication failures. The brutes table stores excessive authentication failures indicative of a brute force attack.

A way to see what's listed in there currently is through MySQL on command line:

===================================================
mysql> connect cphulkd
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Connection id:    id
Current database: cphulkd

mysql> select IP, BRUTETIME from brutes order by BRUTETIME;
Empty set (0.00 sec)

mysql> select IP, LOGINTIME FROM logins order by LOGINTIME;
+---------------------------------+---------------------+
| IP                              | LOGINTIME           |
+---------------------------------+---------------------+
| 220.199.6.48                    | 2009-10-14 11:23:10 |
| 220.199.6.48                    | 2009-10-14 11:23:10 |
| 220.199.6.48                    | 2009-10-14 11:23:10 |
| 118.212.186.59                  | 2009-10-14 11:23:40 |
| 118.212.186.59                  | 2009-10-14 11:23:40 |
| 118.212.186.59                  | 2009-10-14 11:23:40 |
| djdeatheater.liquidweb.com      | 2009-10-14 11:24:03 |
| 221.7.58.37                     | 2009-10-14 11:24:07 |
| 221.7.58.37                     | 2009-10-14 11:24:07 |
| 221.7.58.37                     | 2009-10-14 11:24:07 |
| djdeatheater.liquidweb.com      | 2009-10-14 11:24:09 |
| djdeatheater.liquidweb.com      | 2009-10-14 11:24:15 |
| mail.ingener.com                | 2009-10-14 11:24:53 |
| mail.ingener.com                | 2009-10-14 11:24:57 |
| 123.147.144.45                  | 2009-10-14 11:25:16 |
| 123.147.144.45                  | 2009-10-14 11:25:16 |
| 123.147.144.45                  | 2009-10-14 11:25:16 |
| 119.62.128.42                   | 2009-10-14 11:25:41 |
| 119.62.128.42                   | 2009-10-14 11:25:41 |
| 119.62.128.42                   | 2009-10-14 11:25:41 |
| pomme.sai.msu.ru                | 2009-10-14 11:26:13 |
| pomme.sai.msu.ru                | 2009-10-14 11:26:13 |
| pomme.sai.msu.ru                | 2009-10-14 11:26:13 |
| 84-74-21-119.dclient.hispeed.ch | 2009-10-14 11:26:48 |
| 84-74-21-119.dclient.hispeed.ch | 2009-10-14 11:26:48 |
| 84-74-21-119.dclient.hispeed.ch | 2009-10-14 11:26:48 |
| 114.143.242.51                  | 2009-10-14 11:27:23 |
| 114.143.242.51                  | 2009-10-14 11:27:23 |
| 114.143.242.51                  | 2009-10-14 11:27:23 |
| 222.179.116.53                  | 2009-10-14 11:27:47 |
| 222.179.116.53                  | 2009-10-14 11:27:47 |
| 222.179.116.53                  | 2009-10-14 11:27:47 |
+---------------------------------+---------------------+
32 rows in set (0.00 sec)
===================================================

This will give you a list of the IPs and the LOGINTIME they were entered into the database.

The first way to reconnect would be to disable cPHulk to regain access, log into WHM, clear out the the block by using the “Flush DB” option in the cPHulk settings page, and then re-enable cPHulk. A number of people recommended this method, but it is not secure for the server. What would happen if a huge wave of brute force authentication attempts hit the box in the time between disabling and re-enabling cPHulk? The answer is that the box wouldn't protest and would tell the attacking program whether each attempt was successful or not.

If you need to use this method, the two commands you will want to use are: 

/usr/local/cpanel/bin/cphulk_pam_ctl --disable and 
/usr/local/cpanel/bin/cphulk_pam_ctl --enable. 

These two commands will disable and enable cPHulk, respectively.

Here is a better method. This method does not require disabling cPHulk, and thus, does not require reducing protection to regain access. Essentially, clear the tables manually, so that you can log in once again.

While still connected to the database through the MySQL monitor, run a couple more queries.

mysql> delete from brutes;
Query OK, 0 rows affected (0.00 sec)

mysql> delete from logins;
Query OK, 32 rows affected (0.00 sec)

Now, log back into the account.

If you’re not comfortable doing it on a command line or do not have an SSH access at all, you can login to your WebHost Manager/WHM and flush the brutes database from there.

Don’t forget to add your IP address or network address to your whitelist list.

Whitelisted IP address or network address will be allowed access when others are lockout in the system should brute force is detected.

Wednesday, 9 September 2015

How can I disable SSLv3 (Poodle) on my cPanel/WHM Server?

To disable the SSLv3 Vulnerability (Commonly referred to as the Poodle Vulnerability) in cPanel/WHM follow these steps:

1)Visit your server's WHM Panel ( https://<yourserversip>:2087 )

2)Navigate to the Apache Configuration Panel of WHM.

3)Scroll down to the 'Include Editor' Section of the Apache Configuration.

4)Click 'Pre Main Include', which will jump to the corresponding section. Via the drop-down selector, choose 'All Versions'.

5)An empty dialog box will appear allowing you to input the needed configuration updates. In this box, copy and paste the following:

SSLHonorCipherOrder On
SSLProtocol -All +TLSv1 +TLSv1.1 +TLSv1.2

6)Check to make sure the added text is correct and click the Update button to proceed.

7)You will receive confirmation that the includes were updated and be prompted to restart Apache. The settings will not take place until you click Restart Apache.

8)You will then see two pages, one showing the status of the restart and one confirming the restart was successful.

9)That's it! Settings are updated and you're Poodle-Free!

Confirm your site is Poodle-Free

1)Open a SSH terminal and copy/paste the following text (remember to replace yourssldomain.com):

openssl s_client -connect yourssldomain.com:443 -ssl3

2)You should see the following in your output:

sslv3 alert handshake failure

If you see that, SSLv3 is disabled.

This article is no longer accurate based on most recent WHM Updates. Please refer to:

https://documentation.cpanel.net/display/CKB/How+to+Adjust+Cipher+Protocols 

What is DNS propagation delay? How can the IP address of a domain be changed with less downtime?

DNS propagation delay can happen due to two reasons.

1)Name server changes at your registrar.
2)DNS caching due to TTL value set up in resource records.

We have very little control on the first case as any changes done at registrar may take 24 to 48 hours to propagate on Internet (though it's generally nowhere near this long). We can manage the downtime of site in the second case.

To reduce the DNS lookup overhead, every name servers cache the DNS records fetched at a time. The time for which these values remain in cache is controlled by the TTL (Time to Live) value set in these resource records. For example, if the A record of the site has a TTL value, say 14400 seconds (4 hours), then this result will stay in the DNS cache for 4 hours once fetched from the authoritative name servers. If the IP address of this site is changed in between, the name server which caches the previous result will not be able to see this change as it is still fetching the details from it's local cache.

So, to reduce the downtime due to DNS caching, we need to reduce the TTL value in resource records. The procedure mentioned below helps to change the IP address of site with less downtime. It assumes that the present TTL value set is 14400 seconds.

1)Change the TTL value to 300 seconds and wait for 14400 seconds for the new TTL value to be effective.

2)Change the IP address of site. This new IP address will be propagated on Internet within 5 minutes (300 seconds) as name servers will cache the result only for 5 minutes.

3)Once the IP address is changed and site is accessible, you may change the TTL value back to 14400 seconds. This will help to reduce the DNS queries on the authoritative name servers.

Checking a server for "test" email accounts

Test email accounts are typically a security vulnerability. Originally made with an initial purpose of testing, many times with a weak or insecure password and then forgotten about. Regularly checking your server for test@domain.tld email accounts is a great security precaution.

Find "test" email accounts on cPanel servers.
=================================

1)Login to your server via SSH as the root user.

2)Copy and paste this nifty one line command into the terminal:

grep -i 'test' /home/*/etc/*/passwd | awk 'BEGIN { FS = "/" } ; { print "test@" $5 " email address exists!"}'

3)If any test email accounts exist, you should see output similar to this:

grep -i 'test' /home/*/etc/*/passwd | awk 'BEGIN { FS = "/" } ; { print "test@" $5 " email address exists!"}'
test@test.com email address exists!

4)You can search for other possible email aliases, such as "testing@domain.tld" instead of test by replacing 'test' with 'testing' in the one liner.

"Cannot stat() mounted device /dev/simfs" error

After cPanel's latest update (11.46.0 Build 12) many people have been receiving notifications or emails containing similar messages:

======================================================
repquota: Cannot stat() mounted device /dev/simfs: No such file or directory
repquota: Cannot stat() mounted device /dev/simfs: No such file or directory
repquota: Cannot stat() mounted device /dev/simfs: No such file or directory
repquota: Cannot stat() mounted device /dev/simfs: No such file or directory
repquota: Cannot stat() mounted device /dev/simfs: No such file or directory
======================================================

This is a normal error in our virtualization environment, and in almost all cases it can safely be ignored. It's actually a result of security hardening.

We can stop this warning email by creating a symlink.

ln -s /dev/vzfs /dev/simfs

==============================
[~]# ll /dev/vzfs
brw-r--r-- 1 root root 0, 22 Sep  3 02:14 /dev/vzfs
 [~]# 
==============================

Also add a cronjob on the server. 

@reboot ln -s /dev/vzfs /dev/simfs

This will help to make this change after the server reboot too. 

Regenerate expiring cPanel service SSL certificates

Self-Signed Certificates are installed on WHM / cPanel servers by default. These certificates expire just like SSL Certificates from a Signed Certificate Authority. You may have received an email from your server, such as "Certificate for services on hostname will expire in less than 30 days". As long as your SSL Certificates are Self-Signed, this procedure will resolve that email notification.

Regenerating Self-Signed Certificates in cPanel / WHM

You'll need root access to your WHM Panel to proceed with regenerating self-signed SSL Certificates. Simply follow these steps:

1)Login to your WHM Panel. ( https://yourserversip:2087 or http://yourserversip:2086 )

2)Navigate to Home » Service Configuration » Manage Service SSL Certificates.

3)Once on this page, you should see a list of cPanel's Services that use SSL Certificates. When clicking on 'Certificate Details', you should also notice Issuer: (self-signed).

4)To regenerate the Self-Signed Certificates, simply select the "Reset Certificate" link option.

5)You'll receive a popup prompting you to Confirm the reset, simply select Proceed.

6)Another popup dialog will be presented requesting that you restart cpsrvd. If you are planning to reset all 4 certificates, select Cancel until you are resetting the last certificate in the list. However, restarting after each reset is perfectly fine as well.

7)Follow steps 1-6 for all Self-Signed Certificates that need to be regenerated.

How to install Nginx as reverse proxy on cPanel server?

The web-hosting control panel’s default web-server is Apache. cPanel/WHM doesn’t have native support on Nginx. But, we can install and configure it may ways. That will improve the server performance.

Nginx as reverse proxy for Apache is good for its speed and ability to handle large number of requests simultaneously with optimal use of resources. The main concept behind this is Nginx will take care of all the static files of your websites (css, images, swf files, mp4, javascript files, and more), and pass the rest of the requests (dynamic requests — php files –) to Apache web server. This is called Nginx reverse proxy, where Nginx stands as front end server,with a Backend server.

One of the good things of this configuration, is that you don’t need to change a single thing in your websites, as your code and .htaccess files are 100% compatible. This is one of the easiest and fastest way to install Nginx on cPanel server.

This are some of the features of this wonderful script:

*Compatible with WHM: allows you to configure and analyze nginx logs with a GUI.

*Gzip compression: it is compatible with Gzip compression, which will allow you to get faster speeds.

*Integrated with the cPanel Service Monitor configuration

You can choose which domains will use Apache and which will use Nginx as reverse proxy.

Installing Nginx on cPanel.

1)Login to your server via SSH (as root).

ssh root@1.1.1.1 -p 1122

2)Once logged in successfully, you'll need to change into the /usr/local/src directory.

cd /usr/local/src

3)Now, simply download the plugin using wget.

wget http://nginxcp.com/latest/nginxadmin.tar

4)Now that it's been downloaded, you need to extract the compressed plugin.

tar xf nginxadmin.tar

5)Change into the publicnginx directory.

cd publicnginx

6)Run the included executable installer program with the 'install' flag.

./nginxinstaller install

If you've never setup a Remote Access Key, you'll see the following error: access key doesn't exist To fix this, you'll need to login to WHM, visit the "Clusters" section, select " Remote Access Key" then select "Generate New Key".

Move to WHM >  Cluster/Remote Access > Setup Remote Access Key and click at “Generate New Key button”

If all goes smoothly, the end result should print something similar to the following:

 ****************************************************
 *               Installation Complete              *
 *run /etc/init.d/httpd restart to start Nginx Admin*
 ****************************************************

7)Restart the webserver daemon.

/etc/init.d/httpd restart

Which should result with output similar to this:

 ****************************************************
Restarting nginx daemon: nginxRemaining processes: 10207
 ****************************************************

NginxCP (nginx as a reverse proxy) is now installed and can be viewed/accessed/configured via WHM. To access the Control Panel for nginx, visit your WHM Panel » Plugins » nginx Admin.

P.S. Make sure that the Nginx is running on port 80 and Apache on different port (like 8080). Also open the new port the server firewall.

That’s it!

IMAP/Exim failing on cPanel server

If you are on cpanel server you might encounter errors related to imap. Below given is an example of error you might encounter.

===========================================================
Service Check Method: [tcp connect]

Failure Reason: TCP Transaction Log:
<< * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT
THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready.
Copyright 1998-2011 Double Precision, Inc. See COPYING for distribution
information.
>> A001 LOGIN
__cpanel__service__auth__imap__rJgtrLJMaRsNRJAy1xrY3gjTDNA852bfXgTyA8ttThGb3PWtney7UFHfeEuyP9bg
EFRrpZNcstXly7t6cq5gQmthW7w4Qd4xlUeldCDdm67tiIsq_6efRRsLOy3WpZA3
<< * BYE Temporary problem, please try again later
imap: ** [* BYE Temporary problem, please try again later != A001 OK]
: Died at /usr/local/cpanel/Cpanel/TailWatch/ChkServd.pm line 541, line
2.
===========================================================

Error message logged in ” /var/log/maillog”.

===========================================================
/var/log/maillog:Jan 30 20:10:04 s307 imapd-ssl: authentication error: Input/output error
===========================================================

The issue can be fixed by increasing the authentication daemons for courier in the server.

Follow the below given steps:

Login to WHM

Service configuration

Mail server configuration

Number of authentication Daemons (Increase the current value to higher).

Tuesday, 8 September 2015

How to Install and Configure maldet (Linux Malware Detect – LMD)

This is one of the commonly using Malware detector in Linux servers. Linux Malware Detect (LMD) is a malware scanner for Linux released under the GNU GPLv2 license, that is designed around the threats faced in shared hosted environments. It uses threat data from network edge intrusion detection systems to extract malware that is actively being used in attacks and generates signatures for detection.

In addition, threat data is also derived from user submissions with the LMD checkout feature and from malware community resources. The signatures that LMD uses are MD5 file hashes and HEX pattern matches, they are also easily exported to any number of detection tools such as ClamAV.

To install LMD, download the package and run the enclosed install.sh script

Download maldetect package using wget

root@server[~]# wget http://www.rfxn.com/downloads/maldetect-current.tar.gz

Installation steps are very simple and easy to do. Follow the steps below to install maldet on your server.

Step I: SSH to your server
Step II: Download the tar file and install it.

#cd /usr/local/src/
# wget http://www.rfxn.com/downloads/maldetect-current.tar.gz
# tar -xzvf maldetect-current.tar.gz
# cd maldetect-*
# sh install.sh

Now that LMD is installed, you need to open the configuration file located at /usr/local/maldetect/conf.maldet. The configuration file is fully commented so you should be able to make out most options but lets take a moment to review the more important ones anyways.

email_alert
This is a top level toggle for the e-mail alert system, this must be turned on if you want to receive alerts.

email_addr
This is a comma spaced list of e-mail addresses that should receive alerts.

quar_hits
This tells LMD that it should move malware content into the quarantine path and strip it of all permissions. Files are fully restorable to original path, owner and permission using the –restore FILE option.

quar_clean
This tells LMD that it should try to clean malware that it has cleaner rules for, at the moment base64_decode and gzinflate file injection strings can be cleaned. Files that are cleaned are automatically restored to original path, owner and permission.

quar_susp
Using this option allows LMD to suspend a user account that malware is found residing under. On CPanel systems this will pass the user to /scripts/suspendacct and add a comment with the maldet report command to the report that caused the users suspension (e.g: maldet –report SCANID). On non-cpanel systems, the users shell will be set to /bin/false.

quar_susp_minuid
This is the minimum user id that will be evaluated for suspension, the default should be fine on most systems.

The rest of the options in conf.maldet can be left as defaults unless you clearly understand what they do and how they may influence scan results and performance.

Usage & Manual Scans
The usage of LMD is very simple and there is a detailed –help output that provides common usage examples, I strongly recommend you check the –help output and spend a few minutes reviewing it.

The first thing most users are looking to do when they get LMD installed is to scan a certain path or series of paths.

[An important note is that LMD uses the ‘?’ character for wildcards instead of the ‘*’ char.]

In the below examples I will be using the long form flags but they are interchangeable with the short form flags (i.e: –scan-recent vs. -r).

If we wanted to scan all user public_html paths under /home*/ this can be done with:

root@server[~]# maldet –scan-all /home?/?/public_html
If you wanted to scan the same path but scope it to content that has been created/modified in the last 5 days you would run:

root@server[~]# maldet –scan-recent /home?/?/public_html 5
If you performed a scan but forget to turn on the quarantine option, you could quarantine all malware results from a previous scan with:

root@server[~]# maldet –quarantine SCANID
Similarly to the above, if you wanted to attempt a clean on all malware results from a previous scan that did not have the feature enabled, you would do so with:

root@server[~]# maldet –clean SCANID
If you had a file that was quarantined from a false positive or that you simply want to restore (i.e: you manually cleaned it), you can use the following:

root@server[~]# maldet –restore config.php.2384
root@server[~]# maldet –restore /usr/local/maldetect/quarantine/config.php.2384
Once again, I encourage you to fully review the –help output for details on all options and the README file for more details on how LMD operates.

You can also do the Daily scan by setting the cron :-

Daily Scans.

The cronjob installed by LMD is located at /etc/cron.daily/maldet and is used to perform a daily update of signatures, keep the session, temp and quarantine data to no more than 14days old and run a daily scan of recent file system changes.

The daily scan supports Ensim virtual roots or standard Linux /home*/user paths, such as Cpanel. The default is to just scan the web roots daily, which breaks down as /home*/*/public_html or on Ensim /home/virtual/*/fst/var/www/html and /home/virtual/*/fst/home/*/public_html.

Fix – WordPress XMLRPC Vulnerability

While checking the server I noticed increased number of requests to xmlrpc.php.

192.168.1.1- - - [29/Jul/2015:02:06:47 -0500] "POST /en/xmlrpc.php HTTP/1.1" 403 1324 "http://www.yourdomain.com/" "PHP/5.3.04"
192.168.1.1- - - [29/Jul/2015:02:08:46 -0500] "POST /en/xmlrpc.php HTTP/1.0" 403 1324 "http://www.yourdomain.com/" "PHP/5.3.04"
192.168.1.1-- - [29/Jul/2015:02:31:02 -0500] "POST /xmlrpc.php HTTP/1.1" 403 1324 "http://www.yourdomain.com" "PHP/5.3.99"
192.168.1.1- - [29/Jul/2015:02:46:02 -0500] "POST /en/xmlrpc.php HTTP/1.1" 403 1324 "http://www.yourinspirationweb.com/" "PHP/5.3.86"
192.168.1.1 - - [29/Jul/2015:02:41:58 -0500] "POST /xmlrpc.php HTTP/1.1" 403 1324 "http://www.yourinspirationweb.com/" "PHP/5.2.16"

I found that every single of those requests took lots of ram and resulted in system instability and in few occasions it caused the server to go out of memory and eventually crashed the account. And the problem is since WordPress 3.5 you can’t disable the use of xmlrpc, at least not from the WordPress control panel.

There are many ways to do that and I’ll write some:

1. Deleting xmlrpc.php file.

This is really not recommended. Also after WordPress (auto)update the deleted file will be replaced so it’s not really smart to do this, but I just wanted to write this just in case someone doesn’t try to do this.

2. Plugins

There are several plugins that can do this. I found these two to be the most used ones: Disable XML-RPC and XML-RPC Pinkback. Both plugins are really basic (only couple lines of code) but they should be able to help you out and protect your blog against those attacks.

https://wordpress.org/plugins/disable-xml-rpc-pingback/
https://wordpress.org/plugins/disable-xml-rpc/

3. Adding filter to theme functions.php file.

This is basically same thing as the plugin above, but you have one plugin less. All you need to do is to edit your theme’s functions.php and add these couple of lines:

function remove_x_pingback($headers) {
    unset($headers['X-Pingback']);
    return $headers;
}
add_filter('wp_headers', 'remove_x_pingback');
add_filter('xmlrpc_enabled', '__return_false');

4. Block access at .htaccess.

You can simply add this one line of code to your .htaccess file and block the access to the xmlrpc.php file entirely. User accessing the xmlrpc.php will get the 403 Forbidden error.

<Files xmlrpc.php>
Order Deny,Allow
Deny from all
</Files>

5. Blocking access in nginx.

If you are running nginx instead of Apache you should add this code to your nginx configuration:

server {
    location = /xmlrpc.php {
        deny all;
    }
}

6. Block on entire server.

If you have one server or VPS with tens of hundreds of WordPress installations (like me) any of the solutions above will take time to implement. So the best thing to do is to block access to xmlrpc.php file on Apache level, simply by adding this to httpd.conf file:

<FilesMatch "^(xmlrpc\.php)">
Order Deny,Allow
Deny from all
</FilesMatch>

Or even better adding this code (that also blocks wp-trackback.php and also prevent’s trackback hacking attempts).

<FilesMatch "^(xmlrpc\.php|wp-trackback\.php)">
Order Deny,Allow
Deny from all
</FilesMatch>

If you don’t use XML-RPC than you can safely disable it using any of the methods above (except the first one, of-course) and protect your blog against xmlrpc hacks.

Find outdated versions of WordPress,Joomla and Drupal on your server

If you have a server and many sites running on it, it is difficult that to find which accounts are using the outdated WordPress/Joomla. The below scripts helps you to find outdated versions of WordPress and Joomla on your server.

cPanel Server:

Find outdated versions of WordPress.

find /home/*/public_html/ -type f -iwholename "*/wp-includes/version.php" -exec grep -H "\$wp_version =" {} \;

Find outdated versions of Joomla.

find /home/*/public_html/ -type f \( -iwholename '*/libraries/joomla/version.php' -o -iwholename '*/libraries/cms/version.php' -o -iwholename '*/libraries/cms/version/version.php' \) -print -exec perl -e 'while (<>) { $release = $1 if m/ \$RELEASE\s+= .([\d.]+).;/; $dev = $1 if m/ \$DEV_LEVEL\s+= .(\d+).;/; } print qq($release.$dev\n);' {} \; && echo "-"

Find Outdated Drupal Versions.

find /home/*/public_html/ -type f -iwholename "*/modules/system/system.info" -exec grep -H "version = \"" {} \;

Plesk Server:

Find outdated versions of WordPress.

find /var/www/vhosts/*/httpdocs/ -type f -iwholename "*/wp-includes/version.php" -exec grep -H "\$wp_version =" {} \;

Find outdated versions of Joomla.

find /var/www/vhosts/*/httpdocs/ -type f \( -iwholename '*/libraries/joomla/version.php' -o -iwholename '*/libraries/cms/version.php' -o -iwholename '*/libraries/cms/version/version.php' \) -print -exec perl -e 'while (<>) { $release = $1 if m/ \$RELEASE\s+= .([\d.]+).;/; $dev = $1 if m/ \$DEV_LEVEL\s+= .(\d+).;/; } print qq($release.$dev\n);' {} \; && echo "-"

Find Outdated Drupal Versions.

find /var/www/vhosts/*/httpdocs -type f -iwholename "*/modules/system/system.info" -exec grep -H "version = \"" {} \;

Make sure any applications you use are kept up-to-date and limit the use of third-party plug-in's where possible as they can be a source of many issues and may be updated less frequently or created by unscrupulous publishers. If you are writing your own code, be sure to validate your input fields for special characters and ensure you are checking for this type of hacking in your database procedures called from the website.If you are using a database driven program (e.g. WordPress, Joomla, OSCommerce), then all you need to do is upgrade your programs to the latest version available.

Most software that users run on the websites are Open Source software. Open Source software is software that is freely available for anyone to download and use. For example, both Joomla and Wordpress are very commonly used, and they are both Open Source. One of the drawbacks of Open Source software is that anyone can download and view the software's code, which makes it easier for hackers to find ways to compromise a website. The authors of such Open Source Applications release updates and security patches on a regular basis. Please be sure that you are running the most current versions of any third party software on your website, as the most current version is usually the most secure version as well.

The following is a list of links, for Wordpress and Joomla specifically, that point to the software's own information about security:

WordPress
------------------
Wordpress.org - How to Keep WordPress Secure
http://wordpress.org/development/2009/09/keep-wordpress-secure/

Wordpress.org - Hardening WordPress
http://codex.wordpress.org/Hardening_WordPress

Wordpress.org - Upgrading Wordpress
http://codex.wordpress.org/Upgrading_WordPress

Joomla
------------------
The Joomla Security Center includes information about their latest security news, their latest security articles, and more information in general about the Joomla Security Strike Team.

Joomla.org - Joomla Security Center
http://developer.joomla.org/security.html

Joomla.org - Upgrade Instructions
http://docs.joomla.org/Upgrade_Instructions

Joomla.org - Vulnerable Extensions List
http://docs.joomla.org/Vulnerable_Extensions_List

Drupal
------------------
drupal.org  - Drupal  Security Center
https://www.drupal.org/security

drupal.org  - Drupal Downloads
https://www.drupal.org/project/drupal

Keeping any third party plugins / extensions on your website up to date is just as important as keeping the core software up to date as well.

Also You will need to make sure your personal computer is up to date for all software and specifically including:

- Adobe Acrobat Reader
- Adobe Flash Player
- Adobe Shockwave
- Any FTP Program including Filezilla FTP and WS_FTP

Any computer running insecure and outdated software is vulnerable to security issues. Keeping the software running on your computer up to date and regularly changing your passwords are the best precautions to take. 

It is also very possible that your software has been updated already and the attempted hack was possible because some time in the past your personal computer had a combination of software that was not secure. At that time, the method the hackers used would find your FTP username and password from your files and send it from your personal computer out to a repository they set up for future use.

One of the more commonly used exploitable programs is Adobe's Acrobat Reader. Adobe has released security advisories on their website, including information on how to update the version you are running to the latest stable and secure release. You can reach Adobe's Security bulletins and advisories webpage via the following link:

http://www.adobe.com/support/security/

You should also immediately reset your cPanel password (which is your FTP password as well) to a secure password that is at least seven characters long, and uses a combination of letters, numbers, and special characters. You can reset your cPanel password within the "Change Password" section of your cPanel.

Enable DKIM and SPF for all accounts in cPanel

Email authentication lets you verify the origin of an email message sent to or from your domain. This feature assists in combating spam, or unwanted email.

What is DKIM?

DKIM (DomainKeys Identified Mail) provides a way to verify the sender and integrity of a message. It allows an email system to prove that a message was not altered during transit (meaning it is not forged), and that the message came from the specified domain.

Enable or disable DKIM

    Click Enable to enable DKIM.
    Click Disable to disable DKIM.

Note: DKIM is only effective if enabled on the authoritative DNS server for the zone.

What is SPF?

Sender Policy Framework (SPF) allows a recipient server to verify that an email message was truly sent from the domain specified in the From: field. Enabling SPF can prevent your server from receiving replies to spam that forged your domain name as part of the sender’s address (also known as “backscatter”).

SPF works by specifying which servers are authorized to send email from your domain. Only mail sent through an authorized server will appear as valid when a recipient checks a message’s SPF records.
Enable or disable SPF

Click Enable to enable SPF or click Disable to disable SPF.

Note: SPF is only effective if enabled on the authoritative DNS server for the zone and if both the sending and receiving mail servers have SPF enabled.

Change advanced SPF settings

Click Add or Remove to add or remove additional hosts, servers, and IP blocks in the corresponding lists.

Click the All Entry (ALL) box if you have entered all hosts that will send to your domain and wish to exclude all other domains.

Click the Overwrite Existing Entries box if you wish to overwrite all existing SPF records for all your domains with these settings.

If you need to enable DKIM and SPF for all accounts in cPanel, it is not possible to manually enable them via cPanel. In that case you can use the following script.

for user in `ls -A /var/cpanel/users` ; do /usr/local/cpanel/bin/dkim_keys_install $user &&/usr/local/cpanel/bin/spf_installer $user ; done

That’s it!

How to change the file/directory permission using find command?

The find command is used to locate files on a Unix or Linux system. You can search for files by name, owner, group, type, permissions, date, and other criteria.

You can change the permissions of all files and directories using the following commands.

For Files:

find . -type f -exec chmod 644 {} \;

For directories:

find . -type d -exec chmod 755 {} \;

How to Fix an SSH Lockout on a VPS or Dedicated Server w/cPanel

If you're unable to access your server via SSH because of a mistake made to your SSH configuration file, it is simple to reset your SSH configuration file back to defaults if cPanel is installed on your server.

All you need to do is append the following to your WHM URL and then log in using your root log in details when prompted:

/scripts2/doautofixer?autofix=safesshrestart

Substitute the server IP where required and take this in the web browser.

http://your_server_ip:2086/scripts2/doautofixer?autofix=safesshrestart

You will be able to login using SSH now

Enable slow query log in mysql

When its comes to optimizing and tuning Mysql the most important aspect is to identify the inefficient/slow queries.

So the question arises how we can find the queries which are taking long time to execute so we can optimize/improve them to improve the overall performance.

Mysql helps us with its built in support for logging slow queries.

Activating the slow query logging :

We need check if slow query loggin is already enabled or not , it can be checked as below:

mysqladmin var |grep log_slow_queries
| log_slow_queries | OFF

If its already set to ON then you are set, if its set to OFF like above then you will need to enable slow query logging.

The mysql variable long_query_time (default 1) defines what is considered as a slow query. In the default case, any query that takes more than 1 second will be considered a slow query.
Now to enable the slow query logging we will need following entries in the /etc/my.cnf mysql configuration file.

[mysqld]
long_query_time = 1
log-slow-queries = /var/log/mysql/mysql-slow.log

You can define the path for logging according to your requirements. Also the log query time which is by default 1 sec can be adjusted according to your needs.

Once you have done the configuration, restart mysql service to load the new configurations.

Once slow query logging is enabled we can check the log file for each slow query that was executed by the server.

Different details are logged to help you understand how was the query executed:

Time:  the time it took to execute the query
Lock:  how long was a lock required
Rows: how many rows were investigated by the query
Host: this is the actual host that launched/initiated the query
Query: The actual mysql query.

This information will help us to see what queries need to be optimized.

Passwd Infected Chkrootkit

I noticed the following result for chkrootkit:

-----------------------------------
# ./chkrootkit | grep -v not
ROOTDIR is `/'
Checking `passwd'... INFECTED
Checking `aliens'... no suspect files
Checking `bindshell'... INFECTED (PORTS:  465)
-----------------------------------

cPanel forum update:

It's very likely a false positive, however you may want to review your system for any additional signs of an exploit. Check the md5sum of the /bin/passwd file (it should be a symbolic link to /usr/local/cpanel/bin/jail_safe_passwd) to see if it matches up with what's provided by cPanel.

Steps followed:
===========================
Get "passwd" file from official cPanel link:

# wget http://httpupdate.cpanel.net/cpanelsync/11.50.0.30/binaries/linux-c6-x86_64/bin/jail_safe_passwd.bz2

# bunzip2 jail_safe_passwd.bz2

Check the md5sum:
+++++++
]# md5sum jail_safe_passwd
bddb53aea267eeb2550af8bde5b55a87  jail_safe_passwd
+++++++
# md5sum /bin/passwd
bddb53aea267eeb2550af8bde5b55a87  /bin/passwd
 [/usr/local/chkrootkit]#
+++++++

If there is any mismatch please check the file "/bin/passwd".

Apache: No space left on device: Couldn’t create accept lock

Apache went down without a problem, but it refused to start properly, and didn’t bind to any ports. Within the Apache error logs, this message appeared over and over:

[emerg] (28)No space left on device: Couldn't create accept lock

 If this happens to you, check these items in order:

1. Check your disk space.

2. Review filesystem quotas

3. Clear out your active semaphores

Semaphores are used for communicating between the active processes of a certain application. In the case of Apache, they’re used to communicate between the parent and child processes. If Apache can’t write these things down, then it can’t communicate properly with all of the processes it starts.

Run this command as root:

# ipcs -s

If you see a list of semaphores, Apache has not cleaned up after itself, and some semaphores are stuck. Clear them out with this command:

# for i in `ipcs -s | awk '/httpd/ {print $2}'`; do (ipcrm -s $i); done

Now, in almost all cases, Apache should start properly. If it doesn’t, you may just be completely out of available semaphores. You may want to increase your available semaphores, and you’ll need to tickle your kernel to do so. Add this to /etc/sysctl.conf:

kernel.msgmni = 1024
kernel.sem = 250 256000 32 1024

And then run sysctl -p to pick up the new changes.

CSF Advanced Allow/Deny Filters

In /etc/csf.allow and /etc/csf.deny you can add more complex port and ip filters using the following format (you must specify a port AND an IP address):

tcp/udp|in/out|s/d=port|s/d=ip|u=uid

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
tcp/udp : EITHER tcp OR udp OR icmp protocol
in/out : EITHER incoming OR outgoing connections
s/d=port : EITHER source OR destination port number (or ICMP type)
(use a _ for a port range, e.g. 2000_3000)
s/d=ip : EITHER source OR destination IP address
u/g=UID : EITHER UID or GID of source packet, implies outgoing connections, s/d=IP value is ignored
Note: ICMP filtering uses the “port” for s/d=port to set the ICMP type. Whether you use s or d is not relevant as either simply uses the iptables –icmp-type option. Use “iptables -p icmp -h” for a list of valid ICMP types. Only one type per filter is supported
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Examples:

# TCP connections inbound to port 3306 from IP 11.22.33.44
tcp|in|d=3306|s=11.22.33.44
# TCP connections outbound to port 22 on IP 11.22.33.44
tcp|out|d=22|d=11.22.33.44

How to fix Openssl Heartbleed vulnerability

What’s Heartbleed vulnerability (CVE-2014-0160)?

A serious OpenSSL vulnerability has been found, and is named Heartbleed and it affected all servers running OpenSSL versions from 1.0.01 to 1.0.1f. This vulnerability can be used to get the Private key of a SSL connection, so it is important to update / patch your server immediately. This bug is fixed in OpenSSL version 1.0.1g. All major Linux Distros have already released updates for Hartbleed vulnerability.

How to find out if your server is affected from Openssl Heartbleed vulnerability (CVE-2014-0160)?

Login to your SSH and execute following command to get the installed version number of OpenSSL:

openssl version

The result should be something like this:

openssl version
OpenSSL 1.0.1e 11 Feb 2013

If the version is below 1.0.1g your server might be vulnerable and you should patch it (see how below).
If your server is using a 0.9.8 release like it is used on Debian squeeze, then the server is not vulnerable as the HeartBleed function has been implemented in OpenSSL 1.0.1 and later versions only.

openssl version
OpenSSL 0.9.8o 01 Jun 2010

Fixing the Heartbleed vulnerability
CentOS and Fedora:

yum update
Ubuntu and Debian:

apt-get update
apt-get upgrade

OpenSUSE:

zypper update

Ok, now what?
After this you should restart all the services using OpenSSL but better idea is to restart the whole server just in case.

You can also verify on following site if you successfully closed the Heartbleed security hole on your server: http://filippo.io/Heartbleed/

Sunday, 6 September 2015

Spam Check Commands On cPanel Servers

Top 5 users sending maximum emails
============================================
grep "<=.*P=local" /var/log/exim_mainlog | awk '{print $6}' | sort | uniq -c | sort -nr | head -5
eximstats /var/log/exim_mainlog | grep -A7 "Top 50 local senders by message count" | tail -5 | awk '{print $1,$NF}'

Top 5 mail receivers:
============================================
egrep "(=>.*T=virtual_userdelivery|=>.*T=local_delivery)" /var/log/exim_mainlog | awk '{print $7}' | sort | uniq -c | sort -nr | head -5
eximstats /var/log/exim_mainlog | grep -A7 "Top 50 local destinations by message count" | tail -5 | awk '{print $1,$NF}'

Script to check path for the script used for spamming
============================================
awk '{ if ($0 ~ "cwd" && $0 ~ "home") {print $3} }' /var/log/exim_mainlog | sort | uniq -c | sort -nk 1
awk '{ if ($0 ~ "cwd" && $0 ~ "home") {print $4} }' /var/log/exim_mainlog | sort | uniq -c | sort -nk 1

If there is large number of hits from an IP,block the IP
============================================
tail -n1000 /var/log/exim_mainlog |grep SMTP|cut -d[ -f2|cut -d] -f1|sort -n |uniq -c

Following command will show you the maximum no of email currently in the mail queue have from or to the email address in the mail queue with exact figure.
============================================
exim -bpr | grep "<*@*>" | awk '{print $4}'|grep -v "<>" | sort | uniq -c | sort -n

That will show you the maximum no of email currently in the mail queue have for the domain or from the domain with number.
============================================
exim -bpr | grep "<*@*>" | awk '{print $4}'|grep -v "<>" |awk -F "@" '{ print $2}' | sort | uniq -c | sort -n
v
Following command will show path to the script being utilized to send mail
============================================
ps -C exim -fH eww
ps -C exim -fH eww | grep home
cd /var/spool/exim/input/
egrep "X-PHP-Script" * -R

Command to delete frozen mails
============================================
exim -bp | awk '$6~"frozen" {print $3 }' | xargs exim -Mrm

If anyone is spamming from /tmp
============================================
tail -f /var/log/exim_mainlog | grep /tmp

To display the IP and no of tries done the IP to send mail but rejected by the server.
============================================
tail -3000 /var/log/exim_mainlog |grep 'rejected RCPT' |awk '{print$4}'|awk -F\[ '{print $2} '|awk -F\] '{print $1} '|sort | uniq -c | sort -k 1 -nr | head -n 5

Shows the  connections from a certain ip to the   SMTP server
============================================
netstat -plan|grep :25|awk {‘print $5′}|cut -d: -f 1|sort|uniq -c|sort -nk 1

To shows the domain name and the no of emails in queue
============================================
exim -bp | exiqsumm | more

If  spamming from outside domain then you can block that domain or email id on the server
============================================
pico /etc/antivirus.exim
Add the following lines:
if $header_from: contains “name@domain.com” then seen finish endif

Check mail stats
============================================
exim -bp | exiqsumm | more

Check if any php script is causing the mass mailing with
============================================
cd /var/spool/exim/input
egrep “X-PHP-Script” * -R

Just cat the ID that you get and you will be able to check which script is here causing problem for you.

To Remove particular email account email
============================================
exim -bpr |grep “test.org”|awk {‘print $3′}|xargs exim -Mrm
---------------------------------------------------------------------------------------------------------------------

1) Script to check how many emails sent for an account for particular date

exigrep  name@domain.com /var/log/exim_mainlog|grep 2015-08-27 |grep Completed|wc -l

2)Search for emails for particular time.

cat /var/log/exim_mainlog  | grep "2013-04-22 06:" | grep "<= name@domain.com" | wc -l

3)Command to check whether there is any PHP scripts sending emails under any account.

ps -C exim -fH eww
ps -C exim -fH eww | grep home
cd /var/spool/exim/input/
egrep "X-PHP-Script" * -R

4)Command to check the mailing scripts.

awk '{ if ($0 ~ "cwd" && $0 ~ "home") {print $3} }' /var/log/exim_mainlog | sort | uniq -c | sort -nk 1

grep "cwd=" /var/log/exim_mainlog|awk '{for(i=1;i<=10;i++){print $i}}'|sort|uniq -c|grep cwd|sort -n

# count of all messages in queue

exim -bpc

# a list of message in the queue (time queued, message size, message id,
sender, recipient)

exim -bp

Example output
1   4d  1.2K 1Ka6u5-00032Z-Eb <from@example.com>
2             to@example.com
3  
4   62h  1.2K 1KaRH0-0007QZ-B5 <from@example.com>
5             to@example.com
6  
7   3h   22K 1KbLHr-0004ev-An <from@example.com>
8             to@example.com

#Finding the files with the find command

find /var/spool/exim -name "1Ka6u5-00032Z-Eb*"

# lists messages from a specified sender

exiqgrep -f [user]@domain

# lists messages to a specified recipient

exiqgrep -r [user]@domain

# List all queued messages, grouped by sender address

exim -bpr | grep -Eo "<[^ ]*@[^ ]*>" | sort | uniq -c

# List all queued messages, grouped by recipient address

exim -bpr | grep -Eo "^\s*[^ ]*@[^ ]*$" | sort | uniq -c

# Remove all messages older than 12hrs (43000 seconds)

exiqgrep -o 43000 -i | xargs exim -Mrm

# Remove all frozen messages from the queue

exiqgrep -z -i | xargs exim -Mrm

# Remove all messages from a particular sender

exiqgrep -i -f name@domain.com | xargs exim -Mrm

# Remove all messages from a sender that are older than 12hrs

exiqgrep -o 43000 -i -f name@domain.com | xargs exim -Mrm

#Processing all messages in queue to force deliver

exim -qff