Question : Intrusion Attempt on a Debian Etch system. Help needed

Dear all,

I am running a very secure (I think) Debian 4 system, with a monolithic kernel (no module support) and with grsec patched:  2.6.24.4-grsec
System is also running webmin and virtualmin.
Iptables are running too.
SSH is running on a non-standard port and access to it is allowed only from one specific IP (my office).
Ossec is running too.

A few days ago I got this email alert from ossec:
OSSEC HIDS Notification.
2010 Jun 05 23:30:03

Received From: server_name->/var/log/syslog
Rule: 1002 fired (level 2) -> "Unknown problem somewhere in the system."
Portion of the log(s):

Jun  5 23:30:01 server_name kernel: grsec: denied resource overstep by requesting 216629248 for RLIMIT_STACK against limit 8388608 for /[ps:24713] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[sh:24711] uid/euid:0/0 gid/egid:0/0

 --END OF NOTIFICATION

I am VERY concerned with the last line that (if I am not mistaken) it indicates that bash shell with root credentials (!!!) fired the -what-I-assume-to-be buffer overflow...

last command does not report anything suspicious either on the current or on the previous wtmp. (system does automatic archiving).

SSH root access is set to:
PermitRootLogin forced-commands-only
and this corresponds to an rsync backup entry

I would be most obliged if you could point me to a direction of further analyzing the system and finding what exactly happened. I am willing to install any tools you might suggest to closely monitor the system for future attempts at a fine-grain level.

Thanks to you all

Answer : Intrusion Attempt on a Debian Etch system. Help needed

The problem in my optinion are the scripts running care of Webmin/Virtualmin.
You have hardened the system so probably when the cron runs generates output of the failed shell commands.
Please take a closer look at /var/log/autdit.log for failed login attempts.
You caould also try installing "DenyHosts" is a perl daemon that checks for a number specified of failed ssh logins and puts automatically the attacker Ip address in /etc/hosts.deny.

We have install it on more that 16 servers and we actually have more that 160.000 ip addresses automatically blocked.

If you need further help i am here to assist you.
Regards