in general

programs = processes = tasks = job

less is more (security)

run as little software as you absolutely need – uninstall/disable all services you don’t need.

less software = less lines of mistaken code = less security flaws.

if you need a software or service run it as non-root user – so if it gets buffer-overflowed and remote-code run on your cpu … atleast it would not be run with root-privileges – limiting the damage.

usermod -a -G sudo bob; # allows user bob to run processes with root-privileges temporarily

updates

updates mean changes – that could tune or break your systems functionality.

so it’s wise to have (as easy as possible to restore) backups.

if you have no time: Ideally you powerdown – snapshot – powerup – update – test your system.

I know downtime is not an option 😀

if your life depends on it: so 100% professional would be to clone the live system to a test-server – test if updating breaks any functionality – if not – snapshot and update the real-server.

Yeah digital information come with more overhead than expected. Complexity has grown since morse-terminals.

open ports

check what ports are open and used by what process

su; # become root

lsof -i -P -n; # list all open ports and listening services/programs

netstat -a; # list status of all current connections (also just locally used sockets)

netstat -plnt; # alternative way

apt-get install nmap; # install port scanner

nmap localhost; # scan localhost for open ports

Starting Nmap 6.47 ( http://nmap.org ) at 2017-06-09 14:23 CEST
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000014s latency).
Other addresses for localhost (not scanned): 127.0.0.1 127.0.0.1 127.0.0.1
Not shown: 994 closed ports
PORT STATE SERVICE
22/tcp open ssh
25/tcp open smtp
111/tcp open rpcbind
139/tcp open netbios-ssn
445/tcp open microsoft-ds
2049/tcp open nfs

# if you scan this host from “outside”, you see port 25 seems not to be reachable, maybe blocked by firewall

22/tcp open ssh
111/tcp open rpcbind
139/tcp open netbios-ssn
445/tcp open microsoft-ds
2049/tcp open nfs

netstat -npl |grep 22; # you can then nestat through that list to find out what process is behind that port on localhost

tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 639/sshd
tcp6 0 0 :::22 :::* LISTEN 639/sshd

# so you see the sshd service with pid 639 is listening on that port

lsof -i :22; # alternatively you can check what binary is using that port with lsof

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
sshd 643 root 3u IPv4 12621 0t0 TCP *:ssh (LISTEN)
sshd 643 root 4u IPv6 12630 0t0 TCP *:ssh (LISTEN)

netstat -npl |grep 2049; # it kind of fails to detect the nfsd, maybe because it is compiled into the kernel

ps uax|grep nfs; # but it is installed and running
root 631 0.0 0.0 0 0 ? S< 11:42 0:00 [nfsiod]
root 700 0.0 0.0 0 0 ? S< 11:42 0:00 [nfsd4]
root 701 0.0 0.0 0 0 ? S< 11:42 0:00 [nfsd4_callbacks]
root 706 0.0 0.0 0 0 ? S 11:42 0:00 [nfsd]

service nfs-kernel-server status
● nfs-kernel-server.service – LSB: Kernel NFS server support
Loaded: loaded (/etc/init.d/nfs-kernel-server)
Active: active (running) since Fri 2017-06-09 14:57:50 CEST; 1h 1min ago

nmap -O suse12; # try to guess the OS of the target… fails miserably 😀 but just that you know this now useless option exists…

Starting Nmap 6.40 ( http://nmap.org ) at 2017-06-09 14:07 CEST
Warning: OSScan results may be unreliable because we could not find at least 1 open and 1 closed port
Aggressive OS guesses: Linux 2.6.26 – 2.6.35 (94%), Linux 2.6.32 – 3.6 (94%), Linux 3.0 (94%), Linux 2.6.32 (94%), Linux 2.6.32 – 3.9 (93%), Linux 3.0 – 3.9 (93%), Linux 2.6.23 – 2.6.38 (91%), Linux 3.8 (91%), Crestron XPanel control system (90%), Netgear DG834G WAP or Western Digital WD TV media player (90%)
No exact OS matches for host (test conditions non-ideal).
Nmap done: 1 IP address (1 host up) scanned in 8.69 seconds

apt-get install socket; # install program socket

socket -sl 80; # simulate a webserver on port 80 and output communication on screen

# if you use a webserver to access that port on that machine you will get:

GET / HTTP/1.1
Host: 172.20.0.12
User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:53.0) Gecko/20100101 Firefox/53.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Upgrade-Insecure-Requests: 1

# does not work so well with SSL port 443 because that’s binary protocol

what processes have SUID bit set?

usually files such as /etc/shadow are not readably by non-root users – except they are allowed to start processes with SUID bit set – that can read the file.

so i have read – it is important to regularly scan for binaries that have the SUID set (you could mail yourself a list… or compare with the last scan and only mail a report if something changed)

root@Debian8:~# find /usr/bin -perm +4000; # search for binaries that have the SUID(SuperUserID-Bit) set.

find /usr/bin -perm -u+s; # does the same

# which allows normal users to run them as the owner of the file (root)
root@Debian8:~# find /usr -perm -u+s
/usr/bin/chfn
/usr/bin/sudo
/usr/bin/procmail
/usr/bin/passwd
/usr/bin/chsh
/usr/bin/gpasswd
/usr/bin/pkexec
/usr/bin/X
/usr/bin/at
/usr/bin/newgrp
/usr/sbin/exim4
/usr/lib/eject/dmcrypt-get-device
/usr/lib/dbus-1.0/dbus-daemon-launch-helper
/usr/lib/openssh/ssh-keysign
/usr/lib/policykit-1/polkit-agent-helper-1

[root@CentOS7 ~]# find /usr -perm -u+s
/usr/bin/chfn
/usr/bin/chage
/usr/bin/gpasswd
/usr/bin/newgrp
/usr/bin/chsh
/usr/bin/sudo
/usr/bin/mount
/usr/bin/su
/usr/bin/umount
/usr/bin/crontab
/usr/bin/pkexec
/usr/bin/passwd
/usr/bin/Xorg
/usr/bin/fusermount
/usr/sbin/pam_timestamp_check
/usr/sbin/unix_chkpwd
/usr/sbin/usernetctl
/usr/sbin/userhelper
/usr/lib/polkit-1/polkit-agent-helper-1
/usr/lib64/dbus-1/dbus-daemon-launch-helper
/usr/libexec/abrt-action-install-debuginfo-to-abrt-cache

suse12:~ # find /usr -perm -u+s
/usr/lib/polkit-1/polkit-agent-helper-1
/usr/lib/libgnomesu/gnomesu-pam-backend
/usr/bin/opiepasswd
/usr/bin/sudo
/usr/bin/mount
/usr/bin/su
/usr/bin/umount
/usr/bin/fusermount
/usr/bin/chfn
/usr/bin/chsh
/usr/bin/expiry
/usr/bin/gpasswd
/usr/bin/newgrp
/usr/bin/passwd
/usr/bin/rcp
/usr/bin/rlogin
/usr/bin/rsh
/usr/bin/at
/usr/bin/pkexec
/usr/bin/crontab

what processes have GUID set?

root@Debian8:~# find /usr -perm -g+s
/usr/local
/usr/local/bin
/usr/local/games
/usr/local/include
/usr/local/share
/usr/local/share/emacs
/usr/local/share/emacs/site-lisp
/usr/local/share/man
/usr/local/share/fonts
/usr/local/share/ca-certificates
/usr/local/share/xml
/usr/local/share/xml/declaration
/usr/local/share/xml/schema
/usr/local/share/xml/entities
/usr/local/share/xml/misc
/usr/local/share/sgml
/usr/local/share/sgml/dtd
/usr/local/share/sgml/declaration
/usr/local/share/sgml/entities
/usr/local/share/sgml/misc
/usr/local/share/sgml/stylesheet
/usr/local/etc
/usr/local/src
/usr/local/sbin
/usr/local/lib
/usr/local/lib/python2.7
/usr/local/lib/python2.7/site-packages
/usr/local/lib/python2.7/dist-packages
/usr/bin/procmail
/usr/bin/mutt_dotlock
/usr/bin/ssh-agent
/usr/bin/mlocate
/usr/bin/chage
/usr/bin/dotlockfile
/usr/bin/X
/usr/bin/at
/usr/bin/lockfile
/usr/bin/expiry
/usr/bin/bsd-write
/usr/bin/crontab
/usr/games/robots
/usr/games/cribbage
/usr/games/hack
/usr/games/atc
/usr/games/sail
/usr/games/snake
/usr/games/canfield
/usr/games/tetris-bsd
/usr/games/phantasia
/usr/games/battlestar
/usr/lib/libvte-2.90-9/gnome-pty-helper
/usr/lib/libvte9/gnome-pty-helper

[root@CentOS7 ~]# find /usr -perm -g+s
/usr/bin/wall
/usr/bin/write
/usr/bin/ssh-agent
/usr/sbin/netreport
/usr/sbin/postdrop
/usr/sbin/postqueue
/usr/sbin/lockdev
/usr/lib64/vte-2.91/gnome-pty-helper
/usr/libexec/utempter/utempter
/usr/libexec/openssh/ssh-keysign
/usr/libexec/abrt-action-install-debuginfo-to-abrt-cache

suse12:~ # find /usr -perm -g+s
/usr/lib/utempter/utempter
/usr/sbin/lockdev
/usr/sbin/postdrop
/usr/sbin/postqueue
/usr/bin/wall
/usr/bin/write
/usr/bin/chage

 

ssh

use SFTP not FTP, use SSH not TELNET.

you should really really use very strong passwords or better use public-private-key-authentication.

i have seen one ssh server got brute-force hacked.

also specify what users are allowed to login. (root should NOT be allowed to login… rather login as normal user and then change to super-user with su or sudo bash)

changing the port of ssh is also an okay idea – but use a privileged ports.

“TCP/IP port numbers below 1024 are special in that normal users are not allowed to run servers on them.

This is a security feaure, in that if you connect to a service on one of these ports you can be fairly sure that you have the real thing, and not a fake which some hacker has put up for you.” (src)

unless the hacker managed to become root…

who is logged in?

suse12:~ # w
16:21:03 up 4:25, 2 users, load average: 0,00, 0,00, 0,00
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
user pts/0 172.20.0.7 12:28 0.00s 0.20s 0.08s sshd: user [priv]

who was last logged in?

last
user pts/1 172.20.0.7 Fri Jun 9 15:32 still logged in
user pts/0 172.20.0.7 Fri Jun 9 15:13 – 15:51 (00:38)
user :0 :0 Fri Jun 9 14:57 still logged in
reboot system boot 3.16.0-4-686-pae Fri Jun 9 18:03 – 16:23 (-1:-39)
user pts/2 172.20.0.7 Fri Jun 9 14:37 – 14:57 (00:19)
user pts/1 :0.0 Fri Jun 9 14:36 – 14:57 (00:20)

 

sudo

as said – it’s not a good idea to run all sorts of services and tasks as root user.

su; # become root

useradd -m bob; # add user bob to the system
passwd bob; # give him a password

# sudo allows to run processes as a different user
sudo -u bob sleep 30 &
ps uax|grep sleep

root 1335 0.0 0.3 6436 3768 pts/0 S 11:48 0:00 sudo -u bob sleep 30
bob 1339 0.0 0.0 3744 536 pts/0 S 11:48 0:00 sleep 30

usermod -a -G sudo bob; # debian8: add user bob to group sudo -> allows user bob to run (hopefully temporary) processes with root-privileges

usermod -a -G wheel bob; # under suse12 or centos7 this group is called “wheel”

under suse12 you will have to:

  1. uncomment this line:

## Uncomment to allow members of group wheel to execute any command
%wheel ALL=(ALL) ALL

2. comment out those two lines:

# Defaults targetpw # ask for the password of the target user i.e. root
# ALL ALL=(ALL) ALL # WARNING! Only use this together with ‘Defaults targetpw’!

… or it will ask bob to input root’s password if he runs for example “sudo bash”.

For more detailed specification of the privileges of bob, instead of adding him to the group sudoers you can:

sudo visudo; # open up the sudoers config file, this also does syntax-checking

>>> /etc/sudoers: syntax error near line 15 <<<
What now?

type “e” and hit enter to re-edit the file.

vim /etc/sudoers; # you could also do those changes “manually”, but without the syntax-checking

and right below

# User privilege specification
root ALL=(ALL:ALL) ALL

bob ALL=(root) /usr/sbin/useradd, /usr/bin/passwd, !/usr/bin/passwd root

ESC :wq! # force save and quit in vim

what does that line mean?

bob ALL=(root)

bob may sudo to run processes as root (not as any other user)

what follows is a ,comma,separated,list of commands that bob is allowed to run

!/and/some/programs/he/is/NOT/allowed/to/run

(whitelist-blacklist)

bob should now be allowed to add a user – without being member of group sudo or wheel

sudo /usr/sbin/useradd -m jo; # try it 😀 should work

# wait for 5 minutes until sudo password-caching expired

# or you will get “passwd: You may not view or modify password information for jo.”

sudo /usr/bin/passwd jo; # asign password to newly created user jo, should work too

another example:

%LimitedAdmins ALL=NOPASSWD: /usr/bin/apt-get*, /etc/init.d/apache2 restart

# will allow admins to use apt-get install or apt-get update or apt-get upgrade

# will allow admins to restart apache2, without even asking for a password

id of the super-user-group:

root@Debian8:/# cat /etc/group|grep sudo
sudo:x:27:

[root@CentOS7]# cat /etc/group|grep wheel
wheel:x:10:

suse12:/# cat /etc/group|grep wheel
wheel:x:10:

manpage: sudo.man.txt

password expiration

if a server was hacked – you will have to change all passwords and renew all private-keys stored on that system.

probably best would be a complete reinstall 😀 (just like under windows if your system caught a virus)

chage -l user; # check how long passwords are valid
Last password change : Apr 24, 2017
Password expires : never
Password inactive : never
Account expires : never
Minimum number of days between password change : 0
Maximum number of days between password change : 99999
Number of days of warning before password expires : 7

user resource limits and core dumps

In computing, a fork bomb (also called rabbit virus or wabbit[1]) is a denial-of-service attack wherein a process continually replicates itself to deplete available system resources, slowing down or crashing the system due to resource starvation.

how to cause a kernel panic

https://unix.stackexchange.com/questions/66197/how-to-cause-kernel-panic-with-a-single-command

fork bomb, open up a terminal an:

:(){ :|:& };:

ulimit.man.txt from suse12

User limits – limit the use of system-wide resources. (src)

every user may limit himself – but to run

ulimit -c unlimited; # you need to be root
-bash: ulimit: core file size: cannot modify limit: Operation not permitted

Syntax
      ulimit [-acdfHlmnpsStuv] [limit]

Options

   -S   Change and report the soft limit associated with a resource. 
   -H   Change and report the hard limit associated with a resource. 

   -a   All current limits are reported. 
   -c   The maximum size of core files created. 
   -d   The maximum size of a process's data segment. 
   -f   The maximum size of files created by the shell(default option) 
   -l   The maximum size that can be locked into memory. 
   -m   The maximum resident set size. 
   -n   The maximum number of open file descriptors. 
   -p   The pipe buffer size. 
   -s   The maximum stack size. 
   -t   The maximum amount of cpu time in seconds. 
   -u   The maximum number of processes available to a single user. 
   -v   The maximum amount of virtual memory available to the process.

ulimit provides control over the resources available to the shell and to processes started by it, on systems that allow such control.

The soft limit is the value that the kernel enforces for the corresponding resource. The hard limit acts as a ceiling for the soft limit.

An unprivileged process may only set its soft limit to a value in the range from 0 up to the hard limit, and (irreversibly) lower its hard limit. A privileged process can make arbitrary changes to either limit value.

If limit is given, it is the new value of the specified resource. Otherwise, the current value of the soft limit for the specified resource is printed, unless the `-H’ option is supplied.

When setting new limits, if neither `-H’ nor `-S’ is supplied, both the hard and soft limits are set.

Restricting per user processes ( -u) can be useful for limiting the potential effects of a fork bomb.

Values are in 1024-byte increments, except for `-t’, which is in seconds, `-p’, which is in units of 512-byte blocks, and `-n’ and `-u’, which are unscaled values.

The return status is zero unless an invalid option is supplied, a non-numeric argument other than unlimited is supplied as a limit, or an error occurs while setting a new limit.

ulimit is a bash built in command.

ulimit -Hn; # show how many files (debian8: 65536, suse12: 1024, centos7: 1024) user may open simultanously (hardlimit)
65536
ulimit -Sn; # show how many files user may open simultanously (softlimit)
65536

lsof -u user|wc -l; # count how many files and directories user “user” currently has open/are in use by user “user”

What is a core dump?

core.man.txt

“A core dump is a file containing a process’s address space (memory) when the process terminates unexpectedly. Core dumps may be produced on-demand (such as by a debugger), or automatically upon termination. Core dumps are triggered by the kernel in response to program crashes, and may be passed to a helper program (such as systemd-coredump) for further processing. A core dump is not typically used by an average user, but may be passed on to developers upon request where it can be invaluable as a post-mortem snapshot of the program’s state at the time of the crash, especially if the fault is hard to reliably reproduce.”

more: HowTo: Debug Crashed Linux Application Core Files Like A Pro

ulimit -c; # zero means per default no core-dumps of crashing programs are produced by kernel

0

cat /etc/profile |grep ulimit; # in past times default settings for ulimit were stored here

ulimit -c 20000; # set soft and hardlimit of maximum-filesize for core-dumps to 20MBytes

ulimit -Sc 10000; # set soft limit of maximum-filesize for core-dumps to 10MBytes

user@Debian8:~$ ulimit
unlimited
user@Debian8:~$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 7924
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 65536
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 7924
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

[user@CentOS7 ~]$ ulimit
unlimited
[user@CentOS7 ~]$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 3855
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 3855
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

user@suse12:~> ulimit
unlimited
user@suse12:~> ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 3865
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 3865
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

sudo vi /etc/security/limits.conf; # this is specified by this config file

# the config file under Centos7 looks like this:

#Also note that configuration files in /etc/security/limits.d directory,
#which are read in alphabetical order, override the settings in this
#file in case the domain is the same or more specific.
#That means for example that setting a limit for wildcard domain here
#can be overriden with a wildcard setting in a config file in the
#subdirectory, but a user specific setting here can be overriden only
#with a user specific setting in the subdirectory.
#
#Each line describes a limit for a user in the form:
#
#<domain> <type> <item> <value>
#
#Where:
#<domain> can be:
# – a user name
# – a group name, with @group syntax
# – the wildcard *, for default entry
# – the wildcard %, can be also used with %group syntax,
# for maxlogin limit
#
#<type> can have the two values:
# – “soft” for enforcing the soft limits
# – “hard” for enforcing hard limits
#
#<item> can be one of the following:
# – core – limits the core file size (KB)
# – data – max data size (KB)
# – fsize – maximum filesize (KB)
# – memlock – max locked-in-memory address space (KB)
# – nofile – max number of open file descriptors
# – rss – max resident set size (KB)
# – stack – max stack size (KB)
# – cpu – max CPU time (MIN)
# – nproc – max number of processes
# – as – address space limit (KB)
# – maxlogins – max number of logins for this user
# – maxsyslogins – max number of logins on the system
# – priority – the priority to run user process with
# – locks – max number of file locks the user can hold
# – sigpending – max number of pending signals
# – msgqueue – max memory used by POSIX message queues (bytes)
# – nice – max nice priority allowed to raise to values: [-20, 19]
# – rtprio – max realtime priority
#
#<domain> <type> <item> <value>
#

#* soft core 0
#* hard rss 10000
#@student hard nproc 20
#@faculty soft nproc 20
#@faculty hard nproc 50
#ftp hard nproc 0
#@student – maxlogins 4
* – maxlogins 1

this line was added by me which means:

limit the maximum amount of simultaneously logins for all users to 1.

if you try to login with the same user more than once you will get:

Too many logins for ‘user’.
Last login: Fri Jun 9 12:33:49 2017 from 172.20.0.7

you don’t have to restart any servies – but maybe wait 5minutes (sudo?) for the changes to take effect.

links:

“Every man is a damn fool for at least five minutes every day; wisdom consists of not exceeding that limit” ~ Elbert Hubbard

limits.conf – configuration file
quota – Display disk usage and limits

http://www.computerworld.com/article/2693414/setting-limits-with-ulimit.html

show hung processes

who -d

last system boot

who -b
system boot 2017-06-09 18:03

Links:

https://www.theurbanpenguin.com/lpic-1-102-110-1-perform-security-administration-tasks/

liked this article?

  • only together we can create a truly free world
  • plz support dwaves to keep it up & running!
  • (yes the info on the internet is (mostly) free but beer is still not free (still have to work on that))
  • really really hate advertisement
  • contribute: whenever a solution was found, blog about it for others to find!
  • talk about, recommend & link to this blog and articles
  • thanks to all who contribute!
admin