Dies ist eine alte Version des Dokuments!


Proxmox on zeus.brainity.com

This instructions were created during the installation of proxmox on a Hetzner SB87 dedicated Server using Debian Jessie as base system.
17.02.2016

base installation

start hetzner rescue system

login and type

installimage

choose debian - Debian-83-jessie-64-minimal

when editing install.conf set following:

## ===================================================
##  Hetzner Online AG - installimage - standardconfig.
## ===================================================

# Onboard: WDC WD3000FYYZ-01UL1B2
DRIVE1 /dev/sda
# Onboard: WDC WD3000FYYZ-01UL1B2
DRIVE2 /dev/sdb
# Onboard: WDC WD3000FYYZ-01UL1B2
DRIVE3 /dev/sdc
# Onboard: WDC WD3000FYYZ-01UL1B2
#DRIVE4 /dev/sdd
## activate software RAID?  < 0 | 1 >

SWRAID 1

SWRAIDLEVEL 5

BOOTLOADER grub

HOSTNAME zeus.brainity.com

PART /boot  ext3     512M
PART lvm    vg0       all

LV vg0   root   /           ext4         20G
LV vg0   swap   swap        swap          8G
LV vg0   tmp    /tmp        ext4          5G
LV vg0   home   /home       ext4         10G
LV vg0   vz     /var/lib/vz ext4       1000G
IMAGE /root/.oldroot/nfs/install/../images/Debian-83-jessie-64-minimal.tar.gz

save, exit, wait, reboot, login with previous rescue password

set new password

passwd

prepare backup partition (temporary, for grub only)

cfdisk /dev/sdd

create some random partition

porxmox installation

preparations

echo "deb http://download.proxmox.com/debian jessie pve-no-subscription" > /etc/apt/sources.list.d/pve-install-repo.list
wget -O- "http://download.proxmox.com/debian/key.asc" | apt-key add -
apt-get update
apt-get dist-upgrade
GRUB install devices:
[*] /dev/sda (3000592 MB; ???)
[*] /dev/sdb (3000592 MB; ???)
[*] /dev/sdc (3000592 MB; ???)
[*] /dev/sdd (3000592 MB; ???)
[ ] /dev/dm-0 (21474 MB; vg0-root)
[ ] /dev/md0 (536 MB; ???)
reboot

installing Proxmox

apt-get install proxmox-ve ntp ssh postfix ksm-control-daemon open-iscsi systemd-sysv
reboot

select postfix config as 'internet site'

mv  /etc/apt/sources.list.d/pve-enterprise.list  /etc/apt/sources.list.d/pve-enterprise.list_template
apt-get update
apt-get remove linux-image-amd64 linux-image-3.16.0-4-amd64 linux-base
apt-get install mc zip arj
reboot
uname -a

if kernel is now 4.2.8-1-pve or something

apt-get remove linux-image-amd64 linux-image-3.16.0-4-amd64 linux-base
apt-get install proxmox-ve ntp ssh postfix ksm-control-daemon open-iscsi systemd-sysv
update-grub
apt-get autoremove
apt-get install sudo
adduser XXX
adduser XXX sudo
apt-get install mailutils telnet mime-support perl unzip bzip2 links w3m lynx file vim screen ntpdate htop

If you get this error during upgrades: (obsolete)

Setting up pve-cluster (3.0-17) ...
Restarting pve cluster filesystem: pve-clusterstart-stop-daemon: warning: failed to kill 2704: No such process
[main] crit: Unable to get local IP address

remove (uncomment) IPv6 line in hostname file:

vi /etc/hostname

Configuration

init

Datacenter - Options: Keyboard: german

Network

This network setup differs from the standard proxmox settings as we need to use more than one IP and there are no bridge configurations allowed on the Hetzner network.
So we need to setup a bridge that is not connected to eth0. eth0 is ext-iface, vmbr0 will get it's own LAN to which all vm's will be connected. A firewall will then configure explicit port-forwardings. This provides great flexibility especially with limited external IPs.

Interfaces

Anleitung von hetzner

 wget -P/etc/apt/sources.list.d/ http://wertarbyte.de/apt/wertarbyte-apt.list
 wget -O - http://wertarbyte.de/apt/software-key.gpg | apt-key add -
 apt-get update
 apt-get install ifupdown-scripts-wa

edit /etc/network/interfaces

this is the original entry:

### Hetzner Online GmbH - installimage
# Loopback device:
auto lo
iface lo inet loopback

# device: eth0
auto  eth0
iface eth0 inet static
  address   148.251.54.49
  netmask   255.255.255.224
  gateway   148.251.54.33
  # default route to access subnet
  up route add -net 148.251.54.32 netmask 255.255.255.224 gw 148.251.54.33 eth0

iface eth0 inet6 static
  address 2a01:4f8:202:1430::2
  netmask 64
  gateway fe80::1

This was very helpful: http://www.proxmox.com/forum/showthread.php?p=10744

This is how i modified the file.

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static

    # Haupt-IP-Adresse des Servers
        address     148.251.54.49
    # Netzmaske 255.255.255.255 (/32) unabhängig von der
    # realen Netzaufteilung (z.B. /27)
        netmask     255.255.255.255
    # Explizite Hostroute zum Gateway
        gateway     148.251.54.33
        pointopoint 148.251.54.33

    # original settings
    #   address  148.251.54.49
    #   netmask  255.255.255.224
    #   gateway  148.251.54.33

        post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
    # additional ip('s) - seperated by spaces
        create_alias_devices yes
        #label_addresses yes
        addresses-ip2 148.251.54.59/32
        addresses-ip3 148.251.54.60/32
        addresses-ip4 148.251.54.61/32

# default route to access subnet
up route add -net 148.251.54.32 netmask 255.255.255.224 gw 148.251.54.33 eth0

auto vmbr0
iface vmbr0 inet static
        address  10.0.0.138
        netmask  255.255.255.0
        bridge_ports none
        bridge_stp off
        bridge_fd 0
### Hetzner Online GmbH - installimage
# Loopback device:
auto lo
iface lo inet loopback

# device: eth0
auto  eth0
iface eth0 inet static
  # Haupt-IP-Adresse des Servers
    address   148.251.54.49
  # Netzmaske 255.255.255.255 (/32) unabhängig von der
  # realen Netzaufteilung (z.B. /27)
    netmask   255.255.255.255
  # Explizite Hostroute zum Gateway
    gateway   148.251.54.33
    pointopoint 148.251.54.33
  # default route to access subnet
  post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
  # additional ip('s) - seperated by spaces
    create_alias_devices yes
    #label_addresses yes
    addresses-ip2 148.251.54.59/32
    addresses-ip3 148.251.54.60/32
    addresses-ip4 148.251.54.61/32

  # default route to access subnet
    up route add -net 148.251.54.32 netmask 255.255.255.224 gw 148.251.54.33 eth0

iface eth0 inet6 static
  address 2a01:4f8:202:1430::2
  netmask 64
  gateway fe80::1

auto vmbr0
iface vmbr0 inet static
        address  10.0.0.138
        netmask  255.255.255.0
        bridge_ports none
        bridge_stp off
        bridge_fd 0

test your new settings (reboot, ping etc.) use 'ip addr show' to show additional ip's (ipconfig won't show them)

you can also label the ip's using something like

        addresses-https IP1/32 IP2/32
        addresses-vhost IP3/32
        label_addresses yes

WARNING! Do not use the web interface to change the network settings. You may lock yourself because Hetzner needs theses spezial settings, the web interface can't configure!

Firewall

apt-get install arno-iptables-firewall rblcheck

give theses answers

manage with debconf? yes
external Interface: eth0
(DHCP: No)
open external TCP-Ports: 22 22000 80 443 8006 5900 5901 5902 5903 5904 5905 5906 5907 5908 5909 5910
open external UDP-Ports: ""
(Ping: Yes)
internal Interface: vmbr0
internal subnet: 10.0.0.0/24
(NAT: Yes)
(internal Networks: 10.0.0.0/24)
restart firewall? yes

/etc/arno-iptables-firewall/conf.d/00debconf.conf

EXT_IF="eth0"
EXT_IF_DHCP_IP=0
OPEN_TCP="22 22000 80 443 7443 5900 5901 5902 5903 5904 5905 5906 5907 5908 5909 5910"
OPEN_UDP=""
INT_IF="vmbr0"
NAT=1
INTERNAL_NET="10.0.0.0/24"
NAT_INTERNAL_NET=""
OPEN_ICMP=1
invoke-rc.d arno-iptables-firewall start

/etc/init.d/port-forward

#!/bin/sh

### BEGIN INIT INFO
# Provides:        port_forwarding
# Required-Start:  $network $syslog
# Required-Stop:   $network $syslog
# Default-Start:   2 3 4 5
# Default-Stop:    0 1 6
# Short-Description: Start port forwarding
### END INIT INFO

# allow port forwarding to external IPs? (e.g. other internet server)
# set to 0 or 1
ALLOW_EXT_FW=1

#forward to virtual host
PORT_FW="/usr/local/bin/add_port_forwarding"
##redirect to other external server
#port_rd="/usr/local/bin/add_port_redirection"

PATH=/sbin:/bin:/usr/sbin:/usr/bin

#. /lib/lsb/init-functions


case "$1" in
    start)
        $0 stop
        /etc/init.d/arno-iptables-firewall start

        if ! [ ${ALLOW_EXT_FW} -eq 0 ] ; then
            iptables -A POSTROUTING -t nat -j MASQUERADE -o eth0
#            echo 1 > /proc/sys/net/ipv4/ip_forward
        fi


        "$PORT_FW" 148.251.54.59 80 10.0.0.202 8080 tcp
        ;;
    stop)
        /sbin/iptables -F
        /etc/init.d/arno-iptables-firewall stop
        /sbin/iptables -F
        ;;
    reload|force-reload)
        $0 restart
        ;;
    restart)
#        $0 stop
        $0 start
        ;;
    *)
        log_success_msg "Usage: /etc/init.d/portmap {start|stop|force-reload|restart}"
        exit 1
        ;;
esac

exit 0

/usr/local/bin/add_port_forwarding

#!/bin/bash
IPTABLES=$(which iptables)

EXT_IF="eth0"

EXT_IP=$1
EXT_PORT=$2
INT_IP=$3
INT_PORT=$4
PROTO=$5

if [ -z ${EXT_IP} ]; then
        echo "usage: ${0} ext_ip ext_port int_ip int_port protocol"
        exit 1
fi
if [ -z ${EXT_PORT} ]; then
        echo "usage: ${0} ext_ip ext_port int_ip int_port protocol"
        exit 1
fi
if [ -z ${INT_IP} ]; then
        echo "usage: ${0} ext_ip ext_port int_ip int_port protocol"
        exit 1
fi
if [ -z ${INT_PORT} ]; then
        echo "usage: ${0} ext_ip ext_port int_ip int_port protocol"
        exit 1
fi
if [ -z ${PROTO} ]; then
        echo "usage: ${0} ext_ip ext_port int_ip int_port protocol"
        exit 1
fi


${IPTABLES} -t nat -I PREROUTING -p $PROTO -d ${EXT_IP} -i ${EXT_IF} --dport ${EXT_PORT} -j DNAT --to ${INT_IP}:${INT_PORT}
${IPTABLES} -I FORWARD -p $PROTO -i ${EXT_IF} -d ${INT_IP} --dport ${INT_PORT} -j ACCEPT

# other way to do that: which one is better?
#${IPTABLES} -I PREROUTING -t nat -p ${PROTO} -d ${EXT_IP} --dport ${EXT_PORT} -m state --state NEW,ESTABLISHED,RELATED -j DNAT --to ${INT_IP}:${INT_PORT}
#${IPTABLES} -I FORWARD -i ${EXT_IF} -o ${EXT_IF} -p ${PROTO} --dport ${INT_PORT} -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT

FIXME Kernel logging should be modified (should we use syslog-ng here?)

postfix

dpkg-reconfigure postfix
Type: Internet Site
Mail Name: zeus.brainity.com
postmaster, root: logs _ at _ brainity (dot) com
Other Destinationes: unchanged
synchronous updates: No
Local networks: "127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128" (default)
Mailbox limits: 0
Local address extension character: "+"
IPv4/6: all

backup space

cfdisk /dev/sdd

create one big partition sdd1 type 8E (Linux LVM)

vgcreate vg1 /dev/sdd1
lvcreate -n backup --size 2500GB vg1
mkfs.ext4 /dev/vg1/backup
mkdir /backup
echo "/dev/vg1/backup /backup  ext4  defaults 0 0">>/etc/fstab
mount /backup

in the webinterface goto Datacenter - Storage - Add - Directory

Storage Name: backup
Directory: /backup
Content: VZDump Backups
Nodes: All
Enabled
not shared
Max backups 10

now create a backup:

Backup - create a new backup task

Node: All
Storage: backup
Mode: snapshot
Day of week: Mo-So
Selection mode: All
send email to: logs at br..
notification always
send logs to: logs_at_brainity dot com
compress: LZO
Mode: snapshot
enabled
docs/zeus/base-installation.1456068166.txt.gz · Zuletzt geändert: 2016/02/21 16:22 von joggl