Proxmox NAT & CSF

edited November 2019 in Help

I have the above setup, at long last nearing the point of using it, rather than idling. I can connect to a VM from the 'net via redirected ports and the VM is able to connect to the world.
When I ssh in to the VM, it is being logged as coming from the host server, not my home IP, so I'm obviously missing something in the setup. Any pointers for me?
Ta.

depleted.

Tagged:

Comments

  • AnthonySmithAnthonySmith AdministratorHosting Provider
    edited November 2019

    The NAT les servers just use simple SNAT per external IP e.g.
    iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -o br0 -j SNAT --to 104.105.106.107
    Then port forwarding with DNAT e.g for ssh access
    iptables -t nat -A PREROUTING -p tcp -d 104.105.106.107 --dport 321 -j DNAT --to 192.168.20.3:22
    and then a port range through DNAT for the rest:
    iptables -t nat -A PREROUTING -p tcp -d 104.105.106.107 --dport 301:320 -j DNAT --to-destination 192.168.20.3

    But that assumes a few things, its been an age since I used proxmox properly but I remember it had a number of different options on how to pass traffic through when adding an interface type for the VM's the above would only work in a bridged network, I do seem to remember (going back maybe 7 - 8 years) that the default NAT setup in proxmox was to masquerade which may be the cause of your problem, I think a bond would do the same so you need bridged or a seperate VLAN for guest networks using a manually configured bridge would be my guess.

    Could be wrong, just guessing without seeing.

    Inception Hosting - we surveyed 100 people and asked them what a fat husband may hide from his wife in his belly button, the 3rd most popular answer was: "Jewelry"

  • I'm using a bridged network on an OVH host. I have the following if that makes things clearer.
    Part of my iptables-set.sh/csfpost.sh

    • echo 1 >/proc/sys/net/ipv4/ip_forward
    • echo 1 > /proc/sys/net/ipv4/conf/vmbr1/proxy_arp
    • /sbin/iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 8000 -j DNAT --to 10.0.0.100:80
    • ...
    • iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j MASQUERADE
    • /sbin/iptables -F FORWARD
    • /sbin/iptables -P FORWARD ACCEPT

    In /etc/network/interfaces

    • post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j MASQUERADE
    • post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 8000 -j DNAT --to 10.0.0.100:80

    There appears to be a duplication above but it's taken me ages to get packets traversing the host. Seems like I'm missing SNAT entries .. I'm sure I spotted them someplace. :-S

    depleted.

  • AnthonySmithAnthonySmith AdministratorHosting Provider

    I think the issue is the MASQUERADE, so it is well, masquerading :) if i were you I would strip it all back and follow the examples above, its actually a lot more simple than it seems to achieve what you want.

    Thanked by AlwaysSkint mikho

    Inception Hosting - we surveyed 100 people and asked them what a fat husband may hide from his wife in his belly button, the 3rd most popular answer was: "Jewelry"

  • I was following a mashup of 'instructions' on the web and got pretty confused along the way. I also have nginx reverse proxy and webserver VMs on this host, so I'll take my time over the reconfiguration. Thanks!

    depleted.

  • mikhomikho Hosting ProviderOG

    I was about to write something useful, @AnthonySmith beat me to it.
    SNAT is the way to go unless you either want to MASQUERADE the original IP or in some cases uses a dynamic IP.

    Thanked by AlwaysSkint

    Get 4 or more NAT servers (mix/match between packages) and get 45% recurring discount. https://clients.mrvm.net

  • edited November 2019

    Latest configuration:
    /etc/network/interfaces (snippet)

    • post-up echo 1 > /proc/sys/net/ipv4/ip_forward
    • post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j SNAT --to 178.32.xxx.xxx
    • post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j SNAT --to 178.32.xxx.xxx
    • post-up iptables -t nat -A PREROUTING -d 178.32.xxx.xxx -p tcp --dport 8000 -j DNAT --to 10.0.0.100:80
    • post-down iptables -t nat -D PREROUTING -d 178.32.xxx.xxx -p tcp --dport 8000 -j DNAT --to 10.0.0.100:80

    I couldn't get outbound (eg. apt update) from the VM when the host had CSF active, until I placed this in csfpost.sh

    • #!/bin/bash
    • echo 1 >/proc/sys/net/ipv4/ip_forward
    • echo 1 > /proc/sys/net/ipv4/conf/vmbr1/proxy_arp
    • /sbin/iptables -F FORWARD
    • /sbin/iptables -P FORWARD ACCEPT
    • /sbin/iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j SNAT --to 178.32.xxx.xxx

    I dunno the consequences of those iptables FORWARD directives, however.. will need to do a search.

    EDIT: VM still shows host IP for my laptop doing an ssh to it. :'( In other words the originating IP is being stripped during the forwarding.

    depleted.

  • kmmmkmmm OG
    edited November 2019

    I routed all traffic to the bare metal nginx instance before passing traffic to corresponding virtual network interfaces (created and assigned by Proxmox, ofc). This is dead simple, and I do not need to mess with iptables since I am also using block IPs based on ipset. My setup is for personal use, so no idea how it really effects the performance of Proxmox host under load.

    Thanked by AlwaysSkint
  • What I do is simpler
    Access to the NAT vm's is via a http(s) load balancer KVM(dedicated IPv4)

    The individual KVMs serial terminals can be 'accessed' by ' qm terminal 101' in a tmux ssh/mosh session to the proxmox host.

    My vm templates don't have openssh-server. I install it when required

    Thanked by AlwaysSkint
  • edited November 2019

    A proxmox console is one way around it but doesn't address other services. I used ssh as an example because it is relatively simple to monitor activity to it. You also assume that an additional IP is available (I know I didn't cover that aspect) but that's not an option on Kimsufi servers, unfortunately. I have another proxmox instance elsewhere with /29, so that this subject is never considered, except for IPv6 trials.

    TLDR; Originating IP is replaced by host IP at the VM

    Thanked by vimalware

    depleted.

  • Use IPv6 B)

    The all seeing eye sees everything...

Sign In or Register to comment.