26 Comments

  1. Hi,

    “I have 2 web servers running on Apache and mounted the same document root to serve the HTTP content.”

    What kind of storage solution have you implemented?

    Regards

    1. Author

      Hi Ataa, thanks for pointing it out. I have corrected the post

  2. first of all thanks for the post, very helpfull =p
    Have you ever used persistence? I set some servers equal than in your post but I’m facing a problem.
    I set the persistence time to 120s and this is working fine. If a real server turn down the director remove it from the pool of ipvs and doesn’t send new connections to this server BUT if there are some open connections active by persistence time they are mantained in the pool and the user keeps trying to connect on a dead server and every time he tries the persistence time is refreshed, so if he keep refreshing the page he will never get out of this loop…This is normal? I forgot something?
    Thanks very much and congrats for the post

    1. Author

      I have no experience using persistence, but from my understanding LVS remembers the last connection for a specified period of time (120s). If that same client IP address connects again within that period, it will be sent to the same server it connected to previously — bypassing the load-balancing mechanisms.

      Since it says, BYPASSING the load-balancing mechanism, what you were facing is an expected behaviour.

  3. Hello,

    How many real dedicated servers can be used in this system Piranha? Or are only 2 possible?

    Thank you!

    1. Author

      AFAIK, it can only be used with 2 servers with active/passive configuration. Why you want to use many Piranha?

  4. Hi,

    I see in step 4 Add the virtual IP address in the servers:

    I want to know add this virtual IP in what server?

    Loadbalance server or Webserver

    Thanks

    Hoavn

    1. Author

      U need to add the VIP into the web servers as well. Every packet should have source and destination address. If the Web Server do not have that IP, the packet will never get ready because the system cannot bind the VIP (source address) into that packet. This will create invalid packet and your packet will never get delivered to the recipient.

  5. Hi,

    Thanks for this article.
    I’ve configured ipvsadm with direct routing, arptables and firewall marks (80,443). I’ve noticed rejected packets on the firewall on the active load balancer (INPUT table) any idea why? I can access the web server/pages via floating IP but I wonder why some packets are drooped.
    IPtables rules:


    # iptables -nL
    Chain INPUT (policy ACCEPT)
    target prot opt source destination
    ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
    ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0
    ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
    ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:22
    ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:80
    ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:443
    ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:3636
    ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 state NEW udp dpt:539
    REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited

    Chain FORWARD (policy ACCEPT)
    target prot opt source destination
    REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited

    Chain OUTPUT (policy ACCEPT)
    target prot opt source destination

    I’m facing also another problem during failover – the virtual IP is added to the passive LB but it seems like it’s still pointing to the second LB for some time and I can’t access the web servers – is it possible that “send_arp” needs some time to broadcast the new MAC (this IPtable rule solve this problem but I’m not sure if it’s safe: iptables -A FORWARD -d FLOATING_IP -p tcp -m multiport –dports 80,443 -j ACCEPT)?

    1. Author

      May I know what is the packet being rejected? Do you have some logs on that? Depending on firewall rules, you have ‘state NEW’ in your ACCEPT rules, so it will reject any packet which is INVALID.

      Every router/switch should have ARP cache. Try check and disable this feature, or you can try to follow this method to clear ARP cache. Depending on caching, it will need to follow his timeout before these devices refresh their ARP table.

      Is your server really down when the virtual IP is added to passive LB? Or you just turn off pulse service? If you put that rule (-A FORWARD) and it is working, it means that your 1st LB is still up (network and iptables) and do the forwarding to multiport to floating IP which located on the 2nd LB. This method should work but not recommended.

      1. Thanks for reply.

        With regards to ARP cache it must be a side effect of intensive testing as I was stooping/starting pulse on both servers alternately.

        I’ve also added a firewall rule to accept all traffic from a subnet that the servers are running on and I still can see some dropped packets so it maybe be related to a connection states?


        Feb 19 12:37:07 lb1 kernel: IPTables-Dropped: IN=eth0 OUT= MAC=52:54:00:96:b2:97:00:10:db:ff:10:03:08:00 SRC=CLIENT_IP DST=VIRTUAL_IP LEN=40 TOS=0x00 PREC=0x00 TTL=121 ID=31304 DF PROTO=TCP SPT=23548 DPT=80 WINDOW=65050 RES=0x00 ACK FIN URGP=0 MARK=0x50

  6. Hi,
    thank you for the great tutorial.
    i am trying to apply this for a Darwin streaming server cluster.
    i got machine 1 : 4 Darwin streaming servers constituting my cluster with static addresses 192.168.111.6 to 9
    machine 2 : a load balancer and its backup with addresses 192.168.111.30 and 31….the virtual address is 192.168.111.50
    machine 3 : user request testing.

    usually, i simply use rtsp://192.168.111.X/file.sdp to stream….when i try rtsp://192.168.111.50/file.sdp i get nothing while i can directly stream the video directly from the servers…the problem is at the LB level…but i cant find it :((

    1. Author

      Are you forwarding to correct rtsp port on real server? How do you do back-end verification on the monitoring script section? Kindly take note that this tutorial is focusing on using LB on HTTP protocol (tcp port 80).

  7. Worked like a charm. Thank you :)

  8. running perfectly on with LB1, SRV1 and SRV2. But when I add LB2, pulse does not run on it. It says-

    [root@localhost ~]# service pulse restart
    Shutting down pulse: [FAILED]
    Starting pulse: pulse: cannot create heartbeat socket. running as root?
    [FAILED]

    syslog says something like this-
    May 8 13:50:50 localhost pulse: failed to bind to heartbeat address: Address already in use

    Even when all LB1, SRV1 and SRV2 are shut down, pulse keeps saying the same message. I don’t know what I am doing wrong.

    Any suggestions?

    1. Author

      Hi Sarmed,

      Can you verify if /etc/sysconfig/ha/lvs.cf is exists in LB2? I have made some typo on the post under “Load Balancer #1″ section at step 5.

      Previously the value is /etc/sysconfig/ha/lvs.conf

  9. hi,

    i’m trying to implement the same architecture using the direct routing method. In case i have an additionnal layer under the 2 apache real servers and this layer is composed of some tomcat and jboss instances : Will the tomcat instance send the response directly to http user ? or it must move throught the reel server before arriving to http user ?

    Thank you in advance for your help.

    1. Author

      It depends on Apache (real server) role. If it acts like a reverse-proxy, the Tomcat instances will send the response back to Apache to be delivered to HTTP user.

  10. Hi,

    I have a question about sharing sessions between the real servers.
    I don’t know if it’s managed by piranha .. ? For example if a real server crashes, does the second keeps dealing with its sessions ? if not, how can we do that ?

    Thanks very much and congrats for the post.
    Regards.

  11. re
    Hi,

    I have a question about sharing sessions between the real servers.
    I don’t know if it’s managed by piranha .. ? For example if a real server crashes, does the second keeps dealing with its sessions ? if not, how can we do that ?

    Thanks very much and congrats for the post.
    Regards.

  12. Hello. First of all, thank you very much for tutorial! When I start pulse in step 6, it starts but then crashes in several seconds.
    $ tail -f /var/log/message says the following:
    Dec 21 02:30:38 server3 pulse[4425]: STARTING PULSE AS MASTER
    Dec 21 02:30:56 server3 pulse[4425]: partner dead: activating lvs
    Dec 21 02:30:56 server3 pulse[4425]: Failed to open semaphore: Permission denied

    What kind of permissions pulse refers to?
    It’s interesting to note that “partner dead: activating lvs” message persists when I disasble redundency in Piranha.

    1. Author

      Is SELinux running? Are you running VM? Make sure that you have no connection issue with the backup node and both servers has an identical config file.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>