linux-keepalived-02 keepalived+LVS+Apache部署

发表于 | 更新于 | 分类于 ， | 评论数： | 热度： ℃
linux-keepalived-02 keepalived+LVS+Apache部署

linux-keepalived-02 keepalived-+LVS+Apache部署

概念解析

　　LVS是Linux Virtual Server的简写，意即Linux虚拟服务器，是一个虚拟的服务器集群系统。

　　可伸缩网络服务的几种结构，它们都需要一个前端的负载调度器（或者多个进行主从备份）。我们先分析实现虚拟网络服务的主要技术，指出IP负载均衡技术是在负载调度器的实现技术中效率最高的。在已有的IP负载均衡技术中，主要有通过网络地址转换（Network Address Translation）将一组服务器构成一个高性能的、高可用的虚拟服务器，我们称之为VS/NAT技术（Virtual Server via Network Address Translation）。在分析VS/NAT的缺点和网络服务的非对称性的基础上，我们提出了通过IP隧道实现虚拟服务器的方法VS/TUN （Virtual Server via IP Tunneling），和通过直接路由实现虚拟服务器的方法VS/DR（Virtual Server via Direct Routing），它们可以极大地提高系统的伸缩性。VS/NAT、VS/TUN和VS/DR技术是LVS集群中实现的三种IP负载均衡技术。

lvs官网：http://www.linuxvirtualserver.org。

　　当web服务器太多时，RS就压力很大，所以引入DS。

重要参数：

1
2
arp_announce
arp_ignore

优点

1、开源，免费
2、在网上能找到一些相关技术资源
3、具有软件负载均衡的一些优点

缺点

1、最核心的就是没有可靠的支持服务，没有人对其结果负责；
2、功能比较简单，支持复杂应用的负载均衡能力较差，如算法较少等；
3、开启隧道方式需重编译内核；
4、配置复杂；
5、主要应用于LINUX，目前没有专门用于WINDOWS的版本，不过可以通过配置，使windows成为LVS集群中的real server（win2003、win2008中）。

LVS相关概念

ipvs：ipvs称之为IP虚拟服务器（IP Virtual Server，简写为IPVS），是运行在LVS下的提供负载平衡功能的一种技术。工作于内核当中，定义转发规则。
ipvsadm：工作于用户空间，是一个工具，同时它也是一条命令，用于管理LVS的策略规则。
DS：前端负载均衡节点 direct-server
RS：后端真实节点 real-server
DIP：负载均衡节点的IP地址
VIP：虚拟IP地址，必须要与RS在同一网段
RIP：真实节点的IP地址
CIP：客户端IP地址

LVS/NAT模型
　　RS的网关必须为DIP
　　DIP要和RIP处于同一网络
　　RS支持任意操作系统
　　RIP必须为私网地址
　　DS支持端口映射

1VPaCt.png

LVS/DR模型
　　前端路由必须发送给DS
　　VIP要和RIP处于同一网络
　　RS的网关不能够指向DIP
　　不支持端口映射
　　RS响应报文绝不通过DS
　　在环回口上配置VIP地址，因为环回口本来没有ip

1Vi9VH.png

LVS/TUN模型

　　主要是解决DR转发到不同数据中心的RS

lvs负载均衡算法

　　rr：轮询
　　wrr：加权轮询
　　lc：最少连接数
　　wlc：加权最少连接数
　　ip hash：源地址哈希
　　sip hash：目的地址哈希

实验环境

DR1:192.168.141.53
DR2:192.168.141.69
RS1：192.168.141.12
RS2:192.168.141.132
VIP：192.168.141.99（单主模型）

RS部署

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
在rs1和执行下面的脚本：
[root@rs1 ~]# vim rs1.sh
[root@rs1 ~]# cat rs1.sh 
#!/bin/bash
yum install net-tools httpd -y
systemctl stop firewalld
setenforce 0
vip="192.168.141.99"
mask="255.255.255.255"
ifconfig lo:0 $vip broadcast $vip netmask $mask up
route add -host $vip lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "<h1>This is RS1</h1>" > /var/www/html/index.html
systemctl restart httpd
systemctl enable httpd
[root@rs1 ~]# . rs1.sh 


在rs2和执行下面的脚本：
[root@rs2 ~]# vim rs2.sh
[root@rs2 ~]# cat rs2.sh 
#!/bin/bash
yum install net-tools httpd -y
systemctl stop firewalld
setenforce 0
vip="192.168.141.99"
mask="255.255.255.255"
ifconfig lo:0 $vip broadcast $vip netmask $mask up
route add -host $vip lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "<h1>This is RS2</h1>" > /var/www/html/index.html
systemctl restart httpd
systemctl enable httpd
[root@rs2 ~]# . rs2.sh

DR部署

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
dr1设置为主，dr2设置为从，二者配置文件的差异在于：
dr1
state MASTER 
priority 100
router_id node1

dr2
state BACKUP
priority 90
router_id node2

[root@dr1 ~]# yum install keepalived ipvsadm -y
[root@dr1 ~]# systemctl stop firewalld
[root@dr1 ~]# setenforce 0
[root@dr1 ~]# cd /etc/keepalived/
[root@dr1 keepalived]# 
[root@dr1 keepalived]# mv keepalived.conf{,.bak}
[root@dr1 keepalived]# vim keepalived.conf
[root@dr1 keepalived]# cat keepalived.conf
! Configuration File for keepalived
global_defs {
   router_id node1 # 设置lvs的id，一个网络中应该唯一
}
vrrp_instance VI_1 {
    state MASTER # 指定Keepalived的角色，主是MASTER，从是BACKUP
    interface ens33 # 网卡
    virtual_router_id 10 # 虚拟路由器ID，主备需要一样
    priority 100 # 优先级越大越优，backup路由器需要设置比这小！可以设置为90
    advert_int 1 # 检查间隔1s
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.141.99 # 定义虚拟IP地址，可以定义多个
    }
}
# 定义虚拟主机，对外服务的IP和port
virtual_server 192.168.141.99 80 { 
    delay_loop 6 # 设置健康检查时间，单位是秒 
    lb_algo wrr # 负责调度算法
    lb_kind DR # LVS负载均衡机制
    persistence_timeout 0
    protocol TCP
# 指定RS主机IP和port
    real_server 192.168.141.12 80 {
        weight 2
    # 定义TCP健康检查    
        TCP_CHECK {  
        connect_timeout 10         
        nb_get_retry 3  
        delay_before_retry 3  
        connect_port 80
        }
    }
    real_server 192.168.141.132 80 {
        weight 1
        TCP_CHECK {  
        connect_timeout 10         
        nb_get_retry 3  
        delay_before_retry 3  
        connect_port 80
        }
    }
}
[root@dr1 keepalived]#  systemctl restart keepalived
[root@dr1 keepalived]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.141.99:80 wrr
  -> 192.168.141.12:80            Route   2      0          0         
  -> 192.168.141.132:80           Route   1      0          0         
[root@dr1 keepalived]# 



dr2按照上面的同样配置，唯一差异在于：
state BACKUP
 priority 90

[root@dr2 ~]# yum install keepalived ipvsadm -y
[root@dr2 ~]# systemctl stop firewalld
[root@dr2 ~]# setenforce 0
[root@dr2 ~]# cd /etc/keepalived/
[root@dr2 keepalived]# 
[root@dr2 keepalived]# mv keepalived.conf{,.bak}
[root@dr2 keepalived]# vim keepalived.conf
[root@dr2 keepalived]# cat keepalived.conf
! Configuration File for keepalived
global_defs {
   router_id node2 # 设置lvs的id，一个网络中应该唯一
}
vrrp_instance VI_1 {
    state BACKUP # 指定Keepalived的角色，主是MASTER，从是BACKUP
    interface ens33 # 网卡
    virtual_router_id 10 # 虚拟路由器ID，主备需要一样
    priority 90 # 优先级越大越优，backup路由器需要设置比这小！
    advert_int 1 # 检查间隔1s
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.141.99 # 定义虚拟IP地址，可以定义多个
    }
}
# 定义虚拟主机，对外服务的IP和port
virtual_server 192.168.141.99 80 { 
    delay_loop 6 # 设置健康检查时间，单位是秒 
    lb_algo wrr # 负责调度算法
    lb_kind DR # LVS负载均衡机制
    persistence_timeout 0
    protocol TCP
# 指定RS主机IP和port
    real_server 192.168.141.12 80 {
        weight 2
    # 定义TCP健康检查    
        TCP_CHECK {  
        connect_timeout 10         
        nb_get_retry 3  
        delay_before_retry 3  
        connect_port 80
        }
    }
    real_server 192.168.141.132 80 {
        weight 1
        TCP_CHECK {  
        connect_timeout 10         
        nb_get_retry 3  
        delay_before_retry 3  
        connect_port 80
        }
    }
}
[root@dr2 keepalived]# systemctl restart keepalived
[root@dr2 keepalived]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.141.99:80 wrr
  -> 192.168.141.12:80            Route   2      0          0         
  -> 192.168.141.132:80           Route   1      0          0         
[root@dr2 keepalived]#

　　如果用浏览器测试的话，因为有缓存的原因，现象就不是那么的准确。所以我们这里采用windows的cmd来进行测试，多次输入命令：curl 192.168.141.99即可。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
C:\Users\zhong>curl 192.168.141.99
<h1>This is RS2</h1>

C:\Users\zhong>curl 192.168.141.99
<h1>This is RS1</h1>

C:\Users\zhong>curl 192.168.141.99
<h1>This is RS1</h1>

C:\Users\zhong>curl 192.168.141.99
<h1>This is RS2</h1>

C:\Users\zhong>curl 192.168.141.99
<h1>This is RS1</h1>

C:\Users\zhong>curl 192.168.141.99
<h1>This is RS1</h1>

C:\Users\zhong>curl 192.168.141.99
<h1>This is RS2</h1>

C:\Users\zhong>curl 192.168.141.99
<h1>This is RS1</h1>

C:\Users\zhong>curl 192.168.141.99
<h1>This is RS1</h1>

　　可以看到，访问同一个ip：192.168.141.99得到的却是不同的结果，而且访问到RS1和RS2主页的比例正如我们在上面配置文件里设置好的那样，是2:1。

验证keepalive的健康检查功能

将RS1的httpd关掉，查看dr1上转发目的主机列表，列表里已经没有RS1了

1
2
3
4
5
6
7
8
9
10
11
[root@rs1 ~]# systemctl stop httpd
[root@rs1 ~]# 


[root@dr1 keepalived]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.141.99:80 wrr
  -> 192.168.141.132:80           Route   1      0          3         
[root@dr1 keepalived]#

将RS1的httpd重新开启，再次查看dr1上转发目的主机列表，列表里又有RS1了

1
2
3
4
5
6
7
8
9
10
[root@rs1 ~]# systemctl restart httpd

[root@dr1 keepalived]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.141.99:80 wrr
  -> 192.168.141.12:80            Route   2      0          0         
  -> 192.168.141.132:80           Route   1      0          0         
[root@dr1 keepalived]#

验证keepalive的vip切换功能

分别查看DR1和DR2的ip，可以看到vip：192.168.141.99在DR1的ens33网卡上

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
[root@dr1 keepalived]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:8b:6f:8d brd ff:ff:ff:ff:ff:ff
    inet 192.168.141.53/24 brd 192.168.141.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.141.99/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::45c1:b728:e8e7:a1fa/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
[root@dr1 keepalived]# 


[root@dr2 keepalived]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:56:58:3c brd ff:ff:ff:ff:ff:ff
    inet 192.168.141.69/24 brd 192.168.141.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::1edf:37bf:62b9:68e2/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
[root@dr2 keepalived]#

关掉dr1的keepalived

1
[root@dr1 keepalived]# systemctl stop keepalived

再次查看DR1和DR2的ip，可以看到vip：192.168.141.99在DR2的ens33网卡上，切换成功！

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
[root@dr1 keepalived]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:8b:6f:8d brd ff:ff:ff:ff:ff:ff
    inet 192.168.141.53/24 brd 192.168.141.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::45c1:b728:e8e7:a1fa/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
[root@dr1 keepalived]# 



[root@dr2 keepalived]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:56:58:3c brd ff:ff:ff:ff:ff:ff
    inet 192.168.141.69/24 brd 192.168.141.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.141.99/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::1edf:37bf:62b9:68e2/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
[root@dr2 keepalived]#

重启DR1的keepalived

1
[root@dr1 keepalived]# systemctl restart keepalived

再次查看DR1和DR2的ip，可以看到vip：192.168.141.99在DR1的ens33网卡上，切换成功！

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
[root@dr1 keepalived]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:8b:6f:8d brd ff:ff:ff:ff:ff:ff
    inet 192.168.141.53/24 brd 192.168.141.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.141.99/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::45c1:b728:e8e7:a1fa/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
[root@dr1 keepalived]# 



[root@dr2 keepalived]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:56:58:3c brd ff:ff:ff:ff:ff:ff
    inet 192.168.141.69/24 brd 192.168.141.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::1edf:37bf:62b9:68e2/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
[root@dr2 keepalived]#
欢迎打赏，谢谢
------ 本文结束------
0%