正文

5000并发网站生产级架构设计（BAT运维标准）

admin
admin V管理员 /0 评论 /82 阅读
0426
此篇文章发布距今已超过20天，您需要注意文章的内容或图片是否可用！
一、架构核心定位
针对5000并发连接、1500-3000 QPS的中小型电商/资讯/SAAS类网站，采用高可用分层架构，兼顾稳定性、成本与可扩展性，符合大厂运维规范，无冗余、无短板，可直接上线部署。
  • 高可用保障：负载均衡双机热备，数据库主从复制，单点故障自动切换，可用性≥99.9%
  • 性能目标：页面响应时间≤300ms，接口超时率≤0.1%，支持突发流量弹性扩容
  • 机器规划：6台CentOS 7/Rocky Linux 8服务器，内网互通，配置标准化
二、服务器清单与IP规划
层级
主机名
内网IP
核心角色
硬件配置
对外VIP
负载均衡层
lb01
192.168.1.10
Nginx反向代理+Keepalived主节点
4核8G SSD
192.168.1.100（唯一入口）
负载均衡层
lb02
192.168.1.11
Nginx反向代理+Keepalived备节点
4核8G SSD
Web应用层
web01
192.168.1.20
Nginx+PHP-FPM 业务节点
8核16G SSD
-
Web应用层
web02
192.168.1.21
Nginx+PHP-FPM 业务节点
8核16G SSD
-
数据存储层
db01
192.168.1.30
MySQL 主库（读写）
8核16G NVMe SSD
-
数据存储层
db02
192.168.1.31
MySQL 从库（只读+备份）
8核16G NVMe SSD
-
三、高清架构流程图
架构亮点：全程内网通信、无公网暴露风险；分层解耦，单节点故障不影响全局；配置极简，运维成本低，符合大厂中小流量场景部署标准。
四、全节点通用系统优化（所有服务器必执行）
提升文件句柄、TCP参数，解决高并发下端口耗尽、连接超时问题，一键复制执行即可。
# 1. 优化文件描述符限制
cat >> /etc/security/limits.conf <<EOF
* soft nofile 65535
* hard nofile 65535
* soft nproc 65535
* hard nproc 65535
EOF

# 2. 内核TCP参数优化
cat >> /etc/sysctl.conf <<EOF
fs.file-max = 65535
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_syncookies = 1
EOF

# 3. 生效配置
sysctl -p
# 关闭防火墙与SELinux（生产环境可精细化放行端口）
systemctl stop firewalld && systemctl disable firewalld
sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
setenforce 0
五、各节点详细配置文件（可直接复制部署）
（一）负载均衡节点 lb01 配置
1. Nginx主配置 /etc/nginx/nginx.conf
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
    worker_connections 65535;
useepoll;
    multi_accept on;
}

http {
include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
    access_log off; # 高并发关闭访问日志，提升性能

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    gzip on;
    gzip_types text/plain text/css application/json application/javascript;

# Web集群配置
    upstream web_cluster {
        server 192.168.1.20:80 max_fails=3 fail_timeout=10s;
        server 192.168.1.21:80 max_fails=3 fail_timeout=10s;
        ip_hash; # 会话保持，适配登录业务
    }

    server {
        listen 80;
        server_name _; # 适配所有域名

# 反向代理Web集群
        location / {
            proxy_pass http://web_cluster;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_connect_timeout 3s;
            proxy_read_timeout 10s;
        }
    }
}
2. Keepalived主配置 /etc/keepalived/keepalived.conf
global_defs {
    router_id LB01
}

# Nginx健康检查脚本
vrrp_script check_nginx {
    script "/etc/keepalived/check_nginx.sh"
    interval 2
    weight -20
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0 # 改为实际网卡名（ens33等）
    virtual_router_id 51
    priority 150 # 优先级高于备机
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass 666666
    }

    virtual_ipaddress {
        192.168.1.100/24
    }

    track_script {
        check_nginx
    }
}
3. Nginx健康检查脚本 /etc/keepalived/check_nginx.sh
#!/bin/bash
# 检测Nginx进程，不存在则关闭Keepalived触发VIP漂移
if ! pgrep nginx >/dev/null; then
    systemctl stop keepalived
fi
# 赋权并启动服务
chmod +x /etc/keepalived/check_nginx.sh
systemctl start nginx keepalived
systemctl enable nginx keepalived
（二）负载均衡节点 lb02 配置
Nginx配置与lb01完全一致，仅修改Keepalived配置，实现备机切换。
Keepalived备配置 /etc/keepalived/keepalived.conf
global_defs {
    router_id LB02
}

vrrp_script check_nginx {
    script "/etc/keepalived/check_nginx.sh"
    interval 2
    weight -20
}

vrrp_instance VI_1 {
    state BACKUP
interfaceeth0
virtual_router_id 51
priority 100 # 优先级低于主机
advert_int 1

authentication{
        auth_type PASS
        auth_pass 666666
    }

    virtual_ipaddress {
192.168.1.100/24
    }

    track_script {
        check_nginx
    }
}
# 启动服务
chmod +x /etc/keepalived/check_nginx.sh
systemctl start nginx keepalived
systemctl enable nginx keepalived
（三）Web节点 web01/web02 配置（两台完全一致）
1. 安装依赖
yum install -y nginx php php-fpm php-mysqlnd php-opcache php-gd php-mbstring php-xml
2. Nginx配置 /etc/nginx/nginx.conf
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
worker_connections65535;
useepoll;
}

http {
include mime.types;
default_type application/octet-stream;
sendfileon;
keepalive_timeout65;

server {
listen80;
root /var/www/html;
index index.php index.html;

# 解析PHP请求
location~ .php$ {
fastcgi_pass127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
        }

# 静态资源缓存
location~* .(jpg|png|css|js|ico)$ {
expires7d;
access_logoff;
        }
    }
}
3. PHP-FPM配置 /etc/php-fpm.d/www.conf
user = nginx
group = nginx
listen = 127.0.0.1:9000
listen.owner = nginx
listen.group = nginx
listen.mode = 0660

# 进程池优化（适配8核16G）
pm = dynamic
pm.max_children = 128
pm.start_servers = 32
pm.min_spare_servers = 16
pm.max_spare_servers = 48
pm.max_requests = 1000# 防止内存泄漏

request_terminate_timeout = 30s
slowlog = /var/log/php-fpm/slow.log
request_slowlog_timeout = 5s
4. PHP核心配置 /etc/php.ini（关键修改）
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
    worker_connections 65535;
useepoll;
    multi_accept on;
}

http {
include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
    access_log off; # 高并发关闭访问日志，提升性能

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    gzip on;
    gzip_types text/plain text/css application/json application/javascript;

# Web集群配置
    upstream web_cluster {
        server 192.168.1.20:80 max_fails=3 fail_timeout=10s;
        server 192.168.1.21:80 max_fails=3 fail_timeout=10s;
        ip_hash; # 会话保持，适配登录业务
    }

    server {
        listen 80;
        server_name _; # 适配所有域名

# 反向代理Web集群
        location / {
            proxy_pass http://web_cluster;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_connect_timeout 3s;
            proxy_read_timeout 10s;
        }
    }
}0
5. 启动服务
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
    worker_connections 65535;
useepoll;
    multi_accept on;
}

http {
include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
    access_log off; # 高并发关闭访问日志，提升性能

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    gzip on;
    gzip_types text/plain text/css application/json application/javascript;

# Web集群配置
    upstream web_cluster {
        server 192.168.1.20:80 max_fails=3 fail_timeout=10s;
        server 192.168.1.21:80 max_fails=3 fail_timeout=10s;
        ip_hash; # 会话保持，适配登录业务
    }

    server {
        listen 80;
        server_name _; # 适配所有域名

# 反向代理Web集群
        location / {
            proxy_pass http://web_cluster;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_connect_timeout 3s;
            proxy_read_timeout 10s;
        }
    }
}1
（四）数据库主节点 db01 配置
1. 安装MySQL 5.7
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
    worker_connections 65535;
useepoll;
    multi_accept on;
}

http {
include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
    access_log off; # 高并发关闭访问日志，提升性能

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    gzip on;
    gzip_types text/plain text/css application/json application/javascript;

# Web集群配置
    upstream web_cluster {
        server 192.168.1.20:80 max_fails=3 fail_timeout=10s;
        server 192.168.1.21:80 max_fails=3 fail_timeout=10s;
        ip_hash; # 会话保持，适配登录业务
    }

    server {
        listen 80;
        server_name _; # 适配所有域名

# 反向代理Web集群
        location / {
            proxy_pass http://web_cluster;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_connect_timeout 3s;
            proxy_read_timeout 10s;
        }
    }
}2
2. MySQL主配置 /etc/my.cnf
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
    worker_connections 65535;
useepoll;
    multi_accept on;
}

http {
include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
    access_log off; # 高并发关闭访问日志，提升性能

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    gzip on;
    gzip_types text/plain text/css application/json application/javascript;

# Web集群配置
    upstream web_cluster {
        server 192.168.1.20:80 max_fails=3 fail_timeout=10s;
        server 192.168.1.21:80 max_fails=3 fail_timeout=10s;
        ip_hash; # 会话保持，适配登录业务
    }

    server {
        listen 80;
        server_name _; # 适配所有域名

# 反向代理Web集群
        location / {
            proxy_pass http://web_cluster;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_connect_timeout 3s;
            proxy_read_timeout 10s;
        }
    }
}3
3. 初始化与主从授权
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
    worker_connections 65535;
useepoll;
    multi_accept on;
}

http {
include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
    access_log off; # 高并发关闭访问日志，提升性能

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    gzip on;
    gzip_types text/plain text/css application/json application/javascript;

# Web集群配置
    upstream web_cluster {
        server 192.168.1.20:80 max_fails=3 fail_timeout=10s;
        server 192.168.1.21:80 max_fails=3 fail_timeout=10s;
        ip_hash; # 会话保持，适配登录业务
    }

    server {
        listen 80;
        server_name _; # 适配所有域名

# 反向代理Web集群
        location / {
            proxy_pass http://web_cluster;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_connect_timeout 3s;
            proxy_read_timeout 10s;
        }
    }
}4
（五）数据库从节点 db02 配置
1. MySQL安装与主库一致，修改配置 /etc/my.cnf
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
    worker_connections 65535;
useepoll;
    multi_accept on;
}

http {
include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
    access_log off; # 高并发关闭访问日志，提升性能

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    gzip on;
    gzip_types text/plain text/css application/json application/javascript;

# Web集群配置
    upstream web_cluster {
        server 192.168.1.20:80 max_fails=3 fail_timeout=10s;
        server 192.168.1.21:80 max_fails=3 fail_timeout=10s;
        ip_hash; # 会话保持，适配登录业务
    }

    server {
        listen 80;
        server_name _; # 适配所有域名

# 反向代理Web集群
        location / {
            proxy_pass http://web_cluster;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_connect_timeout 3s;
            proxy_read_timeout 10s;
        }
    }
}5
2. 配置主从复制
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
    worker_connections 65535;
useepoll;
    multi_accept on;
}

http {
include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
    access_log off; # 高并发关闭访问日志，提升性能

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    gzip on;
    gzip_types text/plain text/css application/json application/javascript;

# Web集群配置
    upstream web_cluster {
        server 192.168.1.20:80 max_fails=3 fail_timeout=10s;
        server 192.168.1.21:80 max_fails=3 fail_timeout=10s;
        ip_hash; # 会话保持，适配登录业务
    }

    server {
        listen 80;
        server_name _; # 适配所有域名

# 反向代理Web集群
        location / {
            proxy_pass http://web_cluster;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_connect_timeout 3s;
            proxy_read_timeout 10s;
        }
    }
}6
六、业务验证测试
在web01/web02创建测试文件，验证全链路连通性：
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
    worker_connections 65535;
useepoll;
    multi_accept on;
}

http {
include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
    access_log off; # 高并发关闭访问日志，提升性能

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    gzip on;
    gzip_types text/plain text/css application/json application/javascript;

# Web集群配置
    upstream web_cluster {
        server 192.168.1.20:80 max_fails=3 fail_timeout=10s;
        server 192.168.1.21:80 max_fails=3 fail_timeout=10s;
        ip_hash; # 会话保持，适配登录业务
    }

    server {
        listen 80;
        server_name _; # 适配所有域名

# 反向代理Web集群
        location / {
            proxy_pass http://web_cluster;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_connect_timeout 3s;
            proxy_read_timeout 10s;
        }
    }
}7
访问 http://192.168.1.100/test.php，页面正常显示即部署成功。
七、大厂运维兜底建议
  • 监控告警：部署Prometheus+Grafana，监控CPU、内存、连接数、MySQL主从延迟
  • 数据备份：从库每日定时全量备份，保留7天，防止数据丢失
  • 扩容方案：并发上涨时，直接新增Web节点，添加至Nginx upstream即可
  • 安全加固：生产环境开启防火墙，仅放行80、3306、22端口，修改默认端口


推荐站内搜索：最好用的开发软件、免费开源系统、渗透测试工具云盘下载、最新渗透测试资料、最新黑客工具下载……
ZhouSa.com-宙飒天下网