awk；ansible 环境准备及软件部署

课前思考如果查看/var/log/messages中2026-04-14上午九点到下午1点之间段OpenSSH相关日志[rootshell ~ 09:58:35]# sed -n /Apr 14 09:/,/Apr 14 13:/p /var/log/messages | grep OpenSSH Apr 14 09:40:03 centos7 systemd: Starting OpenSSH server daemon... Apr 14 09:40:03 centos7 systemd: Started OpenSSH server daemon.如何过滤出/etc/profile中有效配置内容[rootshell ~ 10:01:48]# egrep -v ^\s*(#|$) /etc/profile使用echo {1..10}生成序列后如何使用sed改写成新序列12..910[rootshell ~ 10:03:36]# echo {1..10} | sed s/ //g 12345678910如何使用sed修改/etc/hosts在127.0.0.1开头的行下方添加新行内容是10.1.8.10 www.wanho.net www[rootshell ~ 10:05:26]# sed -i /^127\.0\.0\.1/a10.1.8.10 www.wanho.net www /etc/hosts [rootshell ~ 10:05:36]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 10.1.8.10 www.wanho.net www ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6今日内容2. awk3. ansible 环境准备4. ansible 配置5. ansible adhoc 命令6. playbook 编写和执行awkawk 介绍awk 是其取了三位创始人 AlfredAhoPeterWeinberger, 和 BrianKernighan 的 Family Name 的首字符。awk 是一个强大的文本分析工具。awk 更像一门编程语言他可以自定义变量有条件语句有循环有数组有正则有函数等。awk 按行读取数据根据给出的条件进行查找并在找出来的行中进行操作。awk 有三种形势awkgawknawk平时所说的awk其实就是gawk。awk 命令awk 命令格式awk [options] script file(s) awk [options] -f scriptfile file(s)script定义如何处理数据。file是 awk 处理的数据来源awk 也可以来自其它命令的输出。-f scriptfile从脚本文件中读取awk命令每行都是一个独立的script。script 格式如下BEGIN { action } pattern { action } END { action }脚本通常是被单引号或双引号包住一个awk脚本通常由四部分组成BEGIN { action }语句块awk 执行pattern { action }前要执行的脚本。pattern { action }语句块决定动作语句何时触发可以是以下任意一种正则表达式使用通配符的扩展集。关系表达式使用运算符进行操作可以是字符串或数字的比较测试。模式匹配表达式使用运算符~匹配和~!(不匹配)。END { action }语句块awk 执行pattern { action }后要执行的脚本。action部分决定对数据如何处理由一个或多个命令、函数、表达式组成之间由换行符或分号隔开并位于大刮号内。常见的action包括变量或数组赋值、输出命令、内置函数、控制流语句。BEGIN { action }、pattern { action }、END { action }都是可选项目。awk 工作流第一步执行BEGIN { commands }语句块中的语句。在awk从输入输出流中读取行之前执行通常在BEGIN语句块中执行如变量初始化打印输出表头等操作。第二步**从文件或标准输入中读取一行然后执行pattern { commands }语句块。**它逐行读取数据从第一行到最后一行重复这个过程直到读取完所有行。pattern语句块中的通用命令是最重要的部分它也是可选的。如果没有提供pattern语句块则默认执行{ print }即打印每一个读取到的行。{}类似一个循环体会对文件中的每一行进行迭代通常将变量初始化语句放在BEGIN语句块中将打印结果等语句放在END语句块中。第三步当读至输入流末尾时执行END { command }语句块。在awk从输入流中读取完所有的行之后执行比如打印所有行的分析结果它也是一个可选语句块。awk 示例示例1获取IP地址是10.1.8.10的网卡名rootshell ~ 10:22:06]# ip addr | grep 10.1.8.10 inet 10.1.8.10/24 brd 10.1.8.255 scope global noprefixroute ens33 [rootshell ~ 10:22:18]# ip addr | grep 10.1.8.10 | awk {print $NF} ens33 # 或者 [rootshell ~ 10:22:24]# ip addr | awk /10.1.8.10/ {print $NF} ens33示例2查看使用率超过10的文件系统[rootshell ~ 10:24:04]# df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 2.0G 0 2.0G 0% /dev tmpfs 2.0G 0 2.0G 0% /dev/shm tmpfs 2.0G 12M 2.0G 1% /run tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup /dev/mapper/centos-root 50G 1.6G 49G 4% / /dev/sda1 1014M 139M 876M 14% /boot /dev/mapper/centos-home 147G 33M 147G 1% /home tmpfs 394M 0 394M 0% /run/user/0 tmpfs 394M 0 394M 0% /run/user/1000 [rootshell ~ 10:24:06]# df -h | sed s/%// | awk $510 {print $0} Filesystem Size Used Avail Use Mounted on /dev/sda1 1014M 139M 876M 14 /boot # 或者 [rootshell ~ 10:24:10]# df -h | awk $51-110 {print $0} /dev/sda1 1014M 139M 876M 14% /boot\示例3提前系统运行数据。包括CPU 使用率内存 使用率当前用户登录数当前系统负载系统运行进程数量[rootshell bin 10:43:30]# pwd /root/bin [rootshell bin 10:43:33]# vim monitor_os.sh#!/bin/bash os_info_file/tmp/os_info.txt top -b -n 1 | head -n 5 ${os_info_file} cpu_usage$(awk /^%Cpu/ {print $2$4} ${os_info_file}) echo CPU 使用率${cpu_usage} memory_usage$(awk /^KiB Mem/ {print ($4-$6)/$4} ${os_info_file}) echo 内存使用率${memory_usage} user_count$(awk /^top/ {print $6} ${os_info_file}) echo 在线用户数量${user_count} load_usage$(awk /^top/ {print $(NF-2),$(NF-1),$NF} ${os_info_file}) echo 当前系统负载${load_usage} running_process_count$(awk /^Tasks/ {print $4} ${os_info_file}) echo 系统运行进程数量${running_process_count} rm -f ${os_info_file}ansible 环境准备ansible 架构控制节点下发指令或文件到受控制节点。受控制节点接受控制节点发过来的指令并执行。ansible 工作原理ansible控制节点通过ssh协议将python 模块推送到受控制节点受控制节点使用python执行python模块进行相应的配置。ansible 环境准备实验环境10.1.8.10 controller.laoma.cloud controller 10.1.8.11 server1.laoma.cloud server1 10.1.8.12 server2.laoma.cloud server2 10.1.8.13 server3.laoma.cloud server3 10.1.8.14 server4.laoma.cloud server4准备虚拟机模版准备1台干净的centos 7 虚拟机。注意模版虚拟机的CPU和内存的配置建议设置为1CPU和2G内存。开发脚本sethost不加参数执行sethost则提示命令使用方法。加参数执行sethost则第一个参数范围是10-14。超出范围也提示命令使用方法。正常执行示例sethost 10这设置正确的主机名和IP地址[rootshell ~ 13:34:31]# vim /usr/local/bin/sethost#!/bin/bash # test root user (( UID!0 )) echo Please run as root. exit 1 usageUsage: $0 10-14 # test args number (($# !1 )) echo $usage exit 2 # 根据实际情况修改网卡名和域名 con_nameens33 domain_namelaoma.cloud host_id$1 if ((host_id10));then HOSTNAMEcontroller.${domain_name} elif ((11host_id host_id14)) ;then HOSTNAMEserver$[host_id-10].${domain_name} else echo $usage exit 3 fi hostnamectl set-hostname $HOSTNAME nmcli connection modify ${con_name} ipv4.addresses 10.1.8.${host_id}/24 nmcli connection up ${con_name} hostname ip -br address [rootshell ~ 13:34:31]# chmod x /usr/local/bin/sethost克隆虚拟机基于模版虚拟机快照ansible克隆出其他虚拟机并使用sethost脚本设置主机名和IP地址。以server1为例[rootshell ~ 13:34:31]# sethost 11配置 ansible 基础环境在模版虚拟机上配置/etc/hosts添加ansible主机清单[rootdeploy ~ 17:17:21]# vim /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 ################# ansible #################### 10.1.8.10 controller.liang.cloud controller 10.1.8.11 server1.liang.cloud server1 10.1.8.12 server2.liang.cloud server2 10.1.8.13 server3.liang.cloud server3 10.1.8.14 server4.liang.cloud server4 ################# ansible ####################配置免密登录ansible节点[rootdeploy ~ 14:52:58]# ssh-keygen -t rsa -N -f .ssh/id_rsa [rootdeploy ~ 14:54:39]# \ for host in controller server1 server2 server3 server4 do sshpass -p123 ssh-copy-id root$host done [rootdeploy ~ 14:56:02]# for host in controller server1 server2 server3 server4 do ssh root$host hostname;ip -br a show ens33;echo done # 输出内容如下 controller.laoma.cloud ens33 UP 10.1.8.10/24 fe80::20c:29ff:fe14:2f8b/64 server1.laoma.cloud ens33 UP 10.1.8.11/24 fe80::20c:29ff:fe6a:559b/64 server2.laoma.cloud ens33 UP 10.1.8.12/24 fe80::20c:29ff:feec:ac41/64 server3.laoma.cloud ens33 UP 10.1.8.13/24 fe80::20c:29ff:fe19:59c1/64 server4.laoma.cloud ens33 UP 10.1.8.14/24 fe80::20c:29ff:feaa:4763/64在模版虚拟机上开发脚本weihu用来集中管理其他机器。weihu cmd command将会在ansible 5台设备上执行command。weihu copy src dest将模版虚拟机上的src文件复制到ansible 5台设备dest位置。[rootdeploy ~ 15:29:04]# vim /usr/local/bin/weihu #!/bin/bash function usage () { echo Usage: weihu cmd COMMAND, 在集群中所有的机器上执行对应COMMAND命令 echo Usage: weihu copy source target将本地source文件推送到集群中所有的机器上 exit } action$1 HOSTLISTcontroller server1 server2 server3 server4 (( $#1 )) usage case $action in cmd) # 删除参数1 shift COMMAND$* for host in $HOSTLIST do ssh root$host $COMMAND done ;; copy) # 删除参数1 shift for host in $HOSTLIST do num$# case $num in 2) scp -r $1 root$host:$2 ;; #[3-9]|[1-9][0-9]) [3-9]) last$(echo $* | awk {print $NF}) args_exclude_last$(echo $* | sed s#$last##) scp -r ${args_exclude_last} root$host:${last} ;; *) usage esac done ;; *) usage ;; esac [rootdeploy ~ 15:30:57]# chmod x /usr/local/bin/weihu测试维护脚本[rootdeploy ~ 15:31:41]# weihu cmd hostname controller.liang.cloud server1.liang.cloud server2.liang.cloud server3.liang.cloud server4.liang.cloud [rootdeploy ~ 15:32:36]# weihu copy /etc/hosts /etc/hosts hosts 100% 451 310.0KB/s 00:00 hosts 100% 451 207.4KB/s 00:00 hosts 100% 451 191.7KB/s 00:00 hosts 100% 451 191.7KB/s 00:00 hosts 100% 451 224.6KB/s 00:00 [rootdeploy ~ 15:33:01]# weihu cmd cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 ################# ansible #################### 10.1.8.10 controller.liang.cloud controller 10.1.8.11 server1.liang.cloud server1 10.1.8.12 server2.liang.cloud server2 10.1.8.13 server3.liang.cloud server3 10.1.8.14 server4.liang.cloud server4 ################# ansible #################### 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 ################# ansible #################### .......准备一个专用的账户devops用于控制节点远程登录其他节点。[rootdeploy ~ 15:33:33]# weihu cmd useradd devops [rootdeploy ~ 15:52:53]# weihu cmd id devops uid1001(devops) gid1001(devops) groups1001(devops) uid1001(devops) gid1001(devops) groups1001(devops) uid1001(devops) gid1001(devops) groups1001(devops) uid1001(devops) gid1001(devops) groups1001(devops) uid1001(devops) gid1001(devops) groups1001(devops) [rootdeploy ~ 15:54:18]# sshpass -p 123456|passwd --stdin devops ^C [rootdeploy ~ 15:55:10]# sshpass -p 123456 ssh devopsserver1 id uid1001(devops) gid1001(devops) groups1001(devops) # 配置devops账户免密提权为root [rootdeploy ~ 15:55:48]# weihu cmd echo devops ALL(ALL)NOPASSWD:ALL /etc/sudoers.d/devops配置控制节点使用devops账户免密登录所有ansible节点。[rootdeploy ~ 16:02:28]# sshpass -p123456 ssh devopscontroller ssh-keygen -t rsa -N -f .ssh/id_rsa [rootdeploy ~ 16:03:38]# for host in controller server1 server2 server3 server4;do ssh devops$host id;done # 验证控制节点免密登录其他节点 [devopscontroller ~ 16:10:48]$ for host in controller server1 server2 server3 server4;do uid1001(devops) gid1001(devops) groups1001(devops) uid1001(devops) gid1001(devops) groups1001(devops) uid1001(devops) gid1001(devops) groups1001(devops) uid1001(devops) gid1001(devops) groups1001(devops) uid1001(devops) gid1001(devops) groups1001(devops) [devopscontroller ~ 16:10:58]$ for host in controller server1 server2 server3 server4;do uid0(root) gid0(root) groups0(root) uid0(root) gid0(root) groups0(root) uid0(root) gid0(root) groups0(root) uid0(root) gid0(root) groups0(root) uid0(root) gid0(root) groups0(root)ansible 配置部署 ansible软件控制节点[devopscontroller ~ 16:32:43]$ sudo yum install ansible -y受管理节点[rootdeploy ~ 17:16:34]# weihu cmd yum install -y python配置主机清单ansible管理的主机信息要保存到一个文件中这个文件称之为主机清单inventory。[devopscontroller ~ 17:15:11]$ mkdir ansible [devopscontroller ~ 17:17:41]$ cd ansible/ [devopscontroller ansible 17:17:48]$ vim inventory controller server1 server2 server3 server4 [devopscontroller ansible 17:18:58]$ ansible -i inventory -m command -a id server1 server1 | CHANGED | rc0 uid1001(devops) gid1001(devops) groups1001(devops) # 参数说明 # -i inventory主机清单位置 # -m command使用命令模块 # -a hostname模块对应的具体参数 # server1针对哪个机器操作 [devopscontroller ansible 17:19:15]$ ansible -i inventory -m user -a namezhangsan statepresent -b server1 server1 | CHANGED { ansible_facts: { discovered_interpreter_python: /usr/bin/python }, changed: true, comment: , create_home: true, group: 1002, home: /home/zhangsan, name: zhangsan, shell: /bin/bash, state: present, system: false, uid: 1002 } [devopscontroller ansible 17:20:41]$ ansible -i inventory -m command -a id zhangsan server1 server1 | CHANGED | rc0 uid1002(zhangsan) gid1002(zhangsan) groups1002(zhangsan) [devopscontroller ansible 17:21:55]$ ansible -i inventory -m user -a namezhangsan stateabsent removeyes -b server1 server1 | CHANGED { ansible_facts: { discovered_interpreter_python: /usr/bin/python }, changed: true, force: false, name: zhangsan, remove: true, state: absent } # 删除用户 [devopscontroller ansible 17:21:55]$ ansible -i inventory -m user -a namezhangsan stateabsent removeyes -b server1 server1 | CHANGED { ansible_facts: { discovered_interpreter_python: /usr/bin/python }, changed: true, force: false, name: zhangsan, remove: true, state: absent } [devopscontroller ansible 17:24:40]$ ansible -i inventory -m command -a id zhangsan server1 server1 | FAILED | rc1 id: zhangsan: no such usernon-zero return code分组主机清单[devopscontroller ansible 17:24:58]$ vim inventory [controllers] controller [webs] server1 server2 [dbs] server3 server4测试# 针对webs主机组操作 [devopscontroller ansible 17:25:52]$ ansible -i inventory -m command -a hostname -o webs server1 | CHANGED | rc0 | (stdout) server1.liang.cloud server2 | CHANGED | rc0 | (stdout) server2.liang.cloud # -o 输出内容合并为一行 # all代表所有机器 [devopscontroller ansible 17:27:24]$ ansible -i inventory -m command -ahostname -o all server1 | CHANGED | rc0 | (stdout) server1.liang.cloud server2 | CHANGED | rc0 | (stdout) server2.liang.cloud server4 | CHANGED | rc0 | (stdout) server4.liang.cloud controller | CHANGED | rc0 | (stdout) controller.liang.cloud server3 | CHANGED | rc0 | (stdout) server3.liang.cloud # web主机组安装nginx [devopscontroller ansible 17:36:43]$ ansible -i inventory -m yum -a namenginx statepresent -b webs # web主机组卸载nginx [devopscontroller ansible 17:37:48]$ ansible -i inventory -m yum -a namenginx stateabsent -b webs查看模块帮助# 查看模块清单并过滤出yum相关信息 [devopscontroller ansible 16:53:41]$ ansible-doc -l | grep yum yum Manages packages with the yum package manager yum_repository Add or remove YUM repositories # 查看yum模块帮助信息 [devopscontroller ansible 16:53:47]$ ansible-doc yum # 直接搜索EXAMPLE # 部分内容如下 EXAMPLES: - name: install the latest version of Apache yum: name: httpd state: latest - name: ensure a list of packages installed yum: name: {{ packages }} vars: packages: - httpd - httpd-tools - name: remove the Apache package yum: name: httpd state: absent - name: install the latest version of Apache from the testing repo yum: name: httpd enablerepo: testing state: present ...ansible最大的特点简单只要能看懂English就行。幂等性多次执行结果一致。假设第一次执行软件包未安装则执行安装。第二次执行则不需要做任何事情。playbook 编写和执行通过编写playbook完成重复、复杂的任务。ansible 命令类似于 shell 中单个命令。playbook 类似于 shell 脚本。[devopscontroller ansible 16:55:36]$ vim deploy_web.yaml# yaml格式起始行一般不省略 --- # Playbook中第一个play # play具有属性namehostsbecometasks缩进一致 # name属性用于简要描述play - name: debploy WebSite # hosts属性用于定义要在哪个受管理节点执行 hosts: webs # tasks属性用于描述play中任务属性是列表格式 tasks: # 第一个任务 # 任务具有属性涵name和模块名等。 # name属性用于简要描述任务 - name: latest version of httpd and firewalld installed # 指明模块名也就是要执行的任务 yum: # 指定要操作的rpm包名称 name: # rpm包名称是-开头的列表格式或者逗号分隔的列表格式 - httpd - firewalld # 定义软件包的状态lastet代表升级为最新版本 state: latest # 第二个任务 - name: prepare index.html # copy 模块用于将content属性值写入到目标文件 copy: content: Welcome to {{ ansible_fqdn }} WebSite!\n dest: /var/www/html/index.html # 第三个任务 - name: enable and start httpd # service模块用于启用并启动httpd服务 service: name: httpd enabled: true state: started # 第四个任务 - name: enable and start firewalld # service模块用于启用并启动firewalld服务 service: name: firewalld enabled: true state: started # 第五个任务 - name: firewalld permits access to httpd service # firewalld用于放行http服务 firewalld: service: http permanent: true state: enabled immediate: yes # Playbook中第二个play-开头表示列表 - name: Test WebSite hosts: localhost become: no tasks: - name: connect to intranet web server # uri模块用于测试网站是否可以访问 uri: url: http://{{item}} loop: - server1 - server2 # yaml格式结束行一般省略 ...# 执行剧本 [devopscontroller ansible 17:01:40]$ ansible-playbook -i inventory -b deploy_web.yaml