awk

Best way to pull repeated table data from a pcap http file (could awk handle the disruptive breaks)?

牧云@^-^@ 提交于 2020-01-25 06:52:03
问题 I am collecting data readings from my PV system. The web client will graph one day of data - I want to collect a whole year or two in one file for patterns etc. So far I capture lines into a cap file with Wireshark and just filter the data I want with headers and a few retransmitted packet. The data of interest is being sent to a js app but I want to lift out the data which repeats in each packet as date time=watts, see sample below... I was hoping to use AWK to parse the data into an array

Find the durations and their maximum between the dataset in shell script

。_饼干妹妹 提交于 2020-01-25 06:51:11
问题 I have a dataset as: ifile.txt 2 3 2 3 2 20 2 0 2 0 0 2 1 2 5 6 7 0 3 0 3 4 5 I would like to find out different duration and their maximum between the 0 values. My desire output is: ofile.txt 7 20 1 2 6 7 1 3 3 5 Where 7 is the number of counts until next 0 (i.e. 2,3,2,3,2,20,2) and 20 is the maximum value among them 1 is the number of counts until next 0 (i.e. 2) and 2 is the maxmimum 6 is the number of counts until next 0 (i.e. 2,1,2,5,6,7) and 7 is the maximum among them and so on

Create timestamp with fractional seconds

柔情痞子 提交于 2020-01-25 05:45:06
问题 awk can generate a timestamp with strftime function, e.g. $ awk 'BEGIN {print strftime("%Y/%m/%d %H:%M:%S")}' 2019/03/26 08:50:42 But I need a timestamp with fractional seconds, ideally down to nanoseconds. gnu date can do this with the %N element: $ date "+%Y/%m/%d %H:%M:%S.%N" 2019/03/26 08:52:32.753019800 But it is relatively inefficient to invoke date from within awk compared to calling strftime , and I need high performance as I'm processing many large files with awk and need to generate

awk的基本使用

我的梦境 提交于 2020-01-25 04:15:49
awk的主要作用是在可以针对查找文本定义模块化功能,实现复杂的字符串操作.awk是一个行文本处理命令,会通过默认的记录分割符(RS)将文本分割为多个记录,然后使用域分割符(FS)来将记录内容进行分割并赋值给临时的变量进行相关的操作. 基本介绍 通过定义不同的条件完成指定的操作,基本格式如下: awk '条件1 {动作 1} 条件2{动作 2} …' 文件名 条件(Pattern): awk支持的主要条件类型 条件类型 条 件 说 明 awk保留字 BEGIN 在 awk 程序一开始,尚未读取任何数据之前执行。BEGIN 后的动作只在程序开始时执行一次 awk保留字 END 在 awk 程序处理完所有数据,即将结束时执行?END 后的动作只在程序结束时执行一次 关系运算符 > 大于 < 小于 >= 大于等于 <= 小于等于 == 等于。用于判断两个值是否相等。如果是给变童赋值,则使用"=” != 不等于 A~B 判断字符串 A 中是否包含能匹配 B 表达式的子字符串 A!~B 判断字符串 A 中是否不包含能匹配 B 表达式的子字符串 正则表达式 /正则/ 如果在“//”中可以写入字符,则也可以支持正则表达式 动作(Action) 格式化输出 流程控制 在默认处理中,awk会首先使用默认的记录分割符(Record Seperator,缩写为RS,默认的记录分割符为"\n")进行记录分割

ubuntu打乱txt文件的顺序

拜拜、爱过 提交于 2020-01-25 02:27:45
源博客地址:http://blog.csdn.net/u010555688/article/details/50475739 留下备忘 将 train_160309-train.txt按行打乱,每行内容则保持不变,命令: cd 存放文件的路径 awk 'BEGIN{ 100000*srand();}{ printf "%s %s\n", rand(), $0}' train_160309-train.txt |sort -k1n | awk '{gsub($1FS,""); print $0}' 但这样处理后只是在屏幕上输出显示,如果需要将输出写入新的文本train.txt,则在末尾加上 | tee train.txt: awk 'BEGIN{ 100000*srand();}{ printf "%s %s\n", rand(), $0}' train_160309-train.txt |sort -k1n | awk '{gsub($1FS,""); print $0}' | tee train.txt 如果不需要在屏幕上输出显示,直接将输出写入新的文本train.txt,则在末尾将 | tee 换作 > 即可: awk 'BEGIN{ 100000*srand();}{ printf "%s %s\n", rand(), $0}' train_160309-train.txt

AWK associative array, mapping

a 夏天 提交于 2020-01-25 00:47:28
问题 Suppose I have two files: file1 - map.txt 1, 178246 2, 289789 3, 384275 4, 869282 file2 - relation.txt 178246, 289789 384275, 178246 384275, 869282 Expected results are: 1, 2 3, 1 3, 4 But the results I got using the following code were: awk 'FNR==NR{map[$2]=$1} {$1=map[$1];$2=map[$2];print $0}' map.txt relation.txt 2, 1, 4, It was confused when I swapped the columns in map.txt like this: 178246, 1 289789, 2 384275, 3 869282, 4 relation.txt doesn't change The results became: awk 'FNR==NR{map[

Combine multiple text files using awk

我与影子孤独终老i 提交于 2020-01-24 23:40:14
问题 I have some minutely stats saved in text files and named as 1min.txt, 2min.txt etc. 1min.txt F1,21 F2,32 F3,22 2min.txt F2,12 F4,32 I would like to combine these files in the following format: combined.txt Field 1min 2min F1 21 0 F2 32 12 F3 22 0 F4 0 32 Some fields may not exist in some files and 0 will be entered for those fields. I've tried to do it using awk but couldn't find an easy way. Can someone please help? Thanks 回答1: Using awk : awk -F, ' !seen[FILENAME]++ { fname[++numFile] =

巡检总合

余生长醉 提交于 2020-01-24 22:05:46
#!/bin/bash ###系统信息#### getsys(){ #系统类型 os_type= uname #系统版本 os_ver= cat /etc/redhat-release #系统内核 os_ker= uname -a|awk '{print $3}' #当前时间 os_time= date +%F_%T #运行时间 os_run_time= uptime |awk '{print $3,$4}'|awk -F ',' '{print $1}' #最后重启时间 os_last_reboot= who -b|awk '{print $3}' #本机名称 os_hostname= hostname echo “系统类型: ${os_type}” echo “系统版本: ${os_ver}” echo “系统内核: ${os_ker}” echo “当前时间: ${os_time}” echo “运行时间: ${os_run_time}” echo “最后重启时间: ${os_last_reboot}” echo “本机名称: ${os_hostname}” } ###网络信息#### getnet(){ ipaddr=( ifconfig |grep -w inet|awk '{print $2}' ) echo “本机的ip地址:${ipaddr[@]}”

How to generate a range of nonweekend dates using tools available in bash?

左心房为你撑大大i 提交于 2020-01-24 17:52:05
问题 I want to generate a list of files where the name consists of ${filename}.${date} , for example file.20111101 , file.20120703 , starting November 1, 2011 until today and it should exclude weekends. Thanks. 回答1: try this for 2011 for y in 2011; do for m in {1..12}; do for d in `cal -m $m $y|tail -n +3|cut -c 1-15`; do printf "file.%04d%02d%02d\n" $y $m $d; done; done; done or this for NOV-2011 to DEC-2013 for ym in {2011' '{11,12},{2012..2013}' '{1..12}}; do read y m <<<$ym; for d in `cal -m

how can i mask the middle line of a pem key in bash script

安稳与你 提交于 2020-01-24 15:04:39
问题 how can I mask the middle lines of a PEM key in bash script I need to echo my pem key with mask(*) the middle lines through a bash script for example:- -----BEGIN CERTIFICATE----- MIICyjCCAbICCQDrpZYh8et7yTANBgkqhkiG9w0BAQsFADAnMQswCQYDVQQGEwJV UzELMAkGA1UECAwCQ0ExCzAJBgNVBAcMAlNGMB4XDTE4MTExMjIwNDEwNVoXDTE4 MTIxMjIwNDEwNVowJzELMAkGA1UEBhMCVVMxCzAJBgNVBAgMAkNBMQswCQYDVQQH DAJTRjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAJnIdgpml8+xk+Oj 1RGMCyJ1P15RiM6rdtszT