1. 技术文章
  2. Unix Shell Script get only needed lines from a file content

Unix Shell Script get only needed lines from a file content

由 百般思念 提交于 2019-12-25 07:32:46

问题


EDIT

I have a file containing line of strings

mediafire.com/?a6o37kfc7m68dri

NOTE: that file on media fire is actually "logs.out" i've just renamed it to logs.txt when i uploded it.

example Content:(logs.out)

BIF in bif_init> login successful
BIF in dbInit> INFO: DB status:20,successful
BIF in dbInit> INFO: get the fileName need be query DB. Size(0) 
BIF in dbInit> INFO: get the fileName need be query DB. Size(30) 
BIF in modInit> INFO: modInit finished
BIF in setInputTag> set input tag value, name/value:ACCOUNT_NO/4302253 
BIF in setInputTag> set input tag value, name/value:bill_ref_no/199863752 
BIF in setInputTag> set input tag value, name/value:bill_ref_resets/1 
BIF in setInputTag> set input tag value, name/value:DISPUTE_AMOUNT/0 
BIF in setInputTag> set input tag value, name/value:NOD_FLAG/0 
BIF in setInputTag> set input tag value, name/value:OUTSTANDING_BALANCE/12904
BIF in setInputTag> set input tag value, name/value:TOTAL_DUE/12904 
BIF in run> INFO: updateParameterValues finished
BIF in getFile> INFO: Feed file(/arbordata/feedfile/PS-M07-1205-20130807-20130911-201711-1-bif01.DAT) for current invoice
BIF in insertNewFile> m_pcurFileNameAttr->file_seq_no:(1)  
BIF in insertNewFile> INFO:Output fileHeader(000020130911201711
    ) to feedfile,successful 
BIF in setInputTag> set input tag value, name/value:ACCOUNT_NO/4302253 
BIF in setInputTag> set input tag value, name/value:bill_ref_no/199863752 
BIF in setInputTag> set input tag value, name/value:bill_ref_resets/1 
BIF in setInputTag> set input tag value, name/value:DISPUTE_AMOUNT/0 
BIF in setInputTag> set input tag value, name/value:NOD_FLAG/0 
BIF in setInputTag> set input tag value, name/value:OUTSTANDING_BALANCE/12904
BIF in setInputTag> set input tag value, name/value:TOTAL_DUE/12904 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(BILLZIPCODE) into ACCT map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(BILLEDACCOUNTNUMBER) into ACCT map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(BILLNUMBER) into ACCT map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(BILLNUMBERRESETS) into ACCT map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(OWNINGACCOUNTNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBERRESET) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(DEFAULTEXTERNALID) into SI inherited_map 
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=2813096,0
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(OWNINGACCOUNTNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBERRESET) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(DEFAULTEXTERNALID) into SI inherited_map 
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=13415970,0
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(OWNINGACCOUNTNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBERRESET) into SI inherited_map
BIF in setInputTag> set input tag value, name/value:ACCOUNT_NO/2726650 
BIF in setInputTag> set input tag value, name/value:bill_ref_no/190155402 
BIF in setInputTag> set input tag value, name/value:bill_ref_resets/1 
BIF in setInputTag> set input tag value, name/value:DISPUTE_AMOUNT/0 
BIF in setInputTag> set input tag value, name/value:NOD_FLAG/0 
BIF in setInputTag> set input tag value, name/value:OUTSTANDING_BALANCE/164888
BIF in setInputTag> set input tag value, name/value:TOTAL_DUE/164888 
BIF in run> INFO: updateParameterValues finished
BIF in getFile> INFO: Feed file(/arbordata/feedfile/PS-M02-1204-20130602-20130911-201711-1-bif01.DAT) for current invoice
BIF in insertNewFile> m_pcurFileNameAttr->file_seq_no:(1)  
BIF in insertNewFile> INFO:Output fileHeader(000020130911201711
) to feedfile,successful 
BIF in setInputTag> set input tag value, name/value:ACCOUNT_NO/2726650 
BIF in setInputTag> set input tag value, name/value:bill_ref_no/190155402 
BIF in setInputTag> set input tag value, name/value:bill_ref_resets/1 
BIF in setInputTag> set input tag value, name/value:DISPUTE_AMOUNT/0 
BIF in setInputTag> set input tag value, name/value:NOD_FLAG/0 
BIF in setInputTag> set input tag value, name/value:OUTSTANDING_BALANCE/164888
BIF in setInputTag> set input tag value, name/value:TOTAL_DUE/164888 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(BILLZIPCODE) into ACCT map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(BILLEDACCOUNTNUMBER) into ACCT map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(BILLNUMBER) into ACCT map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(BILLNUMBERRESETS) into ACCT map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(OWNINGACCOUNTNUMBER) into SI inherited_map

Only line that wanted to have:

BIF in setInputTag> set input tag value, name/value:ACCOUNT_NO/4302253
BIF in setInputTag> set input tag value, name/value:bill_ref_no/199863752 
BIF in setInputTag> set input tag value, name/value:bill_ref_resets/1

BIF in setInputTag> set input tag value, name/value:ACCOUNT_NO/4302253 
BIF in setInputTag> set input tag value, name/value:bill_ref_no/199863752 
BIF in setInputTag> set input tag value, name/value:bill_ref_resets/1 

BIF in setInputTag> set input tag value, name/value:ACCOUNT_NO/2726650 
BIF in setInputTag> set input tag value, name/value:bill_ref_no/190155402 
BIF in setInputTag> set input tag value, name/value:bill_ref_resets/1 

BIF in setInputTag> set input tag value, name/value:ACCOUNT_NO/2726650 
BIF in setInputTag> set input tag value, name/value:bill_ref_no/190155402 
BIF in setInputTag> set input tag value, name/value:bill_ref_resets/1

and i want to output it into a file with the format like this:(horizontaly and with pipe delimiter)

BIF in setInputTag> set input tag value, name/value:ACCOUNT_NO/4302253|BIF in setInputTag> set input tag value, name/value:bill_ref_no/199863752|BIF in setInputTag> set input tag value, name/value:bill_ref_resets/1 
BIF in setInputTag> set input tag value, name/value:ACCOUNT_NO/4302253|BIF in setInputTag> set input tag value, name/value:bill_ref_no/199863752|BIF in setInputTag> set input tag value, name/value:bill_ref_resets/1
BIF in setInputTag> set input tag value, name/value:ACCOUNT_NO/2726650|BIF in setInputTag> set input tag value, name/value:bill_ref_no/190155402|BIF in setInputTag> set input tag value, name/value:bill_ref_resets/1
BIF in setInputTag> set input tag value, name/value:ACCOUNT_NO/2726650|BIF in setInputTag> set input tag value, name/value:bill_ref_no/190155402|BIF in setInputTag> set input tag value, name/value:bill_ref_resets/1

PLEASE HELP ME :( I want to create .sh file and it will run on PuTTy

when executing this code it is working

sed -n -e '/value:ACCOUNT_NO/{N;N;s/..\n\(.\)/|\1/g;p;}' logs.out> OUPUT_logs.out

BUT WHEN 

sed -n -e '/value:ACCOUNT_NO/{N;N;s/..\n\(.\)/|\1/g;p;}' logs.out> OUPUT_logs.txt

ther is "BLANK LINE" between every record.


回答1:


sed -n -e '/value:ACCOUNT_NO/{N;N;s/\n/|/g;p;}' logs.txt > outfile.txt

EDIT:

I've modified the command above to redirect the output to a file, but it's more important that we correct the output. I've tested the command using the example logs.txt you gave; I don't know enough about your system to say why it doesn't work there, so let us experiment. Try this:

sed -n -e '/value:ACCOUNT_NO/p' logs.txt

What is the output?

EDIT:

So far, so good. Now try this:

sed -n -e '/value:ACCOUNT_NO/{p;}' logs.txt

Same output?

EDIT:

Good! Now:

sed -n -e '/value:ACCOUNT_NO/{N;N;p;}' logs.txt

What is the output?

EDIT:

Last step, we redirect to outputfile.txt and remove the newlines. As @ChrisJester-Young said, we can remove the trailing spaces in the same step:

sed -n -e '/value:ACCOUNT_NO/{N;N;s/ \n/|/;p;}' logs.txt > outputfile.txt

EDIT

Try this, and let's call it code 6:

sed -n -e '/value:ACCOUNT_NO/{N;N;s/..\n/|/g;p;}' logs.txt > outputfile.txt

EDIT:

The problem is that your logs.txt has DOS-style line endings. I think that Code 6 changes them to UNIX-style, and ultra edit is warning you that they are strange. Try this (code 7):

sed -n -e '/value:ACCOUNT_NO/{N;N;s/..\n\(.\)/|\1/g;p;}'



回答2:


Make a shell script named, say, summarise_accounts.sh:

#!/bin/bash
exec paste -d'|' <(grep value:ACCOUNT_NO "$1") <(grep value:bill_ref_no "$1") <(grep value:bill_ref_resets "$1")

Make it executable (chmod +x summarise_accounts.sh), then run it like so:

./summarise_accounts.sh logs.txt >account_summary.txt

You can, of course, change the file names on the command line.




回答3:


You are running it on PuTTY. I guess it is a Windows version of logs.txt you have. First run dos2unix command on the log file and then execute the sh file as I have written :

dos2unix logs.txt

This is a.sh. The content of a.sh is the following :

#!/bin/sh  
awk '/ACCOUNT_NO/,/resets\/1/ {printf $0" "}' logs.txt | awk '{gsub(/resets\/1/,"resets/1\n");print}' | sed -e 's/ BIF/|BIF/g' -e 's/^ |BIF/BIF/g'

change the permission of the file before executing.

chmod a+x ./a.sh

Then execute the file

./a.sh > new_file.txt


来源：https://stackoverflow.com/questions/18839558/unix-shell-script-get-only-needed-lines-from-a-file-content

标签
bash
shell
unix
sh
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!