An awk / sed command to remove invalid rows

Status
This thread has been Locked and is not open to further replies. Please start a New Thread if you're having a similar issue. View our Welcome Guide to learn how to use this site.

grinder182533

Thread Starter
Joined
Jun 1, 2009
Messages
72
My file has rows with 10 columns, separated by 9 “|” (pipe) separators.

However, every now and again, I am getting spurious rows appearing with no separators at all, or other rows appearing with fewer than the 9 separators.

Unfortunately there is no way for me to prevent them from appearing from within the application.

Is there an awk command, or, preferably, a sed command, which anyone can provide which will strip out these non-standard rows ??

Many thanks in advance.
 
Joined
Sep 12, 2003
Messages
20,583
Hi grinder182533,

In awk (and nawk, gawk), fields by default are any character that is not a space or a tab, so to process your input file for awk processing, it is first necessary to define your field separator, '|' and then test each line for the number of fields discarding or redirecting output to an error file for any the do not meet the number of fields requirement, and to pass on the remaining input that satisfies your criteria.

The following code demonstrates a solution which satisfies the input file requirement of the application which can be piped into the application, but drops the faulty input lines:

$ cat input-file | nawk 'BEGIN { FS = "|" } END { if ( NF == 9) print $0 }'

-- Tom
 

grinder182533

Thread Starter
Joined
Jun 1, 2009
Messages
72
Thanks Tom
Is your code designed to update input-file, or should I append " > output-file" ?
I ran your code as is, but input-file remained unaltered, with the short rows still present.
I then reran the code, this time with "> output-file", but unfortunatley it was empty.
Thanks again for your support.
 
Joined
Sep 12, 2003
Messages
20,583
Hi grinder182533,

Sorry - I should have tested it - simply change 9 to 10 and it should work!

-- Tom
 
Status
This thread has been Locked and is not open to further replies. Please start a New Thread if you're having a similar issue. View our Welcome Guide to learn how to use this site.

Users Who Are Viewing This Thread (Users: 0, Guests: 1)

As Seen On
As Seen On...

Welcome to Tech Support Guy!

Are you looking for the solution to your computer problem? Join our site today to ask your question. This site is completely free -- paid for by advertisers and donations.

If you're not already familiar with forums, watch our Welcome Guide to get started.

Join over 807,865 other people just like you!

Latest posts

Staff online

Top