1. Computer problem? Tech Support Guy is completely free -- paid for by advertisers and donations. Click here to join today! If you're new to Tech Support Guy, we highly recommend that you visit our Guide for New Members.

An awk / sed command to remove invalid rows

Discussion in 'Linux and Unix' started by grinder182533, Jan 5, 2011.

Thread Status:
Not open for further replies.
Advertisement
  1. grinder182533

    grinder182533 Thread Starter

    Joined:
    Jun 1, 2009
    Messages:
    72
    My file has rows with 10 columns, separated by 9 “|” (pipe) separators.

    However, every now and again, I am getting spurious rows appearing with no separators at all, or other rows appearing with fewer than the 9 separators.

    Unfortunately there is no way for me to prevent them from appearing from within the application.

    Is there an awk command, or, preferably, a sed command, which anyone can provide which will strip out these non-standard rows ??

    Many thanks in advance.
     
  2. lotuseclat79

    lotuseclat79

    Joined:
    Sep 12, 2003
    Messages:
    20,583
    Hi grinder182533,

    In awk (and nawk, gawk), fields by default are any character that is not a space or a tab, so to process your input file for awk processing, it is first necessary to define your field separator, '|' and then test each line for the number of fields discarding or redirecting output to an error file for any the do not meet the number of fields requirement, and to pass on the remaining input that satisfies your criteria.

    The following code demonstrates a solution which satisfies the input file requirement of the application which can be piped into the application, but drops the faulty input lines:

    $ cat input-file | nawk 'BEGIN { FS = "|" } END { if ( NF == 9) print $0 }'

    -- Tom
     
  3. grinder182533

    grinder182533 Thread Starter

    Joined:
    Jun 1, 2009
    Messages:
    72
    Thanks Tom
    Is your code designed to update input-file, or should I append " > output-file" ?
    I ran your code as is, but input-file remained unaltered, with the short rows still present.
    I then reran the code, this time with "> output-file", but unfortunatley it was empty.
    Thanks again for your support.
     
  4. lotuseclat79

    lotuseclat79

    Joined:
    Sep 12, 2003
    Messages:
    20,583
    Hi grinder182533,

    Sorry - I should have tested it - simply change 9 to 10 and it should work!

    -- Tom
     
As Seen On
As Seen On...

Welcome to Tech Support Guy!

Are you looking for the solution to your computer problem? Join our site today to ask your question. This site is completely free -- paid for by advertisers and donations.

If you're not already familiar with forums, watch our Welcome Guide to get started.

Join over 733,556 other people just like you!

Loading...
Similar Threads - command remove invalid
  1. binny18
    Replies:
    10
    Views:
    222
  2. david112211
    Replies:
    3
    Views:
    300
Thread Status:
Not open for further replies.

Short URL to this thread: https://techguy.org/972762

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice