1. Computer problem? Tech Support Guy is completely free -- paid for by advertisers and donations. Click here to join today! If you're new to Tech Support Guy, we highly recommend that you visit our Guide for New Members.

Solved: Crawling websites ...

Discussion in 'Web Design & Development' started by JiminSA, Jun 29, 2012.

Thread Status:
Not open for further replies.
Advertisement
  1. JiminSA

    JiminSA Thread Starter

    Joined:
    Dec 15, 2011
    Messages:
    2,058
    Does anyone know about the legality of crawling or scanning through someone else's website looking for information?
     
  2. allnodcoms

    allnodcoms

    Joined:
    Jun 30, 2007
    Messages:
    613
    It's not the looking mate, that's fine (Google have actually made a reasonably decent living out of doing exactly that), it's what you intend to do with the data you retrieve. That's where the headaches start.

    As long as you follow (or no-follow if you get me!) the server's privacy settings then it's perfectly legal form a personal use perspective. It's all up there on the net (public domain) anyway, and automated searching, scraping and so on is fine. Just respect the robot.txt directives and don't dig too deep, it can get pretty dark down there...

    Danny
     
  3. JiminSA

    JiminSA Thread Starter

    Joined:
    Dec 15, 2011
    Messages:
    2,058
    Yeh, Danny that's what concerns me - legal suits etc, But as you mention if it's displayed it's common data hey?

    I'm not keen but a client has asked me to write something to scan a Classified Ads site for private car sales so that the mobile numbers could be used for an sms (text) group advertising campaign.

    I rather think that it's not viable anyway because of the problem of getting that data into the campaign - unless the sellers have software to receive csv's or dat's or whatever (I shall approach them)??
     
  4. JiminSA

    JiminSA Thread Starter

    Joined:
    Dec 15, 2011
    Messages:
    2,058
    Ok, the seller interface won't be a problem, but the information I need will almost certainly be on a database, to which I do not have legal access, no?

    I'll tell my client it's a no-go unless he wants to go to court!
     
  5. allnodcoms

    allnodcoms

    Joined:
    Jun 30, 2007
    Messages:
    613
    That is a Judge Judy moment waiting to happen mate. Bad Idea.

    Danny
     
  6. truebluexxx

    truebluexxx

    Joined:
    Aug 6, 2007
    Messages:
    51
    That is definitely illegal in the UK. You can only contact someone for marketing purposes if they have previously contacted you regarding your business, I think.

    Which is why Yell.com give businesses free listings, they can then legally bother you constantly with marketing calls.

    But if you only provide the code to get the numbers, and are outside the UK, then I have no idea.

    I can't find the UK law covering it, I keep finding similar laws e.g.

    http://www.out-law.com/page-11893

    This talks about unwanted calls, which isn't the exact offence in the example you give. But I am sure it is in the Data protection act somewhere.

    Have a read of this, http://www.tsplegal.com/services_fo...klist_data_protection_and_direct_marketing__1
    It is not the law, but a solicitors view of it.
     
  7. JiminSA

    JiminSA Thread Starter

    Joined:
    Dec 15, 2011
    Messages:
    2,058
    Thanks for your input truebluexxx - interesting point
    consumer protection, I suppose - but if we contact them about their advertisement for their vehicle, having gleaned it from the web, surely that's not illegal? We are simply responding to their Ad.
     
  8. truebluexxx

    truebluexxx

    Joined:
    Aug 6, 2007
    Messages:
    51
    Anyone can contact them about the advertisement, but it doesn't sound like the company you are thinking of working with want to buy the cars. It sounds like they want to advertise to these people.

    Found the ICO's guidelines (they enforce the law, or don't, as it seems but that doesn't make it legal)
    http://www.ico.gov.uk/for_organisations/sector_guides/marketing.aspx

    Read the part headed "Electronic mail marketing" read all of it, and pay attention to the all the 'or's and 'and's

    Someone I know only read bullet point 2 up to the semi colon and got totally the wrong meaning.

    "Electronic mail marketing" confusingly means, electronic messages sent by telephone, fax, email or text.
     
  9. JiminSA

    JiminSA Thread Starter

    Joined:
    Dec 15, 2011
    Messages:
    2,058
    No, I've checked out the bona-fide of my potential clients and they need stock urgently and want to acquire it, legally (in fact the Director is an Advocate).

    Getting back to the original question of legality - it pertained to the legality of scanning a website database...
    So my need to know concerns the legality of accessing data which the owner has already made public on their site???
     
  10. truebluexxx

    truebluexxx

    Joined:
    Aug 6, 2007
    Messages:
    51
    Sorry for focusing on the legality of how the information is going to be used.(One of the few laws I know a bit about)
    I agree with everything Allnodcoms said, "respect the robot.txt".
     
  11. JiminSA

    JiminSA Thread Starter

    Joined:
    Dec 15, 2011
    Messages:
    2,058
    Yup - I have already approached the site owners asking permission and will continue according to their response a) for common courtesy and b) for legality... But thanks for your participation - most enlightening!
     
  12. Sponsor

As Seen On
As Seen On...

Welcome to Tech Support Guy!

Are you looking for the solution to your computer problem? Join our site today to ask your question. This site is completely free -- paid for by advertisers and donations.

If you're not already familiar with forums, watch our Welcome Guide to get started.

Join over 733,556 other people just like you!

Loading...
Thread Status:
Not open for further replies.

Short URL to this thread: https://techguy.org/1059046