Advertisement

There's no such thing as a stupid question, but they're the easiest to answer.
Login
Search

Advertisement

Web Design & Development Web Design & Development
Search Search
Search for:
Tech Support Guy > > >

Solved: Crawling websites ...


(!)

JiminSA's Avatar
JiminSA   (Jim) JiminSA is offline JiminSA has a Profile Picture
Computer Specs
Member with 1,736 posts.
THREAD STARTER
 
Join Date: Dec 2011
Location: Johannesburg South Africa
Experience: Intermediate
29-Jun-2012, 01:14 PM #1
Solved: Crawling websites ...
Does anyone know about the legality of crawling or scanning through someone else's website looking for information?
allnodcoms's Avatar
allnodcoms   (Danny) allnodcoms is offline allnodcoms has a Profile Picture
Computer Specs
Member with 610 posts.
 
Join Date: Jun 2007
Location: Hertfordshire - England
Experience: Advanced
29-Jun-2012, 02:17 PM #2
It's not the looking mate, that's fine (Google have actually made a reasonably decent living out of doing exactly that), it's what you intend to do with the data you retrieve. That's where the headaches start.

As long as you follow (or no-follow if you get me!) the server's privacy settings then it's perfectly legal form a personal use perspective. It's all up there on the net (public domain) anyway, and automated searching, scraping and so on is fine. Just respect the robot.txt directives and don't dig too deep, it can get pretty dark down there...

Danny
JiminSA's Avatar
JiminSA   (Jim) JiminSA is offline JiminSA has a Profile Picture
Computer Specs
Member with 1,736 posts.
THREAD STARTER
 
Join Date: Dec 2011
Location: Johannesburg South Africa
Experience: Intermediate
30-Jun-2012, 12:54 AM #3
Yeh, Danny that's what concerns me - legal suits etc, But as you mention if it's displayed it's common data hey?

I'm not keen but a client has asked me to write something to scan a Classified Ads site for private car sales so that the mobile numbers could be used for an sms (text) group advertising campaign.

I rather think that it's not viable anyway because of the problem of getting that data into the campaign - unless the sellers have software to receive csv's or dat's or whatever (I shall approach them)??
JiminSA's Avatar
JiminSA   (Jim) JiminSA is offline JiminSA has a Profile Picture
Computer Specs
Member with 1,736 posts.
THREAD STARTER
 
Join Date: Dec 2011
Location: Johannesburg South Africa
Experience: Intermediate
30-Jun-2012, 03:15 AM #4
Ok, the seller interface won't be a problem, but the information I need will almost certainly be on a database, to which I do not have legal access, no?

I'll tell my client it's a no-go unless he wants to go to court!
allnodcoms's Avatar
allnodcoms   (Danny) allnodcoms is offline allnodcoms has a Profile Picture
Computer Specs
Member with 610 posts.
 
Join Date: Jun 2007
Location: Hertfordshire - England
Experience: Advanced
30-Jun-2012, 04:36 AM #5
Thumbs down Avoid...
Quote:
Originally Posted by JiminSA View Post
I'm not keen but a client has asked me to write something to scan a Classified Ads site for private car sales so that the mobile numbers could be used for an sms (text) group advertising campaign.
That is a Judge Judy moment waiting to happen mate. Bad Idea.

Danny
truebluexxx's Avatar
truebluexxx truebluexxx is offline truebluexxx has a Profile Picture
Member with 51 posts.
 
Join Date: Aug 2007
Location: Dundee
Experience: Advanced
07-Jul-2012, 10:24 AM #6
That is definitely illegal in the UK. You can only contact someone for marketing purposes if they have previously contacted you regarding your business, I think.

Which is why Yell.com give businesses free listings, they can then legally bother you constantly with marketing calls.

But if you only provide the code to get the numbers, and are outside the UK, then I have no idea.

I can't find the UK law covering it, I keep finding similar laws e.g.

http://www.out-law.com/page-11893

This talks about unwanted calls, which isn't the exact offence in the example you give. But I am sure it is in the Data protection act somewhere.

Have a read of this, http://www.tsplegal.com/services_for...t_marketing__1
It is not the law, but a solicitors view of it.
JiminSA's Avatar
JiminSA   (Jim) JiminSA is offline JiminSA has a Profile Picture
Computer Specs
Member with 1,736 posts.
THREAD STARTER
 
Join Date: Dec 2011
Location: Johannesburg South Africa
Experience: Intermediate
07-Jul-2012, 10:51 AM #7
Thanks for your input truebluexxx - interesting point
Quote:
You can only contact someone for marketing purposes if they have previously contacted you regarding your business
consumer protection, I suppose - but if we contact them about their advertisement for their vehicle, having gleaned it from the web, surely that's not illegal? We are simply responding to their Ad.
truebluexxx's Avatar
truebluexxx truebluexxx is offline truebluexxx has a Profile Picture
Member with 51 posts.
 
Join Date: Aug 2007
Location: Dundee
Experience: Advanced
07-Jul-2012, 11:19 AM #8
Anyone can contact them about the advertisement, but it doesn't sound like the company you are thinking of working with want to buy the cars. It sounds like they want to advertise to these people.

Found the ICO's guidelines (they enforce the law, or don't, as it seems but that doesn't make it legal)
http://www.ico.gov.uk/for_organisati...marketing.aspx

Read the part headed "Electronic mail marketing" read all of it, and pay attention to the all the 'or's and 'and's

Someone I know only read bullet point 2 up to the semi colon and got totally the wrong meaning.

"Electronic mail marketing" confusingly means, electronic messages sent by telephone, fax, email or text.
JiminSA's Avatar
JiminSA   (Jim) JiminSA is offline JiminSA has a Profile Picture
Computer Specs
Member with 1,736 posts.
THREAD STARTER
 
Join Date: Dec 2011
Location: Johannesburg South Africa
Experience: Intermediate
07-Jul-2012, 11:50 AM #9
Quote:
It sounds like they want to advertise to these people.
No, I've checked out the bona-fide of my potential clients and they need stock urgently and want to acquire it, legally (in fact the Director is an Advocate).

Getting back to the original question of legality - it pertained to the legality of scanning a website database...
Quote:
but the information I need will almost certainly be on a database, to which I do not have legal access, no?
So my need to know concerns the legality of accessing data which the owner has already made public on their site???
truebluexxx's Avatar
truebluexxx truebluexxx is offline truebluexxx has a Profile Picture
Member with 51 posts.
 
Join Date: Aug 2007
Location: Dundee
Experience: Advanced
07-Jul-2012, 12:31 PM #10
Sorry for focusing on the legality of how the information is going to be used.(One of the few laws I know a bit about)
I agree with everything Allnodcoms said, "respect the robot.txt".
JiminSA's Avatar
JiminSA   (Jim) JiminSA is offline JiminSA has a Profile Picture
Computer Specs
Member with 1,736 posts.
THREAD STARTER
 
Join Date: Dec 2011
Location: Johannesburg South Africa
Experience: Intermediate
07-Jul-2012, 12:51 PM #11
Yup - I have already approached the site owners asking permission and will continue according to their response a) for common courtesy and b) for legality... But thanks for your participation - most enlightening!
As Seen On

BBC, Reader's Digest, PC Magazine, Today Show, Money Magazine
WELCOME TO TECH SUPPORT GUY!

Are you looking for the solution to your computer problem? Join our site today to ask your question. This site is completely free -- paid for by advertisers and donations.

If you're not already familiar with forums, watch our Welcome Guide to get started.


(clock)
THIS THREAD HAS EXPIRED.
Are you having the same problem? We have volunteers ready to answer your question, but first you'll have to join for free. Need help getting started? Check out our Welcome Guide.

Search Tech Support Guy

Find the solution to your
computer problem!




Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools


WELCOME
You Are Using: Server ID
Trusted Website Back to the Top ↑