Thursday, October 27, 2005

Gmail indexing URLs?

I’ve never really been a conspiracy theory kinda guy…but even this one made pause for a moment…

A friend of mine from Australia wondered if I could send him a few shows from my Tivo, so after a bit of network debauchery, I was able to expose my transferred recordings via my web server. I put a link to the filesystem in one of the web accessible directories (but not somewhere you could browse to) and sent the link off in an email (via gmail).

A small number of minutes later, my connection ground to a halt. I’d told him not to xfer any recording until after 2am my time. I did a quick sniff to see who was using all my bandwidth, and it was a Google crawler!!

Security through obscurity is dead….if it wasn’t already. In the end, I passwd protected the directory and bounced the web server…the crawler buggered off after that.

It brings up an interesting debate…should they crawl any links they encounter, even in ppl’s private emails? One way to ‘secure’ a directory is to only give the URL’s to ppl you trust. In theory, the URL is then as secure as those ppl decide to make it (they could tell more ppl, or add publicly accessible links to the content, etc), but if they do nothing, it should remain ‘secure’. What this means is that to secure a directory, you have to employ passwd protection, IP checks, user agent blocking, or referrer checks….all of which are a pain in the ass for a non-tech person.

Just beware of what links you send out. It’s all being crawled, man.


At 4:07 PM, Anonymous Anonymous said...

N.J. students ordered to take down blogs
A Roman Catholic high school has ordered its students to remove personal blogs from the Internet in the name of protecting them from cyberpredators.
Find out how to buy and sell anything, like things related to highway construction delay on interest free credit and pay back whenever you want! Exchange FREE ads on any topic, like highway construction delay!

At 4:12 PM, Anonymous Martin said...

Well, technically... security through obscurity isn't. :-)

At 9:21 PM, Blogger Brian said...

Blimey, I think you'd better turn on the comment protection Blogspot (or is it Blogger?) offers :)

As others have said, Google has shown itself to be a good citizen so far, indexing the bejeezus out of everything but not being Evil yet. Of course, Phase 2 could begin any day now, and I don't have a seat on the rocket... I say let 'em keep going...hell, we can't stop them anyway...and see what they do with it.

At 12:26 PM, Blogger markmiller00400359 said...

I read over your blog, and i found it inquisitive, you may find My Blog interesting. My blog is just about my day to day life, as a park ranger. So please Click Here To Read My Blog

At 12:14 AM, Blogger joeprehiem6809471598 said...

I read over your blog, and i found it inquisitive, you may find My Blog interesting. So please Click Here To Read My Blog

At 10:34 AM, Blogger benprice60415059 said...

Get any Desired College Degree, In less then 2 weeks.

Call this number now 24 hours a day 7 days a week (413) 208-3069

Get these Degrees NOW!!!

"BA", "BSc", "MA", "MSc", "MBA", "PHD",

Get everything within 2 weeks.
100% verifiable, this is a real deal

Act now you owe it to your future.

(413) 208-3069 call now 24 hours a day, 7 days a week.


Post a Comment

<< Home