From TNPC issue #4.12...

Snooping Around on the Internet

by Dan Butler
June 14, 2001

If you live in the United States and possibly elsewhere, you most likely have been receiving "Privacy Policies" in the mail. Businesses are getting these policies into the hands of their customers in an effort to help educate consumers on what information is being collected about them and how it is being used.

The Internet has brought this "problem" to the forefront of consumers' minds. Data has always been collected and sold in the past through catalog, direct mail, telemarketing, and even software companies. Remember all those registration cards you used to fill out? Much of that information is sold to other companies. Many companies collect a sizeable portion of their yearly income through mailing list rental fees. However, this article is not about the pros and cons of gathering names and renting them. I have my opinions about the practice but don't want to start a debate about that topic either here or in email.

Instead I'd like to focus on the Internet and information gathered by the Web sites you visit. Many people would have you believe that Web sites gather all sorts of information about you as you surf, thus directly violating your privacy. I've spent a considerable amount of time over the past few years analyzing the data collected by Web sites for various companies and I'd like to set the record straight on what is and isn't gathered about you by most sites.

Every time you visit a Web site certain information is stored in a log file. There is one line written to the log file for each element on a Web page that you request. Each page could have several elements: the page itself, each graphic, sound file, style sheet, some script files, etc. are each an element. Each line of the log file contains the following information:

1. The IP address you are surfing from
2. Your username on a site that requires authentication
3. The date and time you are actually requesting an element
4. The element of the page you are viewing (a graphic for example)
5. The protocol you are using (i.e., HTTP or FTP)
6. If the element was successfully sent to your browser
7. The Web address of the page you were viewing before requesting this page (called the "referer" and historically misspelled)
8. The Web browser you are using
9. The operating system your machine is running

In addition, JavaScript (a programming language) can be run on the site you're visiting and it can determine other bits of information like your current screen resolution (more on scripts in a moment).

The most revealing items in the list are #1 (the IP address you're surfing from), #2 (your username on a site that requires authentication), and #9 (your machine's operating system).

#1 This can be the address of your DSL or cable modem if you have a static IP address. It can reveal the name of your ISP, giving clues about the general vicinity where you live or from where you are surfing the net. If you are surfing from the office it will generally be the address for your company.

#2 Your username can tell a Website exactly which pages you as a registered user have looked at.

#9 This can give someone an idea of where you were surfing just before coming to their site. This is an issue on a Web site that records your username, password, or other information in the URL. The next site you visit could wind up with that information in their logs. If you came from a search engine it will usually contain the terms you were searching for.

As you can imagine the log files tend to grow very large very quickly. Programs are written to analyze the data helping Webmasters optimize their Web sites.

An example of good uses of this information is a private site combining items #1 and #2 to determine if passwords are being shared instead of used by a single individual. Item #9 can help a Webmaster fine-tune a site's navigation and determine which parts of a site are most popular.

Here's a perfectly safe sample script to show you how this works; it's available on our Web site:
http://www.TheNakedPC.com/t/412/tr.cgi?dan1

Type the URL in a couple of times after visiting other sites to see the Referer information change.

What's the bottom line? Do log files contain information about you? Yes! Should be worried? In general, no, you should not. The sheer size of the log files on most sites is staggering. The better bet for protecting your privacy is to be careful what information you supply in forms you fill out both online and off.

Quick tips:

* Since the username is stored in the log and often in the referer, don't use your username for your password.

* If you don't want your referer information passed along, go to a neutral site before surfing to the next site. Example: Use your own home page if you have one or set your browser home page to be a blank page. Then just click your home button to clear out the referer setting.

Of course there are software programs to do all of this for you. I do not recommend you use them. It just slows you down and doesn't really protect you from anything. On the other hand cookies and some advertising tracking software, notably Double- Click.com, can and does track you with more certainty. I do use software to block that sort of activity. Specifically I use Norton Internet Security to block ads and pop-up windows.

Next issue I'll talk a bit about cookies, Double-Click, and how companies track your email reading habits. And I'll tell you what to do about it.

**********
Have a large Web site and need to know what's hiding in your logs? I can help. Drop me a line with your needs.
**********

You can reach Dan Butler at:
mailto:danbutler@TheNakedPC.com

Return to Top

TNPC Hot Tips:

• Email out of control? Spam filling your inbox? People trying to steal your identity? Same here - until I applied these tips. You can too in a new multimedia e-book. Tame Your Email.

• DO YOU MAKE THESE MONEY MISTAKES? Do you know that trying to pay off your high interest rate debts first and/or paying extra on more than one debt is the SLOWEST way to get out of debt? Don't make these same mistakes. Learn more at by clicking here...