From: Tatu Ylonen <ylo@cs.hut.fi>
Date: Sun, 21 Jan 1996 15:06:32 +0800
To: Alan Horowitz <alanh@infi.net>
Subject: Re: NSA vacuuming down Internet traffic
In-Reply-To: <199601190923.KAA10526@utopia.hacktic.nl>
Message-ID: <199601200205.EAA07759@trance.olari.clinet.fi>
MIME-Version: 1.0
Content-Type: text/plain


> Is there any open source - or otherwise - knowledge or speculation about
> which words/phrases the Terra-cycle cpu's are text-searching *for*?  If it
> were your responsibility to eavesdrop on Iranian terrorists - or French
> Commercial Attache reports to Paris - or to have UK nationals, off in
> their private room of your building, write down the name of every in
> America who expresses a libertarian dissatisfaction with the Republicrat
> regime - would you know for sure which words/phrases to key on?   It 
> doesn't sound like a tractable problem to me.

To me it does sound completely feasible (you don't need very good
accuracy).  I've personally run packet filters (for statistical
purposes only) on busy 10-mbit ethernets using BPF, FreeBSD, and 486
or pentium machines.  They easily keep up with little packet loss.

I understand T3 is 34 mbits, so only three times faster.  No problem
to optimize that much by specially written software, especially if you
can do some of the low level stuff in hardware.

As for the keyword search problem, it would easily be possible to scan
much of the data (say, tcp ports smtp, nntp, login, exec, ident) in
real time against a million-phrase dictionary (containing keywords,
e-mail addresses, names, abbreviations, etc.).  If there are
performance problems, you can first limit by
source/destination/protocol/port.  Only intercepts (e.g., entire tcp
connections) that pass this initial screening are passed on to other
machines for more complicated analysis.

Note also that many parts of the filtering problem parallelize quite
nicely.  For example, you can split the traffic to a number of
machines based on the value of the numerically smaller of the
source/destination addresses.

I don't see any technical problems in doing large-scale internet
monitoring.  The equipment needed is even cheap enought to be done by
motivated amateurs/individuals, assuming they can get a copy of the
raw data from the T3.

This is one of the reasons why strongly encrypting internet data is so
important.

    Tatu

See http://www.cs.hut.fi/ssh for information on SSH, the secure remote
login program.

See http://www.cs.hut.fi/crypto for information cryptography available
to anyone worldwide.