From: tcmay@netcom.com (Timothy C. May)
Date: Wed, 14 Sep 94 13:03:46 PDT
To: cypherpunks@toad.com
Subject: The Importance of Filtering
In-Reply-To: <m0qkwrt-0003prC@ionia.gasco.com>
Message-ID: <199409141915.MAA19709@netcom11.netcom.com>
MIME-Version: 1.0
Content-Type: text/plain



The filtering/reptuations/volume issue has come up again.

And a couple of people are planning "reputation servers." I applaud
them for the effort, but I don't see them as needed for our list,
right now. That is, I don't plan to use them.

* what most people need are tools to filter out messages they don't
want to read. As we don't yet have quasi-intelligent agents that can
do this for us, this generally involves:

- filtering based on thread
- filtering based on author

* having these tools is more important to me than having a database of
what other people think about other people (reputation data bases).

There are several approaches:

* Unix kill-files, for those with the shells that support them

* Eudora-type filtering (ironically, Eudora is sold by Qualcomm, Phil
Karn's company), which allows various kinds of sorting.

* Extropians-style filtering, now being developed as a commercial
product by Harry Shapiro and Ray Cromwell. (To be clear about things,
Hugh Daniel, Eric Hughes, etc., expressed their willingness to install
the Extropians-style software shortly after it became available,
almost 2 years ago. Various delays ensued, then the offer by the
authors was put in limbo, then the commercialization phase ensued.)

* Anyone can operate a refector for the list, as per several
statements on this. Hal Finney, for example, offered (offers?) an
encrypted-only version. My point: someone could set up a filtering
service, a digester, whatever, and others could subscribe.

(Yes, Robert Hayden did this for a few weeks. While it may not have
been his "fault" that it went down, it shows the generally flaky and
catch-as-catch-can nature of so many part-time, hobbyist systems. Like
the remailers that go down when the laptop running it gets taken to
Spain for the summer :-}. A "for profit" service, at some quarterly
fee and with a contractual relationship to continue service, is a
better long-term approach.)

I sympathize with the concerns of Phil Karn and others, but let me
give a warning about this. The Extropians list, when I was on it, had
about the same volume at its peak that we now have, about 50-100
messages a day. Much debate about S/N ensued, much talk about charging
a fee for posting, about setting up quotas, about about official
reputation markets. At least 20% of all list traffic in some weeks was
devoted to kvetching about this problem.

A "reputation market" called the Hawthorne Exchange was set up, as
discussed here by Hal Finney (and also by me in my FAQ..grep for
Hawthorne or HEx). Even more jawboning went on.

And then of course there what the new list software. This allowed
folks to exclude authors, threads, etc., at the _point of
distribution_.

Is this a good idea.? Well, if one excluded 10% of the traffic, then
it would "save" having to receive 5-10 messages a day. Big deal.

I used the ExI software, and found it an interesting experiment, but I
can't say it save me any real effort. The effort of sending the
filtering message to the list site, hassling with the formats, etc.,
clearly outweighed the tiny effort it would have taken to manually
press "d" to delete the messages when then appeared.

I also found it useful to at least spend the 2-5 seconds to see what
was being talked about before pressing "D."

(A side issue: Whatever seconds were saved by the distribution-point
filtering (and I haven't mentioned the CPU time required...an issue
for us to consider with 700 list members) on the Extropians list were
often negated for the others by people asking "What are you talking
about?" or "Could someone send me Joe's posting on foo--I had him in
my ::exclude file.")

In other words, I find just being real fast on the "D" key is my best
way to cope with list volume.

Your mileage may vary, but I doubt that the Extropians-style software
is going to help much...I used it, and my experiences are what I just
described. 

Filtering is the wave of the future.

Paul Baclace, sometimes on this list, was working on filter agents for
Usenet that could learn preferences. And I've seen such things with
WAIS.

--Tim May


-- 
..........................................................................
Timothy C. May         | Crypto Anarchy: encryption, digital money,  
tcmay@netcom.com       | anonymous networks, digital pseudonyms, zero
408-688-5409           | knowledge, reputations, information markets, 
W.A.S.T.E.: Aptos, CA  | black markets, collapse of governments.
Higher Power: 2^859433 | Public Key: PGP and MailSafe available.
"National borders are just speed bumps on the information superhighway."