From: "William H. Geiger III" <whgiii@invweb.net>
Date: Sun, 23 Nov 1997 17:05:13 +0800
To: Tim May <tcmay@got.net>
Subject: Re: Survey: Police Satisfaction [CNN]
In-Reply-To: <v03102803b09d75ffa64e@[207.167.93.63]>
Message-ID: <199711230859.DAA26028@users.invweb.net>
MIME-Version: 1.0
Content-Type: text/plain



-----BEGIN PGP SIGNED MESSAGE-----

In <v03102803b09d75ffa64e@[207.167.93.63]>, on 11/23/97 
   at 01:02 AM, Tim May <tcmay@got.net> said:

>At 8:11 PM -0700 11/22/97, William H. Geiger III wrote:

>>In <199711230158.TAA04577@einstein.ssz.com>, on 11/22/97
>>   at 08:58 PM, Jim Choate <ravage@ssz.com> said:
>>
>>>>      The study was based on a survey of 6,421 people, age 12 and older,
>>>>      and used a sample of residents chosen to represent an entire
>>>>      population. No margin of error was given.
>>
>>6,000 people and that represents the whole country of +250 Million.
>>
>>Ahhh Statistics the Mathematics of Lies.
>>

>The mathematics of sampling is well known, and is not the main source of
>"lies." The law of large numbers, tendency to the mean, etc., are the
>usual terms.

>It's perfectly plausible, and common, to use samples of a few thousand to
>get parameters (one of the few times this word gets used correctly) of a
>population of millions, or even billions.

It is common but it is bad science. As in this study there are too many
factors that would affect the results of the survey for such a small
sample size to be accurate. One of the biggest problems is the different
results that will be acquired by taking samples for different groups. A
sample group from say the projects in Chicago will give much different
results than a sample group taken from say Lake Forest. Now multiply this
"regional" effect several times for each group that will have a
predisposition one way or another (pro or anti police). For a survey to
have an accurate representation it needs to identify and sample from each
of these groups to give an accurate overall picture.

Lets say that you have decided on a sample size of 6000 and have
identified 100 different "reginal" groups that you need to sample from.
Your sample size is now down to 60/group and this is only on one factor of
many that can cause your figures to be bias. Lets now say that you have
discovered 10 key factors per group that need to be taken into account.
Your sample size is now down to 6/group/factor. Not a very comforting
sample size for developing an accurate "global" analysis.

Now lets take into account things like how the interviewer presents
himself to the subject being questioned. A officer wearing a uniform will
get a different response than say someone in a teeshirt and bluejeans.

Then you have the actual setting that the interview is taking place. Ask
someone at the local bar what they think about the police and you will get
one answer ask them at their office with their boss in the room and you
will get a different set of responses.

The single fact that there is another person there giveing the interview
rather then say haveing the subject fill out a questionnaire privately
will bias the results. 

Of course having mail in questionnaire has it's own set of problems as
only a small fraction of the people will respond and the people that do
respond may bias your results because of the reasons that they have for
responding (You may have a disproportionate number of responses from
people who feel passionately on the subject one way or another).

Now that we have gotten through all of that we still have the survey
itself.

First you have to determine what questions to ask to find the answers that
you are looking for. Not as easy as this may first seem (not to mention
the issue of wether you are looking for the right answers).

Then you have to worry about how the questions are phrased.

Now you have to study what effects these will have on the various
group/factors above. While one set of questions, phraseology, environment,
and interviewer may result in an accurate result for one group a
completely different set may be needed for another group.

>Bigger sources of lies are, of course, how questions are phrased.

How the questions are phrased are only one of a large number of factors
that influence the results of such a survey. Statistical Sampling is
difficult enough in the real sciences (chem, bio, phy, ...) let alone the
pseudo-sciences like psychology and sociology.

Any such survey as the one here to be able to show an accurate picture
could happen only by chance. More than likely they could have obtained
more accurate results by sitting in a back room flipping a coin.

- -- 
- ---------------------------------------------------------------
William H. Geiger III  http://users.invweb.net/~whgiii
Geiger Consulting    Cooking With Warp 4.0

Author of E-Secure - PGP Front End for MR/2 Ice
PGP & MR/2 the only way for secure e-mail.
OS/2 PGP 2.6.3a at: http://users.invweb.net/~whgiii/pgpmr2.html                        
- ---------------------------------------------------------------

-----BEGIN PGP SIGNATURE-----
Version: 2.6.3a
Charset: cp850
Comment: Registered_User_E-Secure_v1.1b1_ES000000

iQCVAwUBNHfwOY9Co1n+aLhhAQGOMwP+Mljo1dvhfObhvhAYbZuDHzLvRXVGkHDE
CfnZyFPRcHdVCIZy4c2LBQqIPyOyix1bfwPZ9VtlVepCxlglUGCvti81if59FibR
ivK9a+7yX5xq0weKlr36erMDlvJB80MVO6fggJlQtzSubQJQ2l1L0WtbKb7Eta2b
XUjk38Ou3nI=
=XBuY
-----END PGP SIGNATURE-----