From: C Matthew Curtin <cmcurtin@research.megasoft.com>
Date: Tue, 17 Sep 1996 04:54:26 +0800
To: cypherpunks@toad.com
Subject: Snake Oil FAQ 0.4 [comments appreciated]
Message-ID: <199609161356.JAA09796@goffette.research.megasoft.com>
MIME-Version: 1.0
Content-Type: text/plain



The Snake Oil FAQ version 0.4 is waiting for you to look at it and
comment on
http://research.megasoft.com/people/cmcurtin/snake-oil/snake-oil-faq.html

The text version follows...

-matt

--------------------------- snake-oil-faq ----------------------------

                           Snake-Oil Warning Signs
                        Encryption Software to Avoid
      $Id: snake-oil-faq.html,v 0.4 1996/09/16 13:52:26 cmcurtin Exp $
Distribution

Please do not distribute this beyond the circles of cryptographic competence
yet. This is an incomplete work-in-progress. Feedback is greatly
appreciated.

The Snake Oil FAQ is (to be) posted monthly to cypherpunks, sci.crypt,
alt.security, comp.security, and comp.infosystems. We're targeting those who
have influence over or direct involvement in the purchasing decisions of
computer security software and equipment in the corporate and academic
worlds, as well as individual users who wish to assert their privacy through
the use of good cryptography.

Disclaimer

All contributors' employers will no doubt disown any statements herein.
We're not speaking for anyone but ourselves, based on our own experiences,
etc., etc., etc. This is a general guideline, and as such, cannot be the
sole metric by which a security product is rated, since there can be
exceptions to any of these rules. (But if you're looking at something that
sounds familiar on several of the 'things to watch out for,' you're probably
dealing with snake oil. From time to time, a reputable and decent vendor
will produce something that is actually quite good, but will use some
braindead marketing technique, so be aware of exceptions.)

Every effort has been made to produce an accurate and useful document, but
the information contained herein is completely without warranty. If you find
any errors, or wish to otherwise contribute, please contact the document
keeper, C Matthew Curtin <cmcurtin@research.megasoft.com>

Introduction

Good cryptography is an excellent and necessary tool for almost anyone.
However, there is a multitude of products around. Many good cryptographic
products are available, both commercial (including shareware) and free.
However, there are also some extremely bad cryptographic products (known in
the field as "Snake Oil"), which not only fail do their job of providing
security, but are based on, and add to, the many misconceptions and
misunderstandings surrounding cryptography and security.

Superficially, it is difficult for someone to distinguish the output of a
secure encryption utility from snake oil: both look garbled. The purpose of
this document is present some obvious "red flags" so that people unfamiliar
with the nuts and bolts of cryptography can use as a guideline for
determining whether they're dealing with snake oil or the Real Thing.

For a variety of reasons, this document is general in scope and does not
mention specific products or algorithms as being "good" or "Snake Oil".

When evaluating any product, be sure to understand what your needs are. For
data security products, what do you need protected? Do you want an archiver
that supports strong encryption? An E-mail client? Something that will
encrypt on-line communications? Do you want to encrypt an entire disk or
partition, or selectively some files? Do you need on-the-fly (automatic)
encryption and decryption, or are you willing to select when and which files
you want encrypted? How secure is "secure enough?" Does the data need to be
unreadable by third parties for 5 minutes? One year? 50 years? 100 years?

Different products will serve different needs, and it's rare that a product
will serve every need. (Sometimes a product won't be needed: it may be
better to use a utility to encrypt files, transmit them over a network using
standard file transfer tools, and decrypt them at the other end than to use
a separate encrypted utility in some cases.)

Some basics

The cryptography-faq (found at
ftp://rtfm.mit.edu/pub/usenet/cryptography-faq/) is a more general tutorial
of cryptography, and should also be consulted. In an effort to make this FAQ
more complete, some very basic topics are included below.

Conventional vs. Public Key Cryptography

There are two basic types of cryptosystems: symmetric (also known as
"conventional," sometimes also called "private key") and asymmetric (public
key). Symmetric ciphers require both the sender and the recipient to have
the same key. That key is applied to encrypt the data by the sender, and
again by the recipient to decrypt the data. Asymmetric ciphers are much more
flexible, from a key management perspective. Each user has a pair of keys: a
public key and a private key. The public key is shared widely, given to
everyone, while the private key is kept secret. If Alice wishes to mail Bob
some secrets, she simply gets Bob's public key, encrypts her message with
it, and sends it off to Bob. When Bob gets the message, he uses is private
key to decrypt the message.

Asymmetric cryptosystems are much slower than their symmetric counterparts.
Also, key sizes must be much larger. See the cryptography FAQ for a more
detailed discussion of the topic.

Key Sizes

Some ciphers, while currently secure against most attacks, are not
considered viable in the next few years because of relatively small keysizes
and increasing processor speeds (making a brute-force attacks feasible). The
tables below should give some general guidelines for making intelligent
decisions about the key length you need. If the key is too short, the system
will be easily broken, even if the cipher is a good one.

In [1] and [2], we're presented with some guidelines for deciding
appropriate key length. (It is important to note that this is based on the
ability to predict computing power 40, 65, and 100 years from now. Major
breakthroughs in computing power 30 years from now might render everything
on this chart kiddieplay.)

               Security Requirements for Different Information

              Type of Traffic                Lifetime   Minimum [Symmetric]
                                                             Key Length
 Tactical military information             minutes/hours     56-64 bits
 Product announcements, mergers, interest
 rates                                      days/weeks        64 bits
 Long-term business plans                      years          64 bits
 Trade secrets (e.g., recipe for
 Coca-Cola)                                   decades         112 bits
 H-bomb secrets                              >40 years        128 bits
 Identities of spies                         >50 years        128 bits
 Personal affairs                            >50 years        128 bits
 Diplomatic embarrassments                   >65 years   at least 128 bits
 U.S. Census data                            100 years   at least 128 bits

As mentioned earlier, asymmetric ciphers require significantly longer keys
to provide the same level of security as their symmetric cipher
counterparts. Here is a comparison table, again, from Applied Cryptography,
second edition.
                    Symmetric and Public-Key Lengths With
                  Similar Resistance to Brute-Force Attacks

                 Symmetric Key Length Public-key Key Length
                        56 bits             384 bits
                        64 bits             512 bits
                        80 bits             768 bits
                       112 bits             1792 bits
                       128 bits             2304 bits

Some Common Snake-Oil Warning Signs

The following are some of the "red flags" one should watch for when
examining an encryption product

   * Technobabble

     The vendor's description of the product may contain a lot of
     hard-to-follow use of technical terms to describe how the product
     works. If this appears to be confusing nonsense, it may very well be
     (even to someone familiar with the terminology). Technobabble is a good
     means of confusing a potential user and masking the fact that the
     vendor doesn't understand anything either.

     A sign of technobabble is a descrption which drops a lot of technical
     terms for how the system works without actually explaining how it
     works. Often specifically coined terms are used to describe the scheme
     which are not found in the literature.

   * New Type of Cryptography?

     Beware of any vendor who claims to have invented a "new type of
     cryptography" or a "revolutionary breakthrough". Truly "new
     break-throughs" are likely to show up in the literature, and many in
     the field are unlikely to trust them until after years of analysis, by
     which time they are not so new anymore.

     Avoid software which claims to use 'new paradigms' of computing such as
     cellular automata, neural nets, genetic algorithms, chaos theory, etc.
     Just because software uses to different method of computation doesn't
     make it more secure.

     Anything that claims to have invented a new public key cryptosystem
     without publishing the details or underlying mathematical principles is
     highly suspect. Modern cryptography, especially public key systems, is
     grounded in mathematical theory. The security is based on problems that
     are believed, if not known to be hard to solve.

     The strength of any encryption scheme is only proven by the test of
     time. New crypto is like new pharmaceuticals, not new cars.

   * Proprietary Algorithms

     Avoid software which uses "proprietary" or "secret" algorithms.
     Security through obscurity is not considered a safe means of protecting
     your data. If the vendor does not feel confident that the method used
     can withstand years of scrutiny by the academic community, then you
     should be wary of trusting it. (Note that a vendor who specializes in
     the cryptography may have a proprietary algorithm which they'll show to
     others if they sign a non-disclosure agreement. If the vendor is
     well-reputed in the field, this can be an exception.)

     Beware of specially modified versions of well-known algorithms. This
     may intentionally or unintentionally weaken the cipher.

     The use of a trusted algorithm, if not with technical notes explaining
     the implementation (if not availability of the source code for the
     product) are a sign of good faith on the part of the vendor that you
     can take apart and test the implementation yourself.

     A common excuse for not disclosing how a program works is that "hackers
     might try to crack the program's security." While this may be a valid
     concern, it should be noted that such 'hackers' can reverse engineer
     the program to see how it works anyway. If the program is implemented
     properly and the algorithm is secure, this is not a problem. (If a
     hypothetical 'hacker' was able to get access you your system, access to
     encrypted data might be the least of your problems.)

   * Experienced Security Experts and Rave Reviews

     Beware of any product claiming that "experienced security experts" have
     analyzed it, but it won't say who (especially if the scheme has not
     been published in a reputable journal).

     Don't rely on reviews in newspapers, magazines or television shows,
     since they generally don't have cryptologists (celebrity hackers who
     know about telephone systems don't count) take the software apart for
     them.

     Just because the vendor is a well known company or the algorithm is
     patented doesn't make it secure either.

   * Unbreakability

     Some vendors will claim their software is "unbreakable". This is
     marketing hype, and a common sign of snake-oil. Avoid any vendor that
     makes unrealistic claims.

     No algorithm is unbreakable. Even the best algorithms are breakable
     using "brute force" (trying every possible key), but if the key size is
     large enough, this is impractical even with vast amounts of computing
     power.

     One-time pads are unbreakable, but they must be implemented perfectly,
     which is, at best, very difficult. See the next section for a more
     detailed discussion.

   * One-Time-Pads

     A vendor might claim the system uses a one-time-pad (OTP), which is
     theoretically unbreakable. That is, snake-oil sellers will try to
     capitalize on the known strength of a OTP. It is important to
     understand that any variation in the implementation means that it is
     not an OTP, and has nowhere near the security of an OTP.

     A OTP system is not an algorithm. It works by having a "pad" of random
     bits in the possession of both the sender and recipient. The message is
     encrypted using the next n bits in the pad as they key, where n is the
     number of bits in the message. After the bits are used from the pad,
     they're destroyed, and can never again be used. The bits in the pad
     must be truly random, generated using a real random source, such as
     specialized hardware, radioactive decay timings, etc., and not from an
     algorithm or cipher. Anything else is not a one-time-pad.

     The vendor may confuse random session keys or initialization vectors
     with OTPs.

   * Algorithm or product XXX is insecure

     Be wary of anything that makes claims that particular algorithms or
     other products are insecure without backing up those claims (or at
     least citing references to them).

     Sometimes attacks are theoretical or impractical (requiring special
     circumstances or massive computing power running for many years), and
     it's easy to confuse a layman by mentioning these.

   * Keys and Passwords

     The "key" and the "password" are often not the same thing. The "key"
     generally refers to the actual data used by the cipher, while the
     "password" refers to the word or phrase the user types in, which the
     software converts into the key (usually through a process called
     "hashing" or "key initialization").

     The reason this is done is because the characters a user is likely to
     type in do not cover the full range of possible characters. (Such keys
     would be more redundant and easier for an attacker to guess.) By
     hashing a key can be made from an arbitrary password that covers the
     full range of possible keys. It also allows one to use longer words, or
     phrases and whole sentences as a "passphrase", which is more secure.

     Anything that restricts users' passwords to something like 10 or 16 or
     even 32 characters is foolish. If the actual "password" is the cipher's
     key (rather than hashing it into a key, as explained above), avoid it.

     If the vendor confuses the distinctions between bits, bytes and
     characters when discussing the key, avoid this product.

     Convenience is nice, but be wary of anything that sounds too easy to
     use. Avoid anything that lets anyone with your copy of the software to
     access files, data, etc. without having to use some sort of key or
     passphrase.

     Avoid anything that doesn't let you generate your own keys (ie, the
     vendor sends you a key in the mail, or it's embedded in the copy of the
     software you buy).

     Avoid anything by a vendor who does not seem to understand the
     difference between public-key (asymmetric) cryptography and private-key
     (symmetric) cryptography.

   * Lost keys and passwords

     If there's a third-party utility that can crack the software, avoid it.

     If the vendor claims it can recover lost passwords (without using a
     key-backup or escrow feature), avoid it.

     If there is a key-backup or escrow feature, are you in control of the
     backup, or does the vendor or someone else hold a copy of the key?

   * Exportable from the USA

     If the software is made in North America, can it be exported? If the
     answer is yes, chances are it's not very strong. Strong cryptography is
     considered munitions in terms of export from the United States, and
     requires approval from the State Department. Chances are if the
     software is exportable, the algorithm is weak or it is crackable (hence
     it was approved for export).

     If the vendor is unaware of export restrictions, avoid the software:
     the vendor is not familiar with the state of the art.

     Because of export restrictions, some legitimate (not-Snake Oil)
     products may have a freely exportable version for outside of the USA,
     which is different from a separate US/Canada-only distribution. Also
     note that just because software has made it outside of North America
     does not mean that it is exportable: sometimes a utility will be
     illegally exported and posted on an overseas site.

Other Considerations

Interface isn't everything: user-friendliness is an important factor, but if
the product isn't secure then you're better off with something that is
secure (if not as easy to use).

No product is secure if it's not used properly. You can be the weakest link
in the chain if you use a product carelessly. Do not trust any product to be
foolproof, and be wary any product that claims it is.

Contributors

The following folks have contributed to this FAQ.

Jeremey Barrett <jeremey@forequest.com>
<geeman@best.com>
Jim Ray <liberty@gate.net>
Robert Rothenburg Walking-Owl <wlkngowl@unix.asb.com>

References

  1. B. Schneier, Applied Cryptography, second edition, John Wiley & Sons,
     1996
  2. M. Blaze, W. Diffie, R. L. Rivest, B. Schneier, T. Shimomura, E.
     Thompson, M. Wiener, "Minimal Key Lengths for Symmetric Ciphers to
     Provide Adequate Commercial Security," available via
     ftp://ftp.research.att.com/dist/mab/keylength.ps

----------------------------------------------------------------------------
C Matthew Curtin
Last modified: Mon Sep 16 09:51:41 EDT
----------------------------------------------------------------------

-- 
C Matthew Curtin                MEGASOFT, INC                Chief Scientist
I speak only for myself.  Don't whine to anyone but me about anything I say.
Hacker Security Firewall Crypto PGP Privacy Unix Perl Java Internet Intranet
cmcurtin@research.megasoft.com http://research.megasoft.com/people/cmcurtin/