From: "Ian Farquhar" <ianf@simple.sydney.sgi.com>
Date: Sun, 21 Aug 94 16:42:27 PDT
To: tcmay@netcom.com (Timothy C. May)
Subject: Re: NSA spy machine
In-Reply-To: <199408190637.XAA16716@netcom9.netcom.com>
Message-ID: <9408220938.ZM4720@simple.sydney.sgi.com>
MIME-Version: 1.0
Content-Type: text/plain


On Aug 18, 11:37pm, Timothy C. May wrote:
> > Secret: take lots and lots and lots of money, use the most exotic packaging
> > technologies you can find, pay lots and lots of attention to your memory
> > system and cache, don't forget the importance of a nicely balanced
> > architecture (meaning that I/O does matter), don't forget the importance of
> > good compilers, and implement bit counting instructions just like the NSA
> > tells you to.

> > Hardly a secret, don't you think?

> With no disrepect meant to Ian (indeed, this is my second reply of the
> day to him), I think his point is dead wrong.

No, it's not.  See below.

> The "secret" to general success in this market is not "lots and lots
> of money," at least not when "lots and lots" is the tens of millions
> of dollars that Cray Computer will apparently being getting from NSA
> and the Supercomputer folks in Bowie for the delivery of an ostensible
> Cray 4 or whatever it ends up being called (SMPP, etc.).
>
> $10 million is pocket change. Anyone building a company on that chump
> change is already preparing Chapter 11 papers.

I agree, but I was not talking about that amount of money.  My point possibly
would have been better stated as being "don't worry about the price".

> Here's what "lots and lots of money" *really* is:
> - $1 billion to complete a wafer fab in Ireland, finished last February
> - $1.3 billion to build a wafer fab in Albuquerque, to be finished
> later this year

[...]

You're talking about silicon fab lines here, Tim.  As far as I am aware, Cray
has never fab it's own chips.  Indeed, most of their boards which I have
seen (I, II, Y-MP/8 and Y-MP/EL) have used chips sourced from fairly well-
known vendors, such as VLSI Technologies.

The original series of systems (I, II and X-MP) actually used huge numbers
of three types of chips.  From memory, one was a couple of NAND gates, one
was a register chip, and the third was a couple of K of SRAM.  More recently,
the full Y-MP's have been implemented in commercial ECL gate arrays (6500
gates per chip for the full Y-MP's), and the original EL used CMOS 100K
arrays.  I have been trying to get one of the computing industries choicest
pieces of marketting junk: the Cray Y-MP Gate Array paperweight. :)  Of
course, given my recent career change, I don't think one will be coming
my way anytime soon for some strange reason....

Comparing Cray with Intel is rather specious, because the companies are
entirely different beasts.  Intel's supercomputing division is a tack-on
to it's high-end chip line.  Cray never has had a division even vaguely
like the Intel CPU divisions.

> (And a new generation of hackers are using Linux on cheap Pentium
> boxes to easily outrun Suns.)

Not a fair comparison, really.  Sun is the bottom end of the RISC system
market, and is being continually trounced by almost everyone else.  Comparing
the Pentium to our R4400 chips, or HP's PA, or DEC's Alpha would be much more
instructive, and not nearly as favorable to Pentium.

No, I am not knocking Pentium.  Within it's design limitations it's an
interesting accomplishment.  But those design limitations are crippling.

> So you see why I consider the "lots and lots of money" flowing into
> Cray Computer to be spitting into the ocean. I'm not worried.

Agreed.  But the points I was making concerned a comment from the previous
posters about Seymour Cray's design "secrets", NOT the current grant of money
from our good friends at the Puzzle Palace.  Ok, let me explain what I wrote:

> > Secret: take lots and lots and lots of money,

Cray's traditional client-base is money rich, and possess problem sets which
are not practical on conventional architectures.  Those conventional
architectures exist within a cost/afforability framework which limits the
technologies that they can use.  Cray is not unlimited, but it is not nearly
as limited.  They've also got a hidden advantage in that if they do make
a _really_ bad business decision (and I'd say personally that the Cray-III had
been one such), then their customers will probably support them just to
maintain
their current systems.  It's a nice position to be in.

So it's not that Cray has lots and lots of money, but that you can assume that
your customer base will have.

> > use the most exotic packaging
> > technologies you can find,

Lots of people disregard the implications of putting quarter of a million ECL
chips into a column a metre round and a metre and a half high (ie. the Cray
II).
You have BIG heat problems, and in some configurations even flourinert
immersion isn't going to work.

A lot of the cost of these systems is packaging, and Cray really pushes
the state of the art here.  In Seymour Cray's speech to the ACM, he mentions
that fluid immersion of PCB's had never been tried before, as everyone thought
that the boards would bloat.  But they tried it, and the boards were fine,
with the added bonus that because the heat was much more evenly distributed,
the machines were considerably more reliable than expected.

> > pay lots and lots of attention to your memory
> > system and cache,

(BTW, for those people who want to lecture me that Cray's don't have caches,
just consider the different terminology.  Vector registers are nothing more
than user directed data caches, and Crays call their instruction cache
an "instruction buffer".)

As many RISC system manufactures have found, you can put the fastest CPU's
into machines, but without a damn good memory design they spent most of their
time waiting.

Cray's CPU's are not particularly complex.  Cray estimates 1.5 million
transistors to implement a Y-MP CPU.  Most modern RISC CPU's are considerably
more complex than this.

It's the Cray memory system, which on most of the traditional vector machines
is implemented in 10-15nS SRAM with four ports to memory from EACH CPU, that
is the spectacular part of the design.  What Cray uses for main memory (M90
and EL series excepted), most other vendors use for cache.

(Of course, I have to say that SGI's Power Challenge memory systems are getting
pretty impressive too, now.  You can't avoid it if you're supporting the
sorts of performance our newer supercomputer-class systems provide.)

> > don't forget the importance of a nicely balanced
> > architecture (meaning that I/O does matter),

Lots of the Japanese supercomputing vendors forgot this.  Their peak MFLOP
performance was really spectacular, but with real applications they looked
a lot less spectacular.  The data set size of most applications which are
worth running on supercomputers is HUGE, but it's useless if you cannot get
the data too and from disk in less time than it takes to process it.

> > don't forget the importance of
> > good compilers,

Cray has been known to ship systems without even an operating system, it's
true, but only very early on.  Unless you ship _good_ compilers, most of the
applications for the machine won't get written.  Sure, you can program in
CAL, but most people won't.

> > and implement bit counting instructions just like the NSA
> > tells you to.

Ok, I was being facetious here.  But it masks a good point: customer service
is important, and companies with tiny installed bases (eg. the Cray II sold
a total of 31 systems), need to look after their customers.  Lots of other
supercomputer vendors who rolled nice boxes out of the door and then just
went into a backroom to design the next without any customer involvement
don't exist anymore.

							Ian.

Disclaimer: I am NOT speaking for SGI.