From: Piete Brooks <Piete.Brooks@cl.cam.ac.uk>
Date: Mon, 28 Aug 95 11:16:33 PDT
To: patrick@Verity.COM (Patrick Horgan)
Subject: Re: SSL trouble
In-Reply-To: <9508281635.AA19987@cantina.verity.com>
Message-ID: <"swan.cl.cam.:216660:950828181616"@cl.cam.ac.uk>
MIME-Version: 1.0
Content-Type: text/plain


> I did a distributed scheme for something else that had two levels, a master
> and a group of slaves.  Only the slaves talked to the master.  For this 
> effort I think a variation of the idea would be better.  Have all of the
> brutes contact the master, who will, in the first transaction assign them
> to the next slave in a round-robin fashion.

Well, imagine my suprise  ....  [[ reference to ISIRTA ]]

One of the things that the latest brloop does is make a call to the master
server asking for a list of servers to contact :-))

Note that it is a list, and it tries them in order (all A RRs).

> Then all of the transactions from that point would take place between the
> brute and the slave:)

Currently just all the "allocate" transactions -- I haven't written my
ACK reflector yet, so all ACKs go direct th the ACK master.

> The slaves would each be delegated large chunks of the keyspace,

No -- the slaves will not "be delegated" (as in pre-assigned address space),
they will just ask the master for it as they need it.
Sure, the'll do it in reasonable sized chunks, but not (2**16)/16 ....

> but not keyspace/numslaves.  Maybe 1/16th or something like that, and could
> ask for more when their space was depleted.  Periodically, perhaps when
> requesting more key space, and/or when a timer pops, the slaves could report
> results.

Nah - results still go direct pro tem.

> What I mean is that every so often they'd report even if they didn't need
> more keyspace yet, iff they had any new stats to report.

Sure.

> The nice thing here is that the work of the master and of the slaves is
> almost the same.i

You got it !

> The slaves don't have to do the initial assignment of slave,
(slave -> slaves I assume)

> and the master doesn't have to report results, but everything else
> is the same.

Yup -- code sharing !

> With careful design you could use the same daemon for both
> with a command line argument to tell it if it was the master (-m) or the
> slave (-s).

Well, not even that !

The slaves don't have the config file with the key info in it ...

> Of course I'm sure you see that this allows you to add as
> many levels as you want to the hierarchy.

Indeed.

BUT ....

These cache servers are asking for non trivial amounts of keyspace.
As such there should not be *too* many, and then need to be "managed" ...
If one crashes, the logs need to be scanned to see how to restart it (so that
it starts by doling out the segments that it had no sub-doled to its clients).

> A slave doesn't care whether a slave or a brute talks to it.

Indeed -- that's how it was designed ...

However, note that with big cache servers (as opposed to Local CPU Farm servers
where all clients are the same "ID") reports of sub-allocation have to be
passed back to the root :-(

> You could make the slave software available as well, and a site with many
> machines could have only the slave contact the master to get assigned a
> slave to talk to, and could configure all of their brutes to talk to
> their own slave.

Indeed -- the Local CPU Farm cache server is just about ready for ALPHA testers

> Software like this is easy to write, (and fun), and we should go for it:)

Done ...

> Of course I do everything like this in C++, but I suppose perl would be the
> most portable.  It's a shame it's so aethestically displeasing to the eye.

Yeah -- but being based on C, C++ didn't stand much chance ...

> perl's never a pleasant read.

... but better than C++ -- sure.


PS1: PERL gurus: Anyone know how to test whether there is input waiting on a
     file handle ? I know about seeing if there is data waiting for the next
     sysread type read, but not on the next <SERVE> type read. Ideas ?

PS2: PERL gurus: I fixed the SGI Challenge problem by HACKing it -- as I
     thought it was a probleb with stdio in and out on the same socket.
     The perl mand page warns:
	If  your  stdio  requires an seek or eof between reads and
	writes on  a  particular  stream,  so  does  perl.   (This
	doesn't apply to sysread() and syswrite().)
     so I change the one "print SERVE" line to a "syswrite(SERVE" and that
     fixed it. However, does anyone know the "correct" way to use stdio for I/O?

PS3: I'd like to get the raw date in brloop (a sh script). In perl I'd just
     use "time", and I can't see a way to get "date +" to yield the raw time.
     I could use "date=`perl -e 'print time'`" but that seems OTT, and perl
     may not be on teh users PATH. Any suggestions ?