Date: Tuesday, 21 May 1985, 21:00-EDT
From: George J. Carrette <GJC at MIT-MC>
Subject: needs to run lisp
To: GJC-AT-LMI at MIT-MC
Message-ID: <[MIT-MC].513284.850521.GJC>


     Office of the Under Secretary of Defense (Research & Engineering)
     Forward Area Air Defense Joint Test Force
     P.O. Box 16030
     Attn:  Mr. Bruce Nickle
     Fort Bliss, Texas   79906


     (Phone: (915) 568-3539/1971)




Date: Wednesday, 22 May 1985, 14:29-EDT
From: George J. Carrette <GJC at MIT-MC>
To: GJC-AT-LMI at MIT-MC
Message-ID: <[MIT-MC].514214.850522.GJC>

Date: Tue, 21 May 85 22:00:21 edt
From: allegra!sola!kwc at mit-eddie
To:   allegra!mit-eddie!gjc at mit-mc.ARPA, jdg at mit-eddie,
      jones!myl at mit-eddie, kwc at mit-eddie, mitch at mit-eddie,
      sweet!lispm-wiz at mit-eddie

To: allegra!mit-eddie!gjc@mit-mc, myl, sweet!lispm-wiz, jdg, kwc, mitch
Subject: unix/lambda communication
--Text follows this line--

(GJC currently works for LMI; he used to work with Glenn Burke on
NIL at MIT.)

    From research!mit-vax!uucp@MIT-CCC.ARPA  Mon May 20 20:03:19 1985
    Received: by mit-vax.Mit-chaos.Arpa (4.12/4.8) with CHAOS id AA22123; Mon, 20 May 85 19:54:58 edt
    Received: from MIT-CCC by MIT-MC via Chaosnet; 20 MAY 85  19:56:37 EDT
    Date: 20 May 1985 19:56:33-EDT
    From: mit-vax!uucp@MIT-CCC.ARPA
    Apparently-To: research!sola!kwc
    
    >From gjc@cap Mon May 20 20:02:38 1985 remote from LMI-CAPRICORN
    Date: Monday, 20 May 1985, 20:02-EDT
    From: George Carrette <gjc at cap>
    Subject: the usual stream-of-concience ramblings of a hacker-too-much-today...
    To: mitccc!kwc%mit-mc at cap
    
    >From lisp:
    
    SI:*GLOBAL-SHARED-MEMORY-SIZE*
    SI:*GLOBAL-SHARED-MEMORY-8* ===> an ART-8B array.
    SI:*GLOBAL-SHARED-MEMORY-16* ===> an ART-16B array.
    SI:*GLOBAL-SHARED-MEMORY-32* ===> an ART-32B array.
    
    >From Unix:
    
    #include <share.h>
    
    main()
    { int *p;
      sharesetup();
      p = ((int) sharebase); /* *sharebase points to type char */
    
      ....}
    
    
    There are functions from lisp for writing floating-point numbers, handling
    byte-reversal (68010 is IBM-style, LISPM is DEC-style) of integers and such.
    
    
    This one of the lower-level interfaces of course, it is shared virtual memory
    and extremely fast.
    
    There is also (OPEN "UNIX-PORT-3:") from lisp <===> open("/dev/ttyl3",2) from unix.
    This is probably the easiest interface for most people to use.
    
    There is also open("/dev/nubus",2) from unix <==> (SI:%NUBUS-READ ...) from lisp.
    This is also rather low level and fast. It can be useful in combination with
    device drivers on the multibus.
    
    There is also CHAOSNET level functions, from unix one might say:
     open("/dev/chaos/10203/FOO",2); and from lisp  (CHAOS:OPEN-STREAM "CHAOS#10203" "FOO").
    
    The most popular servers in unix and lisp for getting applications started
    are the EVAL servers of course.

this all sounds very convenient.    
    
    How are you guys on realtime applications? I suspect that a PICON demonstation might
    tickle the fancy of applications-oriented people. PICON is now installed
    in an EXXON OIL refinery, and makes a great demo of user-interface.

i gather that some of us will see this if we go to IJCAI.  Yes, we're
interested in real time ...

    One thing I fear though is that you may not have the kind of hacker power to
    really take advantage of a machine like the LAMBDA. An environment with the
    combination of strong theoretical and reseach people combined with the kind
    of hacker power that made the AI lab what it was would be ideal. Strangely enough,
    only the most commercially oriented customers seem to want to deal with that kind of
    hacker power, with the more researchy places pretty much wanting to stick with
    the cozyness of the 3600 architecture, where your program will run by definition
    just as fast as the other researchers and there isnt much sense in trying to
    be more clever because you cant, its all symbolics proprietary.
    All you research places are *much* harder to convince about being a serious
    business than our business customers (such as multi-billion-dollar wall-street investment
    banking firms). These real business places do things like check up on our investors, many of
    whom they know by name, which is convincing for them.

yes we do lack hacker power.  it can often be harder to get a position
for a first rate hacker than for a first rate researcher.  if i want
something like a real time speech work station, i can often be faced
with the choice of (a) making do with what I have, (b) wait for someone
else to do it, (c) buy the best I can, or (d) do it myself.  I think you
can understand why (d) is often the least desirable choice, even if it
might lead to the best workstation.

    In the signal-processing area, besides doing LAMBDA microcode, there are
    good array processor products from DataCube (Peabody Mass) that have been
    succesfully hacked. (For various reasons the stuff from the west-coast
    silicoln valley places is sufficiently brain damaged as to make it a pain
    interface lispmachines too).
    
who has done what?

    I dont have much of an "IN" with the MIT signal processing people, but if you
    could get ahold of a code we could give it a try. But maybe it would be more
    impressive to run one of *your* programs faster.
    
here is one:

		;;; -*- Mode: LISP; Package: SYSTEM-INTERNALS; Base: 10 -*-
		
		;;; Lisp Machine version of J:/usr/myl/byte/errdiff.c
		;;; Halftoning by error diffusion
		;;;
		;;; TO USE:
		;;; (make-system 'headers)
		;;; (demo-error-diffusion)
		;;;
		;;; Alternatively,
		;;; (setq bytemap (load-data-file "j://s1//jbp//spectro//john3.byte"))
		;;; (show-bitmap (threshold-diffusion bytemap 128.))
		;;; (show-bitmap (error-diffusion bytemap))
		
		(defconst *max-value* 4096.)
		
		(defun demo-error-diffusion nil
		  (let ((bytemap (make-array '(256. 256.) :type 'art-8b)))
		    ;; Initialize the bytemap so that the values gradually
		    ;; move from extremely black in upper left corner
		    ;; to extremely white in lower right corner
		    (loop for i from 0 below 256. do
			  (loop for j from 0 below 256. do
				(aset (// (+ i j) 2.) bytemap i j)))
		    (send standard-output :clear-window)
		    (show-bitmap (threshold-diffusion bytemap 128.))
		    (show-bitmap (error-diffusion bytemap) 0 300.)))
		
		(defun threshold-diffusion (array threshold)
		  (loop with dims = (array-dimensions array)
			with result = (make-array (list (tv:word-align (car dims)) (cadr dims))
						  :type 'art-1b)
			for i from 0 below (first dims)
			do (loop for j from 0 below (second dims)
				 when (< (aref array i j) threshold)
				 do (setf (aref result i j) 1))
			finally (return result)))
		
		(defun show-bitmap (bitmap &optional (x 0) (y 0))
		  (send standard-output :bitblt tv:alu-ior
			(array-dimension-n 1 bitmap) (array-dimension-n 2 bitmap)
			bitmap 0 0 x y))
		
		;(defun make-16b-map (bytemap low-threshold high-threshold)
		;  (loop with dims = (array-dimensions bytemap)
		;	with conversion-table = (make-conversion-table low-threshold high-threshold)
		;	with result = (make-array dims :type 'art-q
		;				  ; 'art-16b - watchout for sign !!!
		;				  )
		;	for i from 0 below (first dims)
		;	do (loop for j from 0 below (second dims)
		;		 do (setf (aref result i j)
		;			  (aref conversion-table (aref bytemap i j))))
		;	finally (return result)))
		
		;;; make-16b-map modified for speed
		
		(defun make-16b-map (bytemap low-threshold high-threshold)
		  (let* ((bmap bytemap)
			 (dims (array-dimensions bmap))
			 (result (make-array dims :type 'art-q
					     ; 'art-16b - watchout for sign !!!
					     )))
		    (DECLARE (COMPILER:ARRAY-REGISTER-1D bmap result))
		    (loop with conversion-table = (make-conversion-table low-threshold high-threshold)
			  for idx below (apply #'* dims)
			  do (sys:%1d-aset (aref conversion-table (sys:%1d-aref bmap idx))
					   result idx)
			  finally (return result))))
		
		(defun make-conversion-table (low-threshold high-threshold)
		  (loop with result = (make-array 256. :type 'art-16b)
			for i from 0 below 256.
			do (cond ((< i low-threshold))
				 ((> i high-threshold) (setf (aref result i) *max-value*))
				 (t (setf (aref result i)
					  (// (* (- i low-threshold) *max-value*)
					      (- high-threshold low-threshold)))))
			finally (return result)))
		
		;(defun error-diffusion (bytemap &key (low-threshold 0) (high-threshold 220.))
		;  (loop with threshold = (// *max-value* 2)
		;	with map = (make-16b-map bytemap low-threshold high-threshold)
		;	with result = (make-array (list (tv:word-align (array-dimension-n 1 map))
		;				       (array-dimension-n 2 map))
		;				 :type 'art-1b)
		;	for i from 0 below (1- (array-dimension-n 1 map))
		;	do (loop for j from 2 below (1- (array-dimension-n 2 map))
		;		 for v = (aref map i j)
		;		 for excess = (if (> v threshold) (- v *max-value*) v)
		;		 do
		;		 (setf (aref result i j) (if (> v threshold) 0 1))
		;		 (incf (aref map i (1+ j))
		;		       (// (* 9. excess) 32.))
		;		 (incf (aref map (1+ i) (1+ j))
		;		       (// (* 6 excess) 32.))
		;		 (incf (aref map (1+ i) j)
		;		       (// (* 10. excess) 32.))
		;		 (incf (aref map (1+ i) (1- j))
		;		       (// (* 4 excess) 32.))
		;		 (incf (aref map (1+ i) (- j 2))
		;		       (// (* 3. excess) 32.)))
		;	finally (return result)))
		
		
		;;; error-diffusion modified for speed
		
		(defun error-diffusion (bytemap &key (low-threshold 0) (high-threshold 220.))
		  (let ((map (make-16b-map bytemap low-threshold high-threshold)))
		    (DECLARE (COMPILER:ARRAY-REGISTER-1D map))
		    (loop with (dim-1 dim-2) = (array-dimensions bytemap)
			  with threshold = (// *max-value* 2)
			  with result = (make-array (list (tv:word-align dim-1) dim-2)
						    :type 'art-1b)
			  for i from 0 below (1- dim-1)
			  do (loop for j from 2 below (1- dim-2)
				   for idx01 from (+ i (* 3 dim-1)) by dim-1
				   for idx1-2 from (1+ i) by dim-1
				   for v00 first (sys:%1d-aref map (- idx01 dim-1)) then v01
				   for v01 = (sys:%1d-aref map idx01)
				   for v1-2 first (sys:%1d-aref map idx1-2) then v1-1
				   for v1-1 first (sys:%1d-aref map (+ idx1-2 dim-1)) then v10
				   for v10 first (sys:%1d-aref map (+ idx1-2 (* 2 dim-1))) then v11
				   for v11 = (sys:%1d-aref map (1+ idx01))
				   for excess = (if (> v00 threshold) (- v00 *max-value*) v00)
				   do (and ( v00 threshold) (setf (aref result i j) 1))
				      (sys:%1d-aset (incf v01 (ash (* 9. excess) -5))
						    map idx01)
				      (sys:%1d-aset (+ v1-2 (ash (* 3 excess) -5))
						    map idx1-2)
				      (incf v1-1 (ash (* 4 excess) -5))
				      (incf v10 (ash (* 10. excess) -5))
				      (incf v11 (ash (* 6 excess) -5))
				   finally (progn (sys:%1d-aset v1-1 map (incf idx1-2 dim-1))
						  (sys:%1d-aset v10 map (+ idx1-2 dim-1))
						  (sys:%1d-aset v11 map (1+ idx01))))
			  finally (return result))))
		

    Ah, there is another thing that I perhaps didnt mention before.
    Gould has what seems to be to be a rather nice Unix machine for general
    and number crunching capabilities. They then built a NuBus board to interface
    our machine with the Gould SEL. A small SEL will run you about the same as
    a 750, and beat a 780 on your average fortran program. The Gould machine
    is popular at Nasa Goddard I'm told, used mostly for realtime control
    and signal processing. 

Unfortunately, we've had rather bad experience with SEL machines, and in
fact we just threw three of them away, because the operating system was
too difficult to deal with (I believe).  I don't think I could push for
them right now.
    
    Maybe you could pass some of the ideas around, see if there is a group in particular that
    could really use something like the Gould-Lambda combination or the multiprocessor
    capabilities. The situation at Bell might be similar to that at MCC.
    The researchy AI people wanted to stick with either their Xerox's (if they
    were broungt-up-on-that-sort-of-thing) or with the 3600, but the electrical-CAD
    people who needed to cruch real-sized projects and the Multiprocessor architecture
    people who needed a good simulation and development environment went with
    LMI.
    
Here, I believe it is the speech people who would push for cruch.  The
CAD folks also need crunch, but they go to the CRAY for that.  We might
also start doing that...

    If there are people presently using multibus hardware, build-their-own or
    customize something commercial, who are also possibly using 68010 Unix
    boards of some kind on the Multibus, they could benefit a great deal
    in performance capabilities by moving to a LAMBDA architecture.
    
    *** On the other hand there are some nicer systemic things about the LMI
	software, even for just pure lisp programmers.
	* Multiple ZMACS processes. You can be doing CONTROL-X CONTROL-F or
	  compile-file in one processes, and still edit in another.
	  Or have an ERROR-CONDITION being debugged in one ZMACS and be editing in
	  another. This you cant do in the symbolics software.

yes, this would be a big win.  I can't believe that symbolics hasn't
done it yet.

	* infinite UNDO capability in ZMACS.
	* better lexical-closure implementation using LAZY-STACKS.
    
    In fact the lexical-closure implementation is so good that my programming
    style has changed completely.

i must say that I still don't use lexical closures much.  perhaps I'm
old fashioned and will just have to wait until we get some crack young
guy to come out here and show me what i'm missing.  Can you give me
some examples where they are a big win.
    
    **** Other things: patents pending, must sign non-disclosure agreements.

we'll probably have to wait until you can talk ...

		       Most of these things have to do with, of course,
		       lisp in a multiprocessing large-address space (57 bits or more)
		       environment. Shades of multics. But remember, historically
		       Bell backed out of the multics project and went with Unix.

       ** Graphics: A 3d package implemented in Object-lisp which runs in
		    any common-lisp, and various devices such as regular LISPM screens,
		    Silicoln Graphics (3d-color realtime hidden-surface).

would this be available on the TI explorer?  we will probably have a few
of these around.  if your software really turns out to be better, we
might be more interested.
       
       ** We can also share-virtual-memory interface to the VAX-11/750
	  the device is from DataCube and has a window of 256 bytes.
	  Not impressive but usable.
    
I'd also be interested in this, but I suspect that we probably don't
want the first one.  I'll probably be at the IJCAI.  Perhaps we can
talk more then.

    -gjc
    
- kwc