Neural Nets and the Passage of Time
Thirty years ago, Diane and I were engaged in a madcap dash across the country, a 25-hour road-trip from our home in Fort Worth to the then-unknown environs of San Diego, California. Why would we do such a thing? It looks unlikely, even in the perspective of hindsight. The surface answer, and shallowest, is that we were headed there to attend the IEEE First International Conference on Neural Networks, held at the Sheraton on Harbor Island. Yes, it really has been thirty years since the ICNN.
We were, at the time, pretty newly minted graduate students at the University of Texas at Arlington, me in computer science, and Diane in biomedical engineering. For the summer of 1987, Diane had convinced me to join her in taking a mathematics course that she had run across. It took some convincing; I wasn’t terribly interested in working really hard for an elective, and usually I found electives out of fine arts more relaxing and rewarding. The instructor, Dan Levine, was a professor in the math department, but had research interests in psychology and behavior. Prof. Levine convinced me I belonged in the class. We were just a few weeks into the term when Prof. Levine suggested that everyone in the class attend a workshop put on by the region’s interest group on neural nets, the Metroplex Institute for Neural Dynamics (MIND).
MIND’s leadership had had a brainstorm. They knew that the IEEE was planning a huge conference on neural nets on the west coast. They also knew that many researchers heading west for the IEEE conference would have layovers in the Dallas area. They made it known that people could get in a preliminary presentation in Dallas simply by extending the layover they would have anyway for a day instead of a few hours. MIND would cover hotel stays, cab rides, and meals for the extension. MIND scored a coup with a lot of top talent taking them up on that and giving talks there just ahead of the ICNN.
One of the presenters at the MIND conference was Dr. Harold Szu. Szu did research into subjects like optical supercomputing and networks based on developmental neurobiology. He gave a darned interesting talk at the MIND conference, and Diane and I were able to talk to him some afterward. As I recall it, Szu’s talk was one of the last of the conference, and the organizers were rounding up presenters to carry them off to dinner. Prof. Levine came up to invite Dr. Szu to dinner, and I think the Southern reflex came into play then. Because we were standing right there, Prof. Levine invited us to join the group at dinner. So we did.
We ended up gabbing with people in the group, but especially Dr. Szu. And after dinner, we inquired if Dr. Szu was interested in more conversation, which he was. This had started as a Thursday evening, and continued into the small hours of Friday. Before we left, given that Diane had work to go to early in the morning, Dr. Szu brought up an idea he had: we should attend the ICNN. He offered to put us up in his hotel room and iron out any knotty logistical problems; he was, after all, one of the organizers of the ICNN. We’ve described this as Harold’s unreality field in years since. Things that don’t have a hope of happening otherwise somehow become possibilities under Harold Szu’s persuasion. But we initially demurred despite Harold’s enthusiasm for the project. He had explained that filling his hotel room with grad students would help him, because he had already promised a solitary female graduate student a place, and having a married couple there, too, would serve nicely for chaperonage.
So Diane headed off to work, groggily, on Friday morning, and things were looking like turning back into plain reality. Then Harold called me up and discussed the stuff I needed to be doing then in order to get to the ICNN. Which I started doing. I called Diane and passed messages along. We went from not doing anything differently from usual to frantically setting up a transcontinental drive. Part of the logistics included the glitch that the in the conference notices the conference hotel, the Sheraton, quite forcefully stated its “no pets” policy. At the time, we had a red-shouldered hawk, Quilli, who couldn’t be left on his own for a week, and there was no time to arrange boarding him with another falconer. So I called the Sheraton to talk about my situation. I ended up talking to the manager, who turned out to be about as enthusiastic at having Quilli come visit as Harold seemed to be to have us come out. I ended up packing myself, Quilli, and making a first pass at packing things for Diane. We contacted Diane’s cousin Johnny who lived near San Diego to see if we could crash there late Saturday and got an affirmative, though his wife Laurie was a bit suspicious of these relatives dropping out of the blue. Friday turned out to be a whirl of activity, which only settled down when we were actually loaded in our little Nissan Sentra hatchback and out on the road heading west.
It was our first sustained cross-country road trip. We got to El Paso about twelve hours after leaving Fort Worth, impressing upon us just how large Texas was. We swapped off driving, letting the other one of us nap while the miles went by. Johnny and Laurie were having a late dinner Saturday at the Spaghetti Factory, and we pulled up there and had a pretty surreal-seeming dinner with a good dash of fatigue poisoning. Johnny and Laurie tucked us in a bit later at their condo. And so we got up on Sunday and headed into San Diego proper and the ICNN.
Saturday had been tutorial day, a preface to the official Sunday start to the conference. So when we met up with Harold that Sunday, his first question was, “Where have you been?” We explained that the drive was 25 hours and we couldn’t have made it any sooner. Harold was flummoxed. “I did not know, I would not have asked you to come!” But given that we were there, he fully delivered on his promises of hospitality. We set ourselves and Quilli up in his room with the other grad student. And we started attending various lectures, picking likely-looking things from the conference program. When we could, we tagged along with Harold, which was an interesting experience. It turns out that if you are plastered to a conference organizer, people assume you know something about the topic.
The ICNN was a hopeful venture of a bunch of smart people looking to make a renaissance out of a technology that had been declared dead once before. The field was just then beginning to recover from the rhetorical wounding that Minsky and Papert’s “Perceptrons” had done it, with things like the Hopfield and Tank network getting prominent exposure in “Science”, and the back-propagation training method showing exactly how one could set up a neural network to learn “interesting” problems, where “interesting” meant “non-linear”. Even then, there was incredible diversity in approaches. The PDP group led by Rumelhart and McClelland were well represented, Stephen Grossberg and Gail Carpenter were there, Bart Kosko, Fukushima, and loads of other researchers were in attendance. There were people trying to make biologically relevant models, and folks like Robert Hecht-Nielsen, who would be much more comfortable once neural networks could be described entirely in the terminology of spin glasses and leave the confusing and irrelevant biology behind. There were vendors with special-purpose hardware for parallel floating-point calculations, and others with software; I think NeuralWare had an early version there. The plenary talk I remember most vividly was the one by Bernard Widrow, who recounted the development of ADALINE and MADALINE, and the LMS algorithm (with M.E. Hoff, he of microprocessor development fame) that was their foundation. Widrow did a great job, I thought, of putting a personable face on this technology, and noting that while some (Minsky and Papert) might deem it not “interesting” to solve certain problems, the LMS algorithm was ineluctably a part of signal processing for every modem communicating at 2400 baud or better and thus doing good work anyway.
I don’t know what path my life would have followed without that mad dash across the country and the immersion in neural networks, technology and society, that happened that week. I doubt I would have had the level of interest in the technology without that, and that played a role in my later employment and studies. I had already gravitated to the AI track in my master’s work, but I shifted to doing a neural network thesis. I set up a bulletin board system (BBS) in 1989, and set it up as the “Central Neural System BBS”, with a large collection of programs and source code for neural networks, genetic algorithms, and artificial intelligence. (That body of file archives was later given a home by Carnegie Mellon University. It’s actually still there: https://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/areas/neural/cns/0.html ) While neural nets were definitely not a technology flavor appreciated when I worked for General Dynamics Data Systems Division, my drive to set up a special interest group on the topic was appreciated at Pacific Northwest Laboratory, which went on for several years after my departure to grad school again. Having neural networks on our resumes helped get us recruited at Texas A&M in the marine mammal program there, since Bill Evans was a forward-thinking kind of guy. The interdisciplinary nature of it helped in working through animal behavior classes (the only essay question I got full marks on from Jane Packard happened to have overlap with my master’s thesis). Thirty years on I still have a debt of gratitude to Prof. Levine, who took on effective mentorship for my master’s work and served on my Ph.D. committee, and for Dr. Harold Szu, who drew Diane and I into a closer acquaintance with the technology and the personalities of the field.
