Undergraduate Research at Johns Hopkins University
I got my first real exposure to research by writing FORTRAN programs for my undergraduate mentor, Prof. Mandell Bellmore. My initial project involved implementing a subtour elimination algorithm for the symmetric traveling salesman's problem, developed by Prof. Bellmore's Ph.D. student, John Malone. The results were published in their 1971 Operations Research paper, Pathology of Traveling-Salesman Subtour-Elimination Algorithms. Subsequently, I implemented an algorithm developed by Prof. Bellmore's Ph.D. student, H. D. Ratliff, that was reported in their 1971 Management Science paper, Optimal Defense of Multi-Commodity Networks.
Ph.D. Research at Stanford University
After graduating from Johns Hopkins in 1970, I was a Ph.D. student in Computer Science at Stanford University from September 1970 until July 1976. During this time, I was affiliated with the Stanford Artificial Intelligence Laboratory (SAIL), which at the time was located in a building on Arastradero Road, above the Stanford campus. My initial funding came from my NSF graduate fellowship. Subsequently, I was supported as a research assistant from a grant under the NSF "Research Applied to National Needs (RANN)" program.
My research activity primarily focused on three major areas:
- Programming Languages: In my first year at Stanford I took over development and enhancement of the SAIL programming language. SAIL is/was an ALGOL-based language initially developed by Dan Swinehart and Bob Sproull. It included an innovative associative retrieval mechanism from an earlier language (LEAP) developed by Jerry Feldman (my thesis advisor) and P. D. Rovner and was heavily used at Stanford and several other sites. I made numerous extensions to the language, including a generalized multi-thread process capability, coroutineing "matching procedures" used in data queries, garbage collected structured data types, improved methods for inter-mixing compiled and hand-coded assembly language code, etc. I also coordinated an informal development team of several other graduate students who made other enhancements to the language, including an innovative macro system based on co-routining parsers (Hanan Samet) and a source-level debugger/interpreter (John Reiser). Some of this work was reported in a 1972 FJCC paper, Recent Developments in SAIL, an Algol-based Language for Artificial Intelligence.
- Programmable Robotic Systems: I worked with colleagues in the Hand-Eye group at SAIL on several generations of programmable robots, primarily on programming languages and real time software infrastructure for these systems. Two tangible products from this work were a high level robot programming language called AL and an interactive programming environment for robots called POINTY. This work provided the context for my Ph.D. thesis research, which is discussed below.
- Task-Level Programming of Robotic Systems: My Ph.D. dissertation, The Synthesis of Manipulator Control Programs from Task-Level Specifications, was directed toward automatic generation of manipulator control programs from task-level specifications. The central assumption is that much manipulator-level coding is a process of adapting known program constructs to particular tasks, in which coding decisions are made by well-defined computations based on planning information. For manipulator programming, the principal elements of planning information are: (I) descriptive Information about the objects belng manipulated; (2) situational information describing the execution-time environment; and (3) action information defining the task and the semantics of the execution-time environment. I used a standard subtask in mechanical assembly, insertion or a pin into a hole, i to focus the technical issues of automating manipulator coding decisions. This task is first analyzed from the point of view of a human programmer writing In the target language, AL, to identify the specific coding decisions required and the planning information required to make them. Then, techniques for representing this information in a computationally useful form are developed. Objects are described by attribute graphs, in which the nodes contain shape information, the links contain structural information, and properties of the links contain location information. Techniques are developed for representing object locations by parameterized mathematical expressions in which free scalar variables correspond to degrees of freedom and for deriving such descriptions from symbolic relations between object features. Constraints linking the remaining degrees of freedom are derived and used to predict maximum variations. Differential approximations are used to predict errors in location values. Finally, procedures are developed which use this planning information to generate AL code automatically.
Early Work at IBM (before I got into Medical Robotics)
I joined IBM's T.J. Watson Research Center in July 1976 as a Research Staff Member. I spent a total of 19 years in IBM as a Research Staff Member and Research Manager before moving to Johns Hopkins University in September 1995. During that time, I was involved in numerous projects, initially focused on manufacturing applications and (later) on medical applications of robotic systems. Here are a few highlights:
- Cartesian control of robot manipulators: I joined the "Mechanical Assembler" project at IBM Research. The robot trajectories were defined and executed in joint space, largely because of the rather limited computational power of the Series/7 minicomputer used to control the robot. During my first year at IBM, I developed a path interpolation scheme for Planning and Generation of Straight Line Manipulator Trajectories that produced multi-segment Cartesian motion with bounded path error when the actual joint control was done by interpolation in joint space. I used quaternions for computational efficiency and some time later wrote an IEEE Transactions paper with a U. Penn (Janez Funda) graduate student and his advisor (Lou Paul) on this aspect.
- Two point testing in IBM manufacturing applications: Also in my first year, I developed the software to control a custom-designed two-armed robot for testing the back-plane wiring on IBM mainframe computers. The challenge was that neither the positioning of the individual "mother boards" in the back-plane nor the robot arms were particularly accurate, although the robot was very precise. We were able to get around these difficulties through a combination of sensing and local calibration/registration of the robot to the motherboards. These systems enabled IBM to eliminate a crucial production bottleneck and to ship more computers to customers. Over the years, we developed multiple generations of two-point testing systems, responding to the evolution of computer technology, which required greater precision in probe placements. The robots got more precise, as well, but we still needed to rely on sensing and feedback to place probes accurately on targets without damaging delicate components.
- Robot Programming Languages and System Architectures: The programming interface for the original IBM robot used a very low level language that had many resemblances to computer assembly language called ML. One of my initial "get acquainted" projects at IBM was to write a compiler that translated a somewhat higher language into ML. Subsequently, I developed an interactive programming language called AML that combined aspects of LISP, APL, and ALGOL. The design of AML was driven by insights gained from my earlier experience with AL. In particular, we noted that code for describing robot motion was a relatively small part of programs needed in "data driven" practical manufacturing applications that our robots were used for. Therefore, the goal of AML was that it should be primarily a very good and easily extended interactive language unencumbered with ornate syntax for motion description. The AML language was used in the IBM RS1 and IBM 7565 robots. I spend a two year internal assignment in Boca Raton developing the product architecture for these systems. Subsequently, I led efforts inside IBM Research to extend the basic RS1 architecture to include multi-threaded robot applications, embedded microprocessor controllers, and other enhancements. I also collaborated with Lee Nackman and Lee Nackman on development of AML/X, an object-oriented version of AML that was used in subsequent IBM robot systems and in some of our initial medical robotics research. I also led efforts in within the Robot Systems and Technology Group the 1980's to develop more general control architectures for programmable automation systems.
- Automatic Programming and Planning for Robots: While at IBM, I continued my interest in automatically generating robot motion plans and programs from high level descriptions of desired behavior. In 1982, I took a sabbatical to the MIT AI lab to work with Tomas Lozano-Perez and Matt Mason, on Automatic Syntesis of Fine Motion Stratgies for robotic assembly using a sequence of compliant motions defined by the local configuration space of peg-in-hole tasks. Our IJRR paper was widely cited and was the starting point for several generations of Ph.D. research at MIT and elsewhere. Other subsequent activities included work with Ken Goldberg and Matt Mason on sensor-based manipulation planning, work with V.T. Rajan on uncertainty space calculation and propagation, and work with N. Boneschanger et al. on automatic assessment of subassembly stability.
- The Robot Systems and Technology Group and the Automation Technology Department: Shortly after returning from my sabbatical at MIT, I became the manager of the Robot Systems and Technology group at IBM Research. Our main focus was on IBM manufacturing applications, together with fundamental technology, algorithmic, and systems challenges related to these problems. Subsequently, I became a middle manager, with several research groups (Robot Technology, Robot Systems, and Machine Vision) while continuing to participate actively in much of the actual technical work, some of which is mentioned above. One further example was system for course-fine positioning with endpoint sensing, which we used internally for various very precise manufacturing and test applications, such as a two-armed circuit board probing application in which a a two-armed clean room robot carried a pair of fast XY stages with sub-micron precision and optical sensors provided real-time feedback of the tool-to-target relationship. This allowed the robot to move quickly over a large work volume to place probes on delicate circuit features while dynamically compensating for residual vibrations and geometric inaccuracy in the large coarse-positioning robot.
- Medical Robotics: While I was managing the Automation Technology Department, we began collaborating with surgeons at the University of California at Davis to do basic feasibility experiments for a robotic system to prepare the femur for cementless hip replacement surgery. I began to see significant possibilities for the use of technology and information to assist surgeons in interventional procedures. So I took an internal sabbatical to develop a system that really could do surgery while also pursuing additional research in this area. This led to the formation of the Computer Assisted Surgery group, which I managed until I moved to Johns Hopkins in 1995. More about my Medical Robotics Work at IBM is on the linked page.
