The IDIMS system came up in a recent post, so as part of my continuing history of computer archaeology, I thought I’d talk a little bit about it. IDIMS was the premier tool for remote sensors in the 80s, before it was made obsolete by the next generation of image processing software made possible by the UNIX-powered, server-driven architectures that started taking over in the 90s. Those too, passed on and were replaced by small but powerful software packages and GUIs running on the Microsoft desktop workstations of the new millenium. I was introduced to IDIMS and remote sensing in the early 80s, and worked on these systems until the end of that decade. I then moved on to Geographic Information Systems, a different discipline altogether. I was born in 1947, and finished my formal education in 1977, so this puts it in a personal context. I was a young man just starting in the field, and having the time of my life.

I saw my first IDIMS system at the Remote Sensing Group of the Gulf Research Labs in Pittsburgh, PA. I brought to the field academic degrees in astronomy, math and geography, plus a knowledge of FORTRAN programming. My industrial experience was 5 years of analog aerial mapping and photogrammetry, plus 2 years of computer cartography (batch processing on mainframes, using flatbed ink plotters for output). In those days, an applications programmer was taught the software skills he needed on the system he would be working on. What you were expected to bring to the table was science, math, a high level programming language, the fundamentals of the data to be processed and how it was going to be applied to decision making in the real world.

My astronomy degree was really my major asset. I was already familiar with optics and optical systems, the nature of light, atmospheric effects, basic orbital mechanics, the principles of spectroscopy, plus enough earth sciences, physics and chemistry to make sense of what I was looking at and talking to others about it. Astronomers are already remote sensing specialists, in both data aquisition and interpretation. Their sensors just point up, not down. I bring this up not to brag about my experience, but to emphasize that knowledge of the computer technology involved was secondary.

When I started in remote sensing, I had never done any real-time processing, and certainly no interactive programming. I would be introduced to both on the IDIMS system. Not only would I be using these concepts, I would actually be coding them up. For me, prior to that, a terminal was simply an editor for typing up your punch cards and submitting them electronically to a massive computer in the basement. It was incapable of displaying images, it only worked with text, and there was no GUI. Those of you who know me are no doubt aware that I am constantly harping on how the human requirements for technologists today are very different than what they were when I was starting in the field. I don’t think I could survive in today’s tech environment, even if I was thirty again.

IDIMS was a system used by both civilian and government users in the fields of satellite reconnaissance and resource monitoring and management. It was developed and marketed by Electromagnetic Systems Laboratories (ESL), a subsidiary of TRW in Sunnyvale, Silicon Valley. I worked for two years on the Gulf Oil IDIMS system, then spent five years on the development team at ESL, helping to expand and maintain it.

When you bought an IDIMS system, it was delivered in a truck; packed in big wooden crates. Today, you can get the same capability on a CD you can stick in your laptop at one thousandth the price. But there were advantages to the old way of doing it. The users played an active role writing the necessary software, not just in executing it, and the users were also familiar with the application. You knew what the software was supposed to do, and could quickly determine whether or not it was actually doing it. And when you got something to work, or if it didn’t, you could explain exactly what you did to someone over the phone. When a process failed because of something you did wrong, you could get help and fix it yourself.

In our team, we had system programmers who configured the sytem, operators who ran it, programmers who designed the applications, and scientists who specified them. My role was as an applications programmer, an intermediary, a translator if you like, between the scientists who used the system and the engineers that made it work. In practice, this meant the geochemist users would hand me a professional monograph detailing a new processing technique or the characteristics of some new sensor, and they would tell me, “code it up”. As I did so, I consulted with the sys and ops guys so they could implement my code and make it run. There was a lot of cross-fertilization and consultation, and it was real group effort. I felt very comfortable and productive in that team environment.

The IDIMS system consisted of hardware and software components, and extensive paper documentation for both. The hardware you got off the truck consisted of an HP3000 minicomputer, a dedicated array processor (a pure number-cruncher), and an HP 1000 mini to handle communications between the two. Peripherals included two 9″ tape drives for image data I/O, a printer, several external disk drives, several dumb terminals, and one or two color display devices. The displays looked like terminals, but they had no keyboards, only a trackball/cursor user interface. This is what you displayed your images on. Displays were expensive, about 50 grand each. If you wanted image hard copy, you took a picture of the screen with a 35mm camera and sent the film off to be developed.

Software items included the HP operating system for the computers, the usual file management, operations and programming software (I/O, editors, compilers, assemblers and linkers, and the IDIMS system manager itself). Along with the latter you got a list of IDIMS image processing functions, or applications; both those that came with the system and those contributed by other users (or which you could write and maintain yourself). Also included was a set of IDIMS utilities, housekeeping and data handling subroutines you could call from your own program when creating new functions. You just linked them in after you compiled your own code. You had the ability to write interactive programs, functions that could periodically stop and present the user with intermediate results, or request instructions for further processing options. There was also an IDIMS interactive command language, which allowed you to process images in real time, and which could be used to create macros for executing multi-function processes. As I recall, there were several hundred functions supported by IDIMS, and many others in the Contributed Library donated by other users in other shops.

Our users would study a geological process they were interested in, and look up the sensor capabilities and products available in the data, and provide you the specifications for the software they needed for their analyses. You would then create a function, either from scratch or by modifying an existing function, that called the necessary IDIMS utilities to do the job. The user would then execute the function interactively, and do his image interpretation, frequently asking for changes and enhancements, or identifying bugs. It sounds cumbersome and complex, but it was a stimulating and exciting collaboration, we all felt we were actively contributing to the science, we weren’t just pencil-necked geeks feeding the machinery. My scientists learned a lot of DP, and I learned a lot of geology. It was not unusual for each of us to make suggestions that would help the other in their area of expertise.

In the last years of IDIMS, the decision was made to migrate the functionality to a new platform, the more capable VAX system. The transfer was never quite totally successful, in spite of every effort to make the change seamlessly, there were subtle differences between the two systems and very often identical results did not result from identical data being processed though identical functions. Although the differences were rarely consequential ones, many shops felt the need to have identical imagery and statistics from both systems in order to ensure the continuity and consistency of the science. Getting through these issues required a strong users group, one where the clients had real clout with the company.

ESL busted its ass keeping its users happy, to its credit. We were very sensitive to complaints and requests, and the annual User’s Group meeting was always Panic Time for the company. (Ohmigod, what do they want NOW?”). Granted, the users were always a small group, at most a few hundred clients, but they were sophisticated. Every change or modification to IDIMS, no matter how technically justifiable or necessary, had to be carefully presented and sold to the user community, and they were picky, picky, picky. But it is still quite a contrast to the way most software vendors operate today. “Like it or lump it, dude. And by the way, we’re overhauling the GUI in next year’s release. You’re going to have to rewrite all your production macros. Deal with it.”

I hope I haven’t bored you all to tears with this rant, but that’s my little piece of what I experienced in image processing/remote sensing during the time I was active in it. Or at least, how I interpreted it. I hope it gives you some insight into whatever you are doing now.