An interview with Sun Labs' Richard Gabriel and Ron Goldman, authors of Innovation Happens Elsewhere

In this Inner Circle Q&A, Richard Gabriel and Ron Goldman, authors of Innovation Happens Elsewhere discuss the benefits and drawbacks of open source, how companies can tap the knowledge of the open-source community.

Q: Can you briefly go through your background and what you do at Sun?

A: Richard: I am a computer scientist and poet. I've been a researcher at Stanford; I started and ran a software company (Lucid, Inc.) for 10 years; I've been a VP of Development at ParcPlace Systems, Inc.; and now I'm a Distinguished Engineer at Sun in Sun Labs.

My company, Lucid, was one of the first companies to try to build a business around open source (free software, as in Free Software Foundation). I was very active in the Lisp community and was the prime mover in getting Common Lisp going. I was one of the designers of the Common Lisp Object System (CLOS).

I was one of the founders of the software patterns community (and am the President of the Hillside Group, the educational nonprofit that puts on the Programming Languages of Programs (PLoP) conferences. You can read more than you ever wanted to know about me here: http://www.dreamsongs.com).

At Sun, I research software systems — in particular (with Ron) I am looking at what could make large systems robust, stable, and, in general, able to take care of themselves.

Ron: For the past twenty years, I have been developing software at various companies. A lot of my work has focused on programming language design and programming environments in the quest to make it easier to develop software. This has led me to think about what future software will be like — in particular large distributed systems — and what it will require to create it.

In 1998 I was offered the opportunity to research this full time at Sun with Dick, whom I had worked with previously. We started by focusing on the social and organizational aspects of how such software would be developed, in particular, developed using open source.

Since 1998, I have been helping groups at Sun Microsystems understand open source and advising them on how to build successful communities around their open-source projects. Projects I have worked with include Jini, NetBeans, OpenOffice, JXTA, java.net, and, most recently, the team working to open source Solaris.

I recently moved to SunLabs, where I am switching from the social to the more technical aspects of what is needed for future software systems. In particular, I am researching how to make software more robust.

One of the sources of inspiration for my work is biology: biological systems are much more complex than software and they are also very robust. By looking at the mechanisms that biology uses, we will hopefully be able to apply them to software to make computer systems that will work more reliably.

Q: What prompted the idea for you and Ron to write your book?

A: Ron: As we were advising different groups at Sun about open-source business strategies, we quickly got tired of having to present the same information to each group. So we looked around to see if there were any good books that we could point people to, but there weren't any.

There were interesting books on the history of various open-source projects, books on the low-level mechanics, and a few theoretical treatises, but nothing that really went into the business reasons for using open source or that looked at open source at a strategic level.

As a result, we started to write up an internal open-source handbook that we could give to folks at Sun. We kept adding material to it, and our handbook grew and grew. Eventually we realized that we had enough material for a book that would be of interest to managers outside of Sun whose companies were also trying to make sense of open source.

Richard: When I first came to Sun, I worked for Bill Joy, and one of my first jobs was to figure out how to use open-source ideas to spread Jini to the world. I hired Ron a little while after I started, and we became a couple of Sun's open-source experts. We researched a variety of issues, and (I believe) we became particularly expert at how companies could work with and use open source as part of their strategy.

We advised several of Sun's open-source projects while they were getting going: Jini, NetBeans, OpenOffice, Solaris, JXTA, and a few others. Ron and I were the prime designers of the java.net thing (Web place, community, strategy, etc.).

Q: Do you have some examples of how innovation has happened elsewhere?

A: Richard: Sun was built on innovation that happened elsewhere. BSD was a version of Unix that was developed at AT&T and Berkeley. Same with Motorola 68000 processors. We are returning to that with our Linux/AMD strategy.

Ron: Professor Eric Von Hippel of MIT has studied a number of examples where companies look to see what their customers are doing with their products. For example, many innovations in skateboarding, snowboarding, and windsurfing were created by users trying to get additional performance or capabilities from their equipment.

The companies manufacturing the equipment were able to harvest the user innovation and improve later versions of their products. Toys like Lego Mindstorms or Sony's Aibo robot dog have created enthusiastic communities of folks that have reverse engineered the toys and then extended them in interesting ways, like teaching Aibo to dance — all of which helps to grow the market for Lego and Sony.

Over 50 companies have created plug-ins for Sun's sponsored NetBeans Java IDE open source project or have otherwise innovated on top of the basic version of NetBeans.

Q: Why should large enterprises consider getting involved in the open-source community?

A: Richard: Well, that's what the whole book is about. The primary reason is to establish lines of communication with natural customer bases. It is possible to determine requirements more accurately and more quickly. And it is in general possible to get to better products faster.

Ron: There are three different ways a company can get involved in open source. The first is taking a current software product and making it available as open source. There are many reasons to do so that we discuss in our book, but one of the main ones is to create new conversations with your customers.

It allows your engineers to talk directly with your customer's engineers and to better understand their problems and how your company can help them to solve them. It can build better relationships with your customers and help you improve your products.

Second is using existing open-source applications to better solve your company's problems. Here the involvement is as a consumer of an open-source project, and the conversations are about how to make the best use of the application, to overcome problems you encounter, to report bugs, and to suggest improvements.

Third is working collaboratively with your peers to develop new solutions using the open-source process. For example, large cities depend on customized software to help provide essential services, and each city has similar needs. By working together they can save time and money and achieve a better result. Note that they are still paying their employees or contractors to write their contributions to the code, but the overall effort is one of organizations voluntarily contributing.

For an example check out the recently launched Government Open Code Collaborative Repository. I believe we will see more and more such efforts as open source becomes more familiar and better understood in the IT world. There are many similar examples of commercial companies collaborating with their competitors using open source to develop common tools and infrastructure.

Q: What are some key benefits of IT teams working with open source when developing their IT projects?

A: Ron: The most obvious benefit is that building on existing open-source components means less work for the IT team to do, which can translate into lower costs. Since open source is often built around open standards, it also brings a potential benefit of greater portability and reduces vendor lock-in.

Another benefit is to lower the risks of development. The open-source process, through its use of public mailing lists to discuss improvements, and by making frequent releases of the code, makes potential problems visible earlier than with closed-source development techniques. That makes it possible to correct such problems sooner and at less cost.

This is in contrast to traditional development, where it can be years between when a project is started and when the actual users get to see a working system.

Risk is also reduced, since with access to the source code it is possible for the IT team to fix critical bugs or make needed enhancements without needing to depend on the priorities of an outside vendor.

Richard: Open source can be used in two different ways. One is to use open-source code in projects. One way to think about this is as a post-modern way of putting together software. In the 1980s and 1990s, people talked about components and how by putting them together we could build larger software or systems.

Open source is components with the source included. This means that IT teams can use open-source components to save time, and if the components are not quite right, the source is available to fix things up. Postmodernism is about (among other things) taking pieces from wherever and making a new whole from them.

The other way to use open source is to use the open-source software development methodology. This methodology has the benefits of continuous integration, writing down decisions, diversity of ideas, distributed development, better requirements gathering, ubiquity, etc.

Q: Any recommendations how to get started with an open-source development project?

A: Richard: Again, the book is all about that. The most important thing is to make sure you believe in the development model. Second most important is to have a clear idea what your business model is and why open source is helping implement/ execute it. You need to adapt to the open-source development model, or else you will fail.

Ron: Getting started with an existing open-source effort is very easy: Just find a project that you have an interest in, download and try out the code, and sign up for some of the project's mailing lists. It is often a good idea to just watch for a bit to get the flavor of the community before jumping in with your own posts.

If you have problems, check their Web site for help and, if necessary, ask for assistance on one of their mailing lists. If you think you've found a bug, report it, and if you have a bug fix to contribute, all the better. As you become familiar with the code and the community, join in the discussions on future directions.

To start a new open-source project based on existing proprietary code, there are a number of steps you need to do that we describe in the book. The biggest change, however, is in opening up your development process: moving internal discussions to public mailing lists, involving the community in decision making, working from the public source-code repository, and similar shifts.

It is also important that you educate your management so that they understand the realities of open source and have reasonable expectations regarding schedules, level of support, resources needed, and so on.

Q: Any areas where you think it is not advisable to use the open-source community?

A: Ron: Definitely. Open source is not always appropriate. You must have good business reasons to make a product available as open source. And it has to be something that others outside your company will care about or you won't get a community.

If there are major aspects of your software or plans that you want to keep secret, then open source will not work — though it may be possible to use the practices of open source, but to limit who can participate.

A number of companies, such as HP, have created gated communities with some of their partners where they can collaborate. You can also use open source within your company, making it easier for employees in different parts of the organization to work together.

When considering using an open-source application it is important to consider a number of factors, just as when selecting any commercial software. First, even if there is no licensing fee, you need to look at the total cost of ownership, which includes support and training.

The maturity of the project, along with the size and involvement of the project's community, indicates how healthy the project is and how much support you can expect from other community members.

Finally, open source requires a different mindset, that of participant and co-developer, not being a passive, demanding customer, so if a company or its IT organization is not comfortable with a more active role, then it should stick to more traditional practices.

Similarly, many organizations want to have a guaranteed level of support that an open-source community by itself may not be able to meet. Fortunately there are an increasing number of companies specializing in supporting open-source applications for just this reason.

Richard: There is a little bit of a myth in the question. There is not a single open source community. Open source is more like a philosophy or ideology. One wouldn't talk about the positivist community or the nihilist community.

The shared philosophy means that there are shared behaviors, but there are different behaviors as well. For example, not all open-source people are antagonistic toward commercial products. Moreover, communities can be built with distinct characteristics.

Given that, open-source is not useful for very niche or very small projects. Or projects with a high degree of patentable or protectable intellectual property. But even in these cases, there could be open-source components (or even when the bulk is open source).

Questions or comments for Ron & Richard? Write to innovationbook@sun.com.