1 / 80 Lutz Prechelt, [email protected]Open Source Software Development • Definition • Process characteristics, strengths • Economical incentives Course "Softwareprozesse" • Project types, leadership and decision-making • The OSS career • OSS license types • OSS tools • Innovation management in OSS Lutz Prechelt Freie Universität Berlin, Institut für Informatik http://www.inf.fu-berlin.de/inst/ag-se/
80
Embed
Open Source Software Development · "A Case Study of Open Source Software Development: The Apache Server", Intl. Conf. on Software Engineering, ACM press, 2000 • Analyzes the apache
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
• core, mpm common, beos, event, mpm netware, mpmt os2, prefork, mpm winnt, worker, mod actions, mod alias, mod asis, mod auth basic, mod auth digest, mod authn alias, mod authn anon, mod authn dbd, mod authn dbm, mod authn default, mod authn file, mod authnz ldap, mod authz dbm, mod authz default, mod authz groupfile, mod authz host, mod authz owner, mod authz user, mod autoindex, mod cache, mod cern meta, mod cgi, mod cgid, mod charset lite, mod dav, mod dav fs, mod dav lock, mod dbd, mod deflate, mod dir, mod disk cache, mod dumpio, mod echo, mod env, mod example, mod expires, mod ext filter, mod file cache, mod filter, mod headers, mod ident, mod imagemap, mod include, mod info, mod isapi, mod ldap, mod log config, mod log forensic, mod logio, mod mem cache, mod mime, mod mime magic, mod negotiation, mod nw ssl, mod proxy, mod proxy ajp, mod proxy balancer, mod proxy connect, mod proxy ftp, mod proxy http, mod rewrite, mod setenvif, mod so, mod speling, mod ssl, mod status, mod suexec, mod unique id, mod userdir, mod usertrack, mod version, mod vhost alias
• Audris Mockus, Roy Fielding, James Herbsleb: "A Case Study of Open Source Software Development: The Apache Server", Intl. Conf. on Software Engineering, ACM press, 2000
• Analyzes the apache project during 1996 to 1998/99
• 50,000 emails sent over mailing list during this timeframe
• essentially all communication goes over the list
• phone and personal email are uncommon in many OSS projects
• A voting system with quorum is used for decision-making
• Common code base is managed in a CVS system
• a central file server, storing and coordinating subsequent versions of each file
• Change requests are managed in a problem reporting database (BUGDB)
• http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/ (Version 3.0 (rev. 1.57), September 2000)
Describes two styles of software development:
• Cathedral style: (=most of the commercial world)
• closely integrated groups of highly skilled individuals plan thoroughly and implement with much care and no haste
• "built like cathedrals, carefully crafted by individual wizards or small bands of mages working in splendid isolation, with no beta to be released before its time"
• Bazaar style: (=most of the open source world)
• open for participation by everyone, hardly any central planning, no competence guarantee, quickly evolving
• "resemble a great babbling bazaar of differing agendas and approaches "
Why finding and fixing defects is easier with OSS:
• In closed source cases, users and developers use different mental models of the system
• users: surface phenomena
• developers: code structure, program state and control flow
• But defect reports stated in terms of surface phenomena are often useless
• because the failure can often not be reproduced
• e.g. because the user did not report some important condition
• In contrast, Open Source gives users the chance to report defects directly in terms of problematic program elements
• For difficult-to-locate defects with multiple symptoms or multiple different paths from symptom to defect, it is useful if many people attempt to find a path: One will stumble over a simple path even if most will fail.
• "It is not only debugging that is parallelizable; development and (to a perhaps surprising extent) exploration of design space is, too."
Preconditions for founding a successful OSS project:
• "A certain base level of design and coding skill is required"
• "It is absolutely critical that the coordinator be able to recognize good design ideas from others"
• But you need not have those ideas yourself (Linux is an example)
• "A bazaar project coordinator or leader must have good people and communications skills."
• "19. Provided the development coordinator has a communications medium at least as good as the Internet, and knows how to lead without coercion, many heads are inevitably better than one."
• A physical good, when available for free, will be overused and hence damaged or destroyed ("free riding")
• "Tragedy of the Commons" (William F. Lloyd, Garrett Hardin)
• However, an intellectual good such as software may even gain from being available for free. Reasons:
• Software/ideas are not damaged when used by more people
• Market share increases and thus quality improvements become more likely: It is sufficient that any one person sees making an improvement as rewarding (for himself/herself)
• That all others get the same improvement for free is irrelevant
• It is impossible to realize the potential market value of small improvements (if that exists at all)
• Rivals are unlikely to profit from the revealing
• There are things that the free-rider cannot get without participating (fun of writing code, community, learning)
• To not openly submit might actually cost something: work for re-integrating the improvement with future versions
• Assume you have created some useful in-house solution for a problem that is not business-critical
• e.g. Cisco built a modification of the Unix print spooling service that could re-route "low on toner" print jobs to nearby printers in a global company network, notify administrators, etc.
• You would like to assure you can maintain the solution even if its (few) developers leave your company
• 2 in Cisco's case
• Your best route is opening source and getting other companies to start using the same solution
OSS economical-view success factors: Company reputation scenario
Commercial OSS justification 5: Give away a recipe, thus advertise your restaurant (The Zope case)
• Service companies can immensely increase their name recognition and reputation by opening source on internal products
• Examples:
• Cygnus Solutions support for GNU tools (1989!)
• Red Hat, SuSe Linux support and services
• Zope Inc. (formerly Digital Creations) web development
• Raymond also describes less important models he calls • "Accessorizing" (O'Reilly), • "Free the future, sell the present" (Aladdin), • "Free the software, sell the brand" (Sun/Java), • "Free the software, sell the content" (could be AOL)
• In small projects there is often a single person with meta-write access who makes the decision at his/her own discretion
• Some large projects define various roles and behavior explicitly and may have formalized decision-making rules and decision-making bodies for granting write-access (join-scripts)
• http://httpd.apache.org/dev/guidelines.html
• http://www.python.org/dev/process/ and /dev/patches
• A group of people use explicit democratic decision processes and drive the project like a society drives a democratic state
• Example: Apache software foundation • Quotes from "http://www.apache.org/foundation/how-it-
works.html#management"
• "Projects are normally auto governing and driven by the people who volunteer for the job. […] "do-ocracy" -- power of those who do. This functions well for most cases.
• When coordination is required, decisions are taken with a lazy consensus approach: a few positive votes with no negative vote is enough to get going. […]
• […] a negative vote includes an alternative proposal or a detailed explanation of the reasons for the negative vote.
• […] In the great majority of cases, the concerns leading to the negative vote can be addressed.
• This process is called "consensus gathering" and we consider it a very important indication of a healthy community."
• A single highly respected person makes all important decisions
• Examples: Linux, Python
• In 1991, the finnish student Linus Torvalds started writing an operating system kernel
• His message on comp.os.minix in August 1991: http://groups.google.com/group/comp.os.minix/ msg/b813d52cbc5a044b
• "I'm doing a (free) operating system (just a hobby, won't be big and professional like gnu) for 386(486) AT clones. […] It is NOT portable (uses 386 task switching etc), and it probably never will support anything other than AT-harddisks"
• Today, the Linux kernel consists of at least 4 MLOC
• Yet Torvalds (or one deputy) still has to personally accept every change to this code before something is official
OSS leadership types 2: Benevolent dictator model (2)
• Guido van Rossum started developing the programming language Python in 1990
• In 1996, he wrote (in the introduction of Mark Lutz' book "Programming Python"): "[…] in December 1989, I was looking for a 'hobby' programming project that would keep me occupied during the week around Christmas."
• Today, Python is one of the most popular scripting languages and teaching languages
• The Python development community calls van Rossum the "Benevolent Dictator For Life" (BDFL)
• for an example of this status, see the next slide
• A formal organization (often called a foundation) is build in order to host a significant group of related projects that have something important in common
• such as technology, leadership/governance principles, or philosophical principles
• May or may not have a protagonist or main sponsor
Examples:
• Apache Software Foundation
• is a non-profit corporation with 501(c)(3) U.S. charity status
• members are individuals, new ones accepted by current member vote
• Goals: Support OSS projects , create a reputable receiver for donations , provide legal shelter to project participants , protect the "Apache" brand
• As of November 2010, runs about 137 highly regarded projects
• Runs an "incubator" for systematically integrating further projects into the foundation
• There is a rather large number of OSS licenses that define the rights of the public with respect to the software
• e.g. (as of 2006-10) Academic Free License , Adaptive Public License , Apache Software License , Apache License, 2.0 , Apple Public Source License , Artistic license , Attribution Assurance Licenses , New BSD license , Computer Associates Trusted Open Source License 1.1 , Common Development and Distribution License , Common Public License 1.0 , CUA Office Public License Version 1.0 , EU DataGrid Software License , Eclipse Public License , Educational Community License , Eiffel Forum License , Eiffel Forum License V2.0 , Entessa Public License , Fair License , Frameworx License , GNU General Public License (GPL) , GNU Library or "Lesser" General Public License (LGPL) , Historical Permission Notice and Disclaimer , IBM Public License , Intel Open Source License , Jabber Open Source License , Lucent Public License (Plan9) , Lucent Public License Version 1.02 , MIT license , MITRE Collaborative Virtual Workspace License (CVW License) , Motosoto License , Mozilla Public License 1.0 (MPL) , Mozilla Public License 1.1 (MPL) , NASA Open Source Agreement 1.3 , Naumen Public License , Nethack General Public License , Nokia Open Source License , OCLC Research Public License 2.0 , Open Group Test Suite License , Open Software License , PHP License , Python license (CNRI Python License) , Python Software Foundation License , Qt Public License (QPL) , RealNetworks Public Source License V1.0 , Reciprocal Public License , Ricoh Source Code Public License , Sleepycat License , Sun Industry Standards Source License (SISSL) , Sun Public License , Sybase Open Watcom Public License 1.0 , University of Illinois/NCSA Open Source License , Vovida Software License v. 1.0 , W3C License , wxWindows Library License , X.Net License , Zope Public License , zlib/libpng license
• for details see http://www.opensource.org/licenses/
• All other licenses can be considered "in between" GPL and BSD license
• Sort of a middle ground is defined by the Mozilla Public License (MPL):
• it discriminates deriving from existing parts (which must keep their previous license) from deriving by adding new parts (for which one can choose a license freely)
• This means that, say, a company can still build proprietary extensions of a work, but has to publish changes in existing parts back to the community.
• Most licenses differ from these 3 types only by comparatively minor additional restrictions or permissions
• or sometimes even only in their wording of what are essentially the same rules
• When creating derived works from multiple OSS products at once, make sure the respective licenses are compatible
• You will sometimes need a lawyer to answer that question
• Example problem: The original Apache Software License was not compatible with the GPL since it contains a patent retaliation clause. (Apache Version 2 has resolved that)
• Since many developers work in parallel on the source code it is often hard to ensure that after committing the project still compiles.
• Also building packages for a large number of platforms is tedious.
• One solution: Continuous Integration
• An automatic script watches the version repository and keeps a current version locally. It then builds the project for all platforms and runs regression tests.
• If builds or tests fail, it "blames" the authors of last commits.
• Communication and coordination are particularly difficult in OSS projects
• We 1_sensed that it might be helpful to actively and explicitly promote coordination-related information in such projects
• We 2_envisioned a new role in OSS projects, the Moderator, whose task is information management:
• explicitly collecting and organizing information that speeds up information access for many participants (in particular new ones) and avoids redundant questions or searches
• We 3_offered this "invention" to a project (GNU classpath)
• We offered to set up a Wiki
• There we could collect and structure information regarding e.g. design decisions
• by actually compiling initial information found in the mailing list archive and putting it there regarding (a) design decisions, (b) newbie instructions, (c) current development topics
• We continued maintaining this information, adding more from time to time and announcing it via email, thus triggering 5_adopting the new practice
• After some time, a few other project members started using the platform, too
• Also for new purposes, such as arranging physical meetings
• Specific actions for 6_sustaining the practice did not appear to be necessary
• The Moderator role has apparently been distributed and filled since
• Empirical research project on aspects of OSS: company use of OSS, company participation in OSS projects, markets and business models, developer motives
• http://www.wizards-of-os.org/
• Conference on freedom of information (has OSS as one aspect)
• http://conferences.oreillynet.com/oscon/
• O'Reilly Open Source Convention
• Michi Henning: "The rise and fall of CORBA", ACM Queue 4(5), June 2006, www.acm.org
• A story how design-by-committee can fail that teaches how and why OSS proceses are often successful. Interesting!