cpython/Doc/howto/advocacy.tex

\documentclass{howto}

\title{Python Advocacy HOWTO}

\release{0.03}

\author{A.M. Kuchling}
\authoraddress{\email{amk@amk.ca}}

\begin{document}
\maketitle

\begin{abstract}
\noindent
It's usually difficult to get your management to accept open source
software, and Python is no exception to this rule.  This document
discusses reasons to use Python, strategies for winning acceptance,
facts and arguments you can use, and cases where you \emph{shouldn't}
try to use Python.

This document is available from the Python HOWTO page at
\url{http://www.python.org/doc/howto}.

\end{abstract}

\tableofcontents

\section{Reasons to Use Python}

There are several reasons to incorporate a scripting language into
your development process, and this section will discuss them, and why
Python has some properties that make it a particularly good choice.

 \subsection{Programmability}

Programs are often organized in a modular fashion.  Lower-level
operations are grouped together, and called by higher-level functions,
which may in turn be used as basic operations by still further upper
levels.  

For example, the lowest level might define a very low-level
set of functions for accessing a hash table.  The next level might use
hash tables to store the headers of a mail message, mapping a header
name like \samp{Date} to a value such as \samp{Tue, 13 May 1997
20:00:54 -0400}.  A yet higher level may operate on message objects,
without knowing or caring that message headers are stored in a hash
table, and so forth.  

Often, the lowest levels do very simple things; they implement a data
structure such as a binary tree or hash table, or they perform some
simple computation, such as converting a date string to a number.  The
higher levels then contain logic connecting these primitive
operations.  Using the approach, the primitives can be seen as basic
building blocks which are then glued together to produce the complete
product.  

Why is this design approach relevant to Python?  Because Python is
well suited to functioning as such a glue language.  A common approach
is to write a Python module that implements the lower level
operations; for the sake of speed, the implementation might be in C,
Java, or even Fortran.  Once the primitives are available to Python
programs, the logic underlying higher level operations is written in
the form of Python code.  The high-level logic is then more
understandable, and easier to modify.

John Ousterhout wrote a paper that explains this idea at greater
length, entitled ``Scripting: Higher Level Programming for the 21st
Century''.  I recommend that you read this paper; see the references
for the URL.  Ousterhout is the inventor of the Tcl language, and
therefore argues that Tcl should be used for this purpose; he only
briefly refers to other languages such as Python, Perl, and
Lisp/Scheme, but in reality, Ousterhout's argument applies to
scripting languages in general, since you could equally write
extensions for any of the languages mentioned above.

 \subsection{Prototyping}

In \emph{The Mythical Man-Month}, Fredrick Brooks suggests the
following rule when planning software projects: ``Plan to throw one
away; you will anyway.''  Brooks is saying that the first attempt at a
software design often turns out to be wrong; unless the problem is
very simple or you're an extremely good designer, you'll find that new
requirements and features become apparent once development has
actually started.  If these new requirements can't be cleanly
incorporated into the program's structure, you're presented with two
unpleasant choices: hammer the new features into the program somehow,
or scrap everything and write a new version of the program, taking the
new features into account from the beginning.

Python provides you with a good environment for quickly developing an
initial prototype.  That lets you get the overall program structure
and logic right, and you can fine-tune small details in the fast
development cycle that Python provides.  Once you're satisfied with
the GUI interface or program output, you can translate the Python code
into C++, Fortran, Java, or some other compiled language.

Prototyping means you have to be careful not to use too many Python
features that are hard to implement in your other language.  Using
\code{eval()}, or regular expressions, or the \module{pickle} module,
means that you're going to need C or Java libraries for formula
evaluation, regular expressions, and serialization, for example.  But
it's not hard to avoid such tricky code, and in the end the
translation usually isn't very difficult.  The resulting code can be
rapidly debugged, because any serious logical errors will have been
removed from the prototype, leaving only more minor slip-ups in the
translation to track down.  

This strategy builds on the earlier discussion of programmability.
Using Python as glue to connect lower-level components has obvious
relevance for constructing prototype systems.  In this way Python can
help you with development, even if end users never come in contact
with Python code at all.  If the performance of the Python version is
adequate and corporate politics allow it, you may not need to do a
translation into C or Java, but it can still be faster to develop a
prototype and then translate it, instead of attempting to produce the
final version immediately.

One example of this development strategy is Microsoft Merchant Server.
Version 1.0 was written in pure Python, by a company that subsequently
was purchased by Microsoft.  Version 2.0 began to translate the code
into \Cpp, shipping with some \Cpp code and some Python code.  Version
3.0 didn't contain any Python at all; all the code had been translated
into \Cpp.  Even though the product doesn't contain a Python
interpreter, the Python language has still served a useful purpose by
speeding up development.  

This is a very common use for Python.  Past conference papers have
also described this approach for developing high-level numerical
algorithms; see David M. Beazley and Peter S. Lomdahl's paper
``Feeding a Large-scale Physics Application to Python'' in the
references for a good example.  If an algorithm's basic operations are
things like "Take the inverse of this 4000x4000 matrix", and are
implemented in some lower-level language, then Python has almost no
additional performance cost; the extra time required for Python to
evaluate an expression like \code{m.invert()} is dwarfed by the cost
of the actual computation.  It's particularly good for applications
where seemingly endless tweaking is required to get things right. GUI
interfaces and Web sites are prime examples.

The Python code is also shorter and faster to write (once you're
familiar with Python), so it's easier to throw it away if you decide
your approach was wrong; if you'd spent two weeks working on it
instead of just two hours, you might waste time trying to patch up
what you've got out of a natural reluctance to admit that those two
weeks were wasted.  Truthfully, those two weeks haven't been wasted,
since you've learnt something about the problem and the technology
you're using to solve it, but it's human nature to view this as a
failure of some sort.

 \subsection{Simplicity and Ease of Understanding}

Python is definitely \emph{not} a toy language that's only usable for
small tasks.  The language features are general and powerful enough to
enable it to be used for many different purposes.  It's useful at the
small end, for 10- or 20-line scripts, but it also scales up to larger
systems that contain thousands of lines of code.

However, this expressiveness doesn't come at the cost of an obscure or
tricky syntax.  While Python has some dark corners that can lead to
obscure code, there are relatively few such corners, and proper design
can isolate their use to only a few classes or modules.  It's
certainly possible to write confusing code by using too many features
with too little concern for clarity, but most Python code can look a
lot like a slightly-formalized version of human-understandable
pseudocode.

In \emph{The New Hacker's Dictionary}, Eric S. Raymond gives the following
definition for "compact":

\begin{quotation}
	Compact \emph{adj.}  Of a design, describes the valuable property
	that it can all be apprehended at once in one's head. This
	generally means the thing created from the design can be used
	with greater facility and fewer errors than an equivalent tool
	that is not compact. Compactness does not imply triviality or
	lack of power; for example, C is compact and FORTRAN is not,
	but C is more powerful than FORTRAN. Designs become
	non-compact through accreting features and cruft that don't
	merge cleanly into the overall design scheme (thus, some fans
	of Classic C maintain that ANSI C is no longer compact).
\end{quotation}

(From \url{http://sagan.earthspace.net/jargon/jargon_18.html\#SEC25})

In this sense of the word, Python is quite compact, because the
language has just a few ideas, which are used in lots of places.  Take
namespaces, for example.  Import a module with \code{import math}, and
you create a new namespace called \samp{math}.  Classes are also
namespaces that share many of the properties of modules, and have a
few of their own; for example, you can create instances of a class.
Instances?  They're yet another namespace.  Namespaces are currently
implemented as Python dictionaries, so they have the same methods as
the standard dictionary data type: .keys() returns all the keys, and
so forth.

This simplicity arises from Python's development history.  The
language syntax derives from different sources; ABC, a relatively
obscure teaching language, is one primary influence, and Modula-3 is
another.  (For more information about ABC and Modula-3, consult their
respective Web sites at \url{http://www.cwi.nl/~steven/abc/} and
\url{http://www.m3.org}.)  Other features have come from C, Icon,
Algol-68, and even Perl.  Python hasn't really innovated very much,
but instead has tried to keep the language small and easy to learn,
building on ideas that have been tried in other languages and found
useful.

Simplicity is a virtue that should not be underestimated.  It lets you
learn the language more quickly, and then rapidly write code, code
that often works the first time you run it.

 \subsection{Java Integration}

If you're working with Java, Jython
(\url{http://www.jython.org/}) is definitely worth your
attention.  Jython is a re-implementation of Python in Java that
compiles Python code into Java bytecodes.  The resulting environment
has very tight, almost seamless, integration with Java.  It's trivial
to access Java classes from Python, and you can write Python classes
that subclass Java classes.  Jython can be used for prototyping Java
applications in much the same way CPython is used, and it can also be
used for test suites for Java code, or embedded in a Java application
to add scripting capabilities.  

\section{Arguments and Rebuttals}

Let's say that you've decided upon Python as the best choice for your
application.  How can you convince your management, or your fellow
developers, to use Python?  This section lists some common arguments
against using Python, and provides some possible rebuttals.

\emph{Python is freely available software that doesn't cost anything.
How good can it be?}

Very good, indeed.  These days Linux and Apache, two other pieces of
open source software, are becoming more respected as alternatives to
commercial software, but Python hasn't had all the publicity.

Python has been around for several years, with many users and
developers.  Accordingly, the interpreter has been used by many
people, and has gotten most of the bugs shaken out of it.  While bugs
are still discovered at intervals, they're usually either quite
obscure (they'd have to be, for no one to have run into them before)
or they involve interfaces to external libraries.  The internals of
the language itself are quite stable.

Having the source code should be viewed as making the software
available for peer review; people can examine the code, suggest (and
implement) improvements, and track down bugs.  To find out more about
the idea of open source code, along with arguments and case studies
supporting it, go to \url{http://www.opensource.org}.

\emph{Who's going to support it?}

Python has a sizable community of developers, and the number is still
growing.  The Internet community surrounding the language is an active
one, and is worth being considered another one of Python's advantages.
Most questions posted to the comp.lang.python newsgroup are quickly
answered by someone.

Should you need to dig into the source code, you'll find it's clear
and well-organized, so it's not very difficult to write extensions and
track down bugs yourself.  If you'd prefer to pay for support, there
are companies and individuals who offer commercial support for Python.

\emph{Who uses Python for serious work?}

Lots of people; one interesting thing about Python is the surprising
diversity of applications that it's been used for.  People are using
Python to:

\begin{itemize}
\item Run Web sites
\item Write GUI interfaces
\item Control
number-crunching code on supercomputers
\item Make a commercial application scriptable by embedding the Python
interpreter inside it
\item Process large XML data sets
\item Build test suites for C or Java code
\end{itemize}

Whatever your application domain is, there's probably someone who's
used Python for something similar.  Yet, despite being useable for
such high-end applications, Python's still simple enough to use for
little jobs.

See \url{http://www.python.org/psa/Users.html} for a list of some of the 
organizations that use Python.

\emph{What are the restrictions on Python's use?}

They're practically nonexistent.  Consult the \file{Misc/COPYRIGHT}
file in the source distribution, or
\url{http://www.python.org/doc/Copyright.html} for the full language,
but it boils down to three conditions.

\begin{itemize}

\item You have to leave the copyright notice on the software; if you
don't include the source code in a product, you have to put the
copyright notice in the supporting documentation.  

\item Don't claim that the institutions that have developed Python
endorse your product in any way.

\item If something goes wrong, you can't sue for damages.  Practically
all software licences contain this condition.

\end{itemize}

Notice that you don't have to provide source code for anything that
contains Python or is built with it.  Also, the Python interpreter and
accompanying documentation can be modified and redistributed in any
way you like, and you don't have to pay anyone any licensing fees at
all.

\emph{Why should we use an obscure language like Python instead of
well-known language X?}

I hope this HOWTO, and the documents listed in the final section, will
help convince you that Python isn't obscure, and has a healthily
growing user base.  One word of advice: always present Python's
positive advantages, instead of concentrating on language X's
failings.  People want to know why a solution is good, rather than why
all the other solutions are bad.  So instead of attacking a competing
solution on various grounds, simply show how Python's virtues can
help.


\section{Useful Resources}

\begin{definitions}

\term{\url{http://www.fsbassociates.com/books/pythonchpt1.htm}}

The first chapter of \emph{Internet Programming with Python} also
examines some of the reasons for using Python.  The book is well worth
buying, but the publishers have made the first chapter available on
the Web.

\term{\url{http://home.pacbell.net/ouster/scripting.html}}
 
John Ousterhout's white paper on scripting is a good argument for the
utility of scripting languages, though naturally enough, he emphasizes
Tcl, the language he developed.  Most of the arguments would apply to
any scripting language.

\term{\url{http://www.python.org/workshops/1997-10/proceedings/beazley.html}}

The authors, David M. Beazley and Peter S. Lomdahl, 
describe their use of Python at Los Alamos National Laboratory.
It's another good example of how Python can help get real work done.
This quotation from the paper has been echoed by many people:

\begin{quotation}
       Originally developed as a large monolithic application for
       massively parallel processing systems, we have used Python to
       transform our application into a flexible, highly modular, and
       extremely powerful system for performing simulation, data
       analysis, and visualization. In addition, we describe how Python
       has solved a number of important problems related to the
       development, debugging, deployment, and maintenance of scientific
       software.
\end{quotation}

%\term{\url{http://www.pythonjournal.com/volume1/art-interview/}}
 
%This interview with Andy Feit, discussing Infoseek's use of Python, can be
%used to show that choosing Python didn't introduce any difficulties
%into a company's development process, and provided some substantial benefits.

\term{\url{http://www.python.org/psa/Commercial.html}} 

Robin Friedrich wrote this document on how to support Python's use in
commercial projects.

\term{\url{http://www.python.org/workshops/1997-10/proceedings/stein.ps}}

For the 6th Python conference, Greg Stein presented a paper that
traced Python's adoption and usage at a startup called eShop, and
later at Microsoft.

\term{\url{http://www.opensource.org}} 

Management may be doubtful of the reliability and usefulness of
software that wasn't written commercially.  This site presents
arguments that show how open source software can have considerable
advantages over closed-source software.

\term{\url{http://sunsite.unc.edu/LDP/HOWTO/mini/Advocacy.html}}

The Linux Advocacy mini-HOWTO was the inspiration for this document,
and is also well worth reading for general suggestions on winning
acceptance for a new technology, such as Linux or Python.  In general,
you won't make much progress by simply attacking existing systems and
complaining about their inadequacies; this often ends up looking like
unfocused whining.  It's much better to point out some of the many
areas where Python is an improvement over other systems.  

\end{definitions}

\end{document}