Re: [Fsfe-france] Free Software Magazine

[Frederic Couchet]

>>>>> "Loic" == Loic Dachary <address@hidden> writes:

    Loic>       Bonjour,

    Loic>       J'ai dans l'idée que FSF France pourrait commander
    Loic> et redistribuer une centaine d'exemplaires du Free Software
    Loic> Magazine. Il nous en couterait 350 Euros frais de port
    Loic> compris, ce qui nous permettrait de les re-vendre à 3,50
    Loic> Euros l'unité. Je sais, je n'ai pas la bosse du
    Loic> commerce.

    Loic>       Le premier numéro paraitra le 15 Janvier ... en
    Loic> Chine ;-) http://www.rons.net.cn/english/FSM/issue01

    Loic>       Qu'est-ce que vous en pensez-vous ?

sur le principe pourquoi pas, mais quand on parcourt le premier numéro
on tombe par exemple sur l'article "Vim, an open-source text editor"
(www.rons.net.cn/english/FSM/vim, cf ci-dessous) avec quelques
passages qui montre que l'auteur n'a pas compris la différence entre
logiciel libre, logiciel propriétaire, logiciel commercial, et le
passage sur la GPL montre une incompréhension sur le sujet. L'auteur
est bien sûr libre de penser ce qu'il veut mais on pourrait attendre
du responsable du magazine une vérification des textes non ?


                      Vim, an open-source text editor
                                      
                             by Bram Moolenaar
                                      
   The main author of the open-source text editor Vim writes about how it
   came to be and how it is being developed. The Charityware concept is
   discussed and why Bram chose to use it for Vim. Diving into the data
   structures and functions to manipulate them, he gives you some idea of
   how this complex program works. Some of the new features that Bram has
   added in Vim 6.0 are discussed. 
   
  Introduction
  
   There is a small chance you have never heard about Vim. It is included
   with almost every Linux distribution as the standard Vi editor. For
   many systems such as FreeBSD and Sun Solaris 8 it comes as an extra
   package you can easily install. For other systems (MS-Windows,
   Macintosh, OS/2, etc.) executables and source code can be downloaded
   from many places on the internet.
   
   Vim is a text editor like Vi, but with many extra features; of the
   Vi-like editors Vim has often been voted the best - in fact the main
   competition comes from Emacs, in all its different versions. In 1999
   Emacs was selected the winner in the Linuxworld text editing category,
   with Vim second, but in February 2000 Vim won the Slashdot Beanie
   award for best open source text editor pushing Emacs into second
   place. So these editors are in the same league.
   
   One of the main advantages of Vim (and Vi) is that the most often used
   commands are typed with alphabetic characters. This is much faster
   than using Control or Meta key combinations, and less stressful on the
   fingers. The extra time it takes to learn the commands is soon repaid
   by allowing you to work efficiently. However Vim, unlike Vi, also
   supports the use of cursor keys in Insert mode and so a beginner can
   get started quickly before gradually learning more commands to gain
   speed.
   
   Compared to other Vi-like editors Vim has very many features: syntax
   highlighting for over 180 languages, automatic indenting for C code, a
   powerful script language, etc. There is something for everyone.
   
   The development of Vim is ongoing. At the moment of writing, the work
   for Vim version 6.0 has finished in September; more about that further
   on. The main goal for Vim is to be a very stable and useful tool;
   besides adding more features, improving the existing features and
   fixing bugs is pursued very actively. Compare that to the original Vi,
   for which not much work has been done since 1985.
   
   [gvim.jpg]
   
   This picture shows the GTK version of Vim, editing C source code with
   syntax highlighting. The search pattern "msg_didany" is highlighted
   with a yellow background. There is a Visual selection (grey
   background) with a green, blinking cursor. At the bottom "-- VISUAL
   --" indicates the current mode and at the right the position in the
   file is shown.
   
  History of Vim
  
   A long time ago I got myself an Amiga computer. Since I was used to
   editing with Vi, I looked around for a program like Vi for the Amiga.
   I did find a few so-called 'clones', but none of them was good enough;
   so I took the best one, and started improving it. At first the main
   goal was to be able to do all that Vi could do. Gradually I added some
   additional features, like multi-level undo.
   
   When it was working reasonably well, I released a version of Vim (then
   called "Vi IMitation") on a public domain disk set for the Amiga, made
   by Fred Fish. Then others started sending me patches. A few people
   took the effort to port Vim to other platforms, like MS-DOS and Unix.
   I added more features and made it work better. By that time it was
   justified to rename it to "Vi IMproved". Over time the code has been
   redesigned and extended so much that almost nothing of the original
   'clone' remains.
   
   When I started working on Vim it was just for my own use. After some
   time I got the impression it was useful for others, and sent it out
   into the world. Since then I'm working more and more on making the
   program work well for a large audience. It's fun to create something
   useful. Also, there is a nice group of co-authors and power users,
   which is very inspiring.
   
   Here is an overview of Vim's history:
-------------------------------------------------------------------------
  1991 Nov 2  - Vim 1.14: First release (on Fred Fish disk #591).
  1992        - Vim 1.22: Port to Unix.  Vim now competes with Vi.
  1994 Aug 12 - Vim 3.0:  Support for multiple buffers and windows.
  1996 May 29 - Vim 4.0:  Graphical User Interface (largely by Robert Webb).
  1998 Feb 19 - Vim 5.0:  Syntax coloring/highlighting.
  2001 Sep 26 - Vim 6.0:  folding, plugins, vertical split
-------------------------------------------------------------------------

    What does IMproved stand for?
    
   By version 1.22 Vim included more features than Vi. I decided to
   change the name from "Vi IMitation" to "Vi IMproved". Over time the
   gap has grown wider, now it's hard to think of a reason to use Vi
   instead of Vim. I would recommend all Vi users to switch to Vim. The
   list of advantages is very long, here are just a few:
- Unlimited line length and allow NUL bytes: Possibility to edit any file,
  including binary files.
- Multi-level undo and redo: Don't worry about destroying your file when
  caps-lock was accidentally on.
- Multiple windows and buffers: Edit several files at the same time, copy text
  between them.
- Syntax highlighting: Quickly understand the structure of the text and spot
  errors.
- Command line history and completion: Correct typos, recall old commands,
  quickly enter long file names.
- Delete and put rectangular text areas: Edit a table.
- Error message parsing: Run the compiler and immediately jump to locations
  where errors have been found.
- On-line help with hyperlinks: Find comprehensive documentation for any
  command and jump to related subjects.
- A powerful script language: Add your own extensions.

    Developing Vim
    
   After a couple of years of hard work version 6.0 is now finished. I
   have done the core items myself, but many people join in. Sometimes
   they create a new feature and send me a patch that I can include right
   away, but mostly the patches I receive need to be integrated into the
   current version. Few people can overview how all the parts of Vim work
   together, since the code has become quite complex. For example,
   someone created a patch for multi-line search patterns. This showed
   how it could be done, and the locations in the sources that had to be
   changed. Unfortunately the author used pointers to text lines, and
   didn't realise that these can become invalid. I had to go through all
   the new code to fix this. Although that was a lot of work, the patch I
   got helped me by setting me on the right track.
   
   Users ask questions on the Vim mailing lists, which indicates to me
   what the most common problems are. Sometimes people send me patches or
   requests for new features. I think that this co-operation of users and
   co-developers is the main strength of how Vim is being developed.
   Users and developers communicate directly with each other and with an
   open spirit. This is how open source software can grow to become
   better than commercial software.
   
  Distribution
  
   Vim can be distributed freely; however, there are some modest
   restrictions. This is the text that comes with Vim:
-------------------------------------------------------------------------
SUMMARY
Vim is Charityware.  You can use and copy it as much as you like, but you are
encouraged to make a donation to orphans in Uganda.  See |iccf| below.

If you include Vim on a CD-ROM, I would like to receive a copy.  Just so I
know which Vim distributions exists in the world (and to show off to my
friends :-)).

DETAILS
There are no restrictions on distributing an unmodified copy of Vim.  Parts of
Vim may also be distributed, but this text must always be included.  You are
allowed to include executables that you made from the unmodified Vim sources,
your own usage examples and Vim scripts.

If you distribute a modified version of Vim, you are encouraged to send the
maintainer a copy, including the source code.  Or make it available to the
maintainer through ftp; let him know where it can be found.  If the number of
changes is small (e.g., a modified Makefile) e-mailing the diffs will do.
When the maintainer asks for it (in any way) you must make your changes,
including source code, available to him.

The maintainer reserves the right to include any changes in the official
version of Vim.  This is negotiable.  You are not allowed to distribute a
modified version of Vim when you are not willing to make the source code
available to the maintainer.

The current maintainer is Bram Moolenaar .  If this changes, it
will be announced in appropriate places (most likely www.vim.org and
comp.editors).  When it is completely impossible to contact the maintainer,
the obligation to send him modified source code is dropped.

It is not allowed to remove these restrictions from the distribution of the
Vim sources or parts of it.  These restrictions may also be used for previous
Vim releases instead of the text that was included with it.
-------------------------------------------------------------------------

   I prefer to give users much freedom in using the Vim source code. The
   main reason to add restrictions is to avoid what happened to Elvis
   some time ago: someone took the Elvis source code, added a few nice
   Windows GUI things, and started selling it. Since those changes were
   not available as source code and most of that editor was still the
   original Elvis code, that didn't sound fair. Not only because people
   have to pay one guy for software that someone else made, but also
   because the author refused to publish the modified source code and
   allow others to further improve it. That's why I added the restriction
   that the source code of modifications must be made available to me.
   That still leaves room for a company to make a modified version of Vim
   and negotiate with me if their changes must be made public or not.
   This gives me the right to decide what happens with the software I
   created.
   
    Why not use the GNU GPL?
    
   The GNU General Public Licence (GPL) is more restrictive. Although it
   claims to ascertain the freedom of software, it restricts the changes
   you can make. That is, you can make changes, but when you distribute
   the modified software, you must make the modified sources available as
   well. Thus people are not free to keep these changes to themselves. I
   would say this in fact restricts your freedom. On the other hand,
   allowing anybody to make changes and keep those changes a secret, even
   though they profit from the part of the program that wasn't changed,
   also doesn't sound fair. That's why I decided to add the condition
   that the changes must be made available to me. I can then decide that
   these changes are useful for most people, and include them in Vim. Or
   decide that these changes have only a very small audience, and allow a
   company to make a bit of money from their work. After all, if the
   source code of a program must be freely available, it is quite
   difficult to require users to pay money and make a living out of your
   work.
   
   I also don't agree with the idea that all software should be free and
   open-source. All people working on free software that I know somehow
   make a living out of commercial software, either with a full-time job
   or by studying to get a job later. Without commercial software, how
   would these people make a living? I think that free, open-source
   software and commercial software will co-exist. Most commercial
   software cannot be open-source, because a company would lose its
   advantage over competitors. Creating source code is very expensive,
   and a company would not want to allow others to get the results for
   free. Since software patents and copyrights are a very weak
   protection, keeping the source code a secret is still the best choice
   in most situations. Unfortunately, this means that you are not able to
   learn from how commercial software was implemented, or add a feature
   or fix a bug in the program you bought. A solution can be making most
   of the program open-source, and keeping a small but essential part a
   secret.
   
    The future of open-source software
    
   Which software will be open-source and which will not? I don't think
   there is a definitive answer to this question. For a given
   application, is there someone willing to create and maintain it as
   open-source software? That depends a lot on the motivation someone has
   to spend time on this without being paid for it. That is an
   uncontrollable process, with unpredictable results. Not too long ago
   it was thought that only small programs would be open-source, since
   large programs would require the long-term commitment and large
   investment that only commercial companies could afford. The
   development of Linux has proven this to be wrong, and that's not a
   one-off exception. Several other large projects have popped up and are
   successful, such as KDE.
   
   In practice I see that most people are only motivated to make software
   they would use themselves. That's certainly the case for Vim: I use it
   myself every day. This is hardly a restriction though, since more and
   more people are learning how to write programs. It does restrict the
   number of people that are motivated to spend time working on a
   specific program. Theoretically it would be possible to compute the
   number of people that could work an open-source program:
 available = interested x ability x motivation

   where:
available        number of people available to work on an open-source program
interested       number of people interested in using the program
ability          percentage of this group that are able to write the program
motivation       chance that someone is motivated to write the program

   Note that the number of people interested in the program also depends
   on the availability of existing programs. If there is no software that
   is good enough or it is too expensive, this number will be higher. If
   there's already a program available that's cheap and good, the number
   will be much lower.
   
   Not everybody is able to write software and there is a large
   difference in required skills for different programs. If the goal is
   to make a program for software engineering, there is a large chance
   that the target audience includes people that have the skills to make
   the software. If the program is to process data for rare birds, the
   percentage is much smaller.
   
   The motivation factor is the big unknown in this formula. How many of
   the people that are able to write the program are actually willing to
   do it? It would be interesting to perform a study on this, and find
   out if this factor can actually be estimated.
   
   A correction to the formula needs to be made for people who are
   willing to write a program for others. That applies especially to
   software written for the disabled. Otherwise, I think this group is
   quite small.
   
   If we have computed the number of people who are available to write
   the program, the big question is if it will actually happen. Or
   better: When it will happen. Given enough time, I'm sure that every
   program for which there is a need will be made. It should be possible
   to make a formula to compute the chance that the program will be
   written this year. That's left as an exercise to the reader...
   
   Overall there are a lot of undetermined factors in this formula. I
   would conclude that it's unpredictable how much open-source software
   there will be in the future.
   
  Charityware
  
   Since Vim is open-source and freely distributable, users don't have to
   pay to use it. Even so quite a few people who use Vim regularly
   expressed to me that they wanted to reward me for my work in some way.
   I didn't really need extra money myself and didn't like the idea of
   some people giving me money for a program that is free. That's when I
   thought of the Charityware concept. The basic idea is that everybody
   who uses Vim is asked to donate to a charity. Thus the use of Vim is
   free, but if you think it's worth something, give that money to a good
   cause.
   
   How did I chose the charity? Well, I have worked for a year as a
   volunteer with a project in the south of Uganda. This is an area that
   has been struck hard by AIDS. Estimates are that 10 to 30% of the
   adults are infected by HIV. Many parents die, leaving their children
   behind. The project helps these needy children in several ways. We
   find a new home for the child. We make sure the child can go to
   school, gets medical attention and care made to measure.
   
   After I returned from Uganda, my heart was still there. I decided the
   least I could do was to keep supporting the project by raising money
   for them. The connection to Vim was a very logical one. Thus now I'm
   asking Vim users to consider donating for the orphans in Uganda. I
   have also setup an adoption program. You can financially adopt a
   child, which means that the child gets long-lasting help, which is
   best for the child. Since we work only with volunteers and the money
   is directly sent to the project, almost all the money is really used
   in Uganda.
   
   You can find more information about this at http://iccf-holland.org.
   [nabasagi_small.jpg]
   
   Nabasagi Morine is one of the children sponsored through ICCF Holland.
   
    Does the Charityware concept really work?
    
   I have received many donations for the orphans in Uganda through the
   Charityware concept. Very irregular, sometimes nothing for a month and
   then several at once. Some small amounts, some quite big. I can't
   mention the exact amounts, because not all donations go through me,
   and it is not always clear if a donation was done because of Vim. An
   estimate is that we received $2000 in 1997 and $4000 in 1998. A
   remarkable portion comes from Germany.
   
   It is not only the money. The Charityware concept also helps to make
   people aware of the need of other people. If I would not have started
   this, few people would know about the project in Uganda. For many
   people it takes time to get used to the idea that there is more to
   life than making money and taking care of yourself. I have had
   reactions from people who could not afford to donate, but were moved
   by the concept. In some way it seems Charityware changes people a bit.
   
    Is Charityware a good idea for other programs?
    
   Check these arguments whether it is a suited to your program:
    1. You don't need the money yourself. If you like spending money on a
       bigger car than your neighbour, Charityware is probably not for
       you.
    2. You currently provide your program as shareware, but you don't get
       much money from it, because people don't like giving it to you.
    3. You are currently providing the program completely for free, and
       think it is worth something.
       
   When you decide to go for Charityware, what good cause can you use?
   The best is one with which you are in contact personally. Users will
   then better understand your motivation. Otherwise, find a small
   organisation that you trust and needs more money and attention. I
   discourage supporting large organisations, especially if they already
   have enough ways to contact potential sponsors.
   
    Variations on Charityware
    
   I do not force Vim users to make a donation to the project in Uganda.
   And there is no amount specified. This gives a lot of freedom to the
   user. He may read the note about AIDS orphans and then forget about
   it.
   
   Alternatively, users can be told that they _must_ make a donation, and
   specify an amount. This remark can be put in a place that does not go
   unnoticed for the user. Hopefully this will generate more donations,
   but be careful not to annoy the user. You must set a minimum,
   otherwise the enforcement to donate is meaningless. It will be
   difficult to decide on an amount though.
   
   A more forceful approach would be to enforce the payment by blocking
   some or all functionality until the donation has been made. This is
   not a good option since it conflicts with the charitable ethic; poor
   people would not be able to use the program.
   
  Vim internals
  
   Vim has grown into a very large program. Here is an overview of the
   main parts and in which source files they can be found. I have
   included the file size for Vim 5.6, to give you an idea of how much
   code is required for each part.
startup code, main command loop                      main.c       43538
handling Normal mode commands                        normal.c     140320
executing operator commands                          ops.c        109742
displaying text on the screen                        screen.c     165136
multi-window handling                                window.c     52168
multi-buffer handling                                buffer.c     63743
typeahead, mappings and abbreviations                getchar.c    76964
terminal input and output                            term.c       109869
reading and writing files                            fileio.c     122685
swap file management                                 memfile.c    31516
managing lines of buffers                            memline.c    114193
undo and redo                                        undo.c       29014
Insert mode                                          edit.c       142302
keeping marks in files                               mark.c       25582
editing the command line                             ex_getln.c   89509
executing Ex commands                                ex_docmd.c   176724
complex Ex commands                                  ex_cmds.c    100549
Quickfix commands                                    quickfix.c   31760
regular expression matching                          regexp.c     73771
search commands                                      search.c     93721
tag commands                                         tag.c        69337
setting options                                      option.c     148884
Vim script commmands, expression evaluation          eval.c       122346
System-specific code                                 os_*.c up to 94640
Generic GUI code                                     gui.c        73468
Motif GUI code                                       gui_motif.c  26963
Athena GUI code                                      gui_athena.c 25220
common Motif and Athena GUI code                     gui_x11.c    62913
GTK GUI code                                         gui_gtk*.c   158289
MS-Windows GUI code                                  gui_w32.c    141809
Macintosh GUI code                                   gui_mac.c    82600
menu handling                                        menu.c       36164
Perl interface                                       if_perl.c    21246
Python interface                                     if_python.c  51814
Sniff interface                                      if_sniff.c   25035

   Notice that the files are generally big; this is because I prefer to
   keep related items together in one file. There could be more and
   shorter files. That would be preferred when using a revision control
   system. For me the current way of working doesn't cause problems, and
   I don't see a good reason to spend time on changing it. All files are
   together in one directory. The main advantage is that a simple "grep"
   command can be used to find all locations where an identifier is used.
   
   There are a few remarkable items in the list. The undo functionality
   is relatively small. That's because it uses a well thought-out
   mechanism, which is simple and powerful. On the other hand, you can
   see that the GUI files are generally big. Coding for the GUI just
   tends to be a lot of work.
   
   Professor Michael Godfrey has done some studies on the Vim
   architecture and its development. You can find this at
   http://plg.uwaterloo.ca/~migod/.
   
   Following are a few specific items that you might find interesting.
   
    Portability
    
   Vim was made such that it works on many different operating systems.
   This wasn't easy. Just supporting the most popular Unix versions is
   already a big task. Adding support for MS-DOS and MS-Windows gives the
   additional problem of using file names with a backslash in them.
   Machines such as the Amiga and the Macintosh have a different kind of
   operating system, which require specific solutions.
   
   One of the main choices in Vim was to use the good old K&R C. This can
   be compiled on just about any system. The advantages of using ANSI C
   can be mixed into this, by using the pre-processor. For example,
   function prototypes look like this:
   
 void ml_open_file __ARGS((BUF *buf));

   For non-ANSI compilers the "__ARGS(())" part evaluates to "()". These
   prototypes are generated with the cproto program. This not only avoids
   the need to type these prototypes, it also avoids errors. It's a bit
   more work to use the __ARGS() construct, but it makes Vim work on old
   systems, and still allows modern compilers to check the function
   prototypes, which avoids a lot of trouble.
   
   To be able to compile Vim on many different systems, there is a
   system-specific file for each of them, such as "os_unix.c" and
   "os_amiga.c". It contains most of the system-dependent code. However,
   this doesn't cover everything. There are a lot of "#ifdef" constructs
   in the generic files. Sometimes I clean this up to avoid the clutter.
   This involves creating a "mch_xxx()" function in each of the
   system-specific files, and calling it from the generic files. Since I
   don't have all these different systems, it requires co-developers to
   test this change.
   
   For Unix "autoconf" is used to detect all kinds of things about the
   system. It finds out which include files there are, the location of
   libraries, etc. This uses a description file with the tests to be
   done, called "configure.in". Autoconf processes this file and produces
   the "configure" shell script. Care has been taken that this shell
   script can be run on any Unix system. Although using autoconf isn't
   easy, I still recommend it. If you want your program to be portable
   over different Unix systems, autoconf is the way to go.
   
   The standard way is for the user to run "configure", which generates
   the Makefile. Vim is a bit different, because there is already a
   Makefile in the distribution. When running "make" for the first time,
   it will start configure for you, with the default arguments. This
   generates a "config.mk" file, which is included from the Makefile. The
   main advantage of this different approach is that the Makefile can
   contain examples of configure arguments, which you can uncomment.
   Otherwise you would have to figure out the configure arguments by
   running "configure --help", inspecting its output and typing the
   arguments to configure. For Vim you can edit the Makefile, read the
   comments, and remove or insert a few "#" characters right there. This
   is much more convenient. Moreover, running configure directly still
   works, thus this approach doesn't have drawbacks.
   
    Storing text
    
   The main task of a text editor is to read a file, modify it and write
   it. There are quite a few demands for how the text is kept inside the
   editor:
- Reading and writing files should be fast.
- No restrictions on line length, file length or character set.
- Making changes in the text should be quick.
- The code to handle the text shouldn't be too big.
- Allow for undoing changes.
- When the system crashes, recovering unsaved changes must be possible.

   There is no easy way to meet all these. First of all, it's not
   possible to keep all text in memory. Some systems just don't have
   enough, and for others it's very inefficient to use a lot of memory.
   That is why I chose to use a swap file. File access can be slow, so
   some of the text should be kept in memory via a caching mechanism.
   Since file I/O works faster when using fixed-size blocks of a few
   Kbyte, it's better not to write individual text lines to the swap
   file. After considering a few alternatives, I decided to use a
   construct that stores a number of lines in a block. These blocks are
   handled similar to the BSD "db" functions, but optimised for how Vim
   uses text.
   
   You can find the code to deal with blocks of text lines in memfile.c.
   It maintains a cache to reduce the disk I/O. This works very much like
   any cache system: blocks are written to the swap file when using too
   much memory, or to save changes that the user made. This allows
   recovering the changes from the swap file, in the event that the
   system goes down unexpectedly
   
   Packing text lines in a block is done in memline.c. Since the blocks
   are fixed in size, the number of lines that fit in them varies. Quite
   a lot of code is required to handle this, to be able to insert lines,
   delete lines and change lines. The complexity mostly comes from
   situations where an inserted line doesn't fit in an existing block,
   requiring the need to split that block in two. There is also an
   exception: If there is a single line that doesn't fit in a normal
   block, a block is created that is big enough to contain the line. This
   allows handling lines of any length, without compromising the
   efficiency of handling for short lines.
   
   The blocks with text lines are stored in the swap file without a
   specific ordering. If the blocks were ordered, inserting a block
   halfway into the file would require all remaining blocks to be
   shifted, which is very slow. To be able to find a line by its number,
   index blocks are used. An index block contains a list that tells which
   line is in which block. If a file is big, this list doesn't fit in a
   single block. It is then split over several blocks, and another index
   block is made to refer to these index blocks. This forms a balanced
   tree of index blocks, with the text blocks as the leaves. This
   construction has proven to be very reliable and efficient.
   
    Syntax highlighting
    
   Parsing a file to recognize its structure can be complex. Add to that
   the demand that parsing might have to start anywhere in the file, and
   you end up with a difficult task. Fortunately we were able to split
   the task into two pieces: The core syntax engine, which has been
   implemented in C, and the specification of the syntax items for each
   language with patterns. This allows adding support for a new syntax
   without recompiling Vim.
   
   The basic idea is that a regular expression pattern is used to specify
   an item of the language. Vim tries to match that pattern in each line
   of the text. If it matches, the matching text is highlighted. For
   items that are kept within one line this is quite simple to implement.
   For items that cross a line boundary it becomes more complicated. In
   Vim you need to specify a pattern for where the item starts and where
   it ends. For example, a C comment starts with "/*" and ends with "*/".
   But what happens when Vim starts displaying a line halfway a comment?
   There is no "/*" there, thus it would not be recognized as a comment
   line. This requires searching backwards in the text to find out if the
   text is inside a comment or not. This is called syncing.
   
   Another construct is nesting. This happens if you want to highlight
   the word "TODO" when it's inside a comment. You then have a todo
   syntax item inside a comment syntax item. On the other hand, if you
   find a "/*" inside a string, it is not the beginning of a comment.
   Thus a string item cannot contain a comment item. All this is done
   with contained items. For each syntax item the user can specify which
   other syntax items it may contain. This fits quite well with the
   structure of most languages.
   
   The syntax code maintains a stack of nested syntax items. For any
   point in the file, Vim will look for items that can be contained in
   the item that is currently at the top of the stack. Thus this stack
   represents the current state of the syntax engine. To speed up the
   syntax highlighting, this state is saved and used again later when the
   same line is displayed. When making changes to the text the saved
   syntax state may become invalid. A change in one line may cause a
   change in syntax for many following lines. For example: when inserting
   "/*" to comment-out some code, all syntax states until the matching
   "*/" will become invalid. The code notices that the state after the
   "*/" is the same as the old state, and so all states after this will
   also be valid. Thus recomputing the syntax can stop at this point.
   
    Data structures
    
   Vim uses many different data structures to store information. There is
   a trade-off between using a generic kind of structure and using a
   structure specifically tailored to an application. For example, a
   string can be stored in a fixed size array on the stack, a fixed sized
   chunk of allocated memory or a dynamically sized chunk of allocated
   memory. Which one to use depends on the type of string to be stored:
     * For strings that have a known maximum size, using the stack has
       the advantage that the time consuming malloc() and free() don't
       have to be used. This can usually be done with file names, but Vim
       tries not to restrict the user, thus other strings can be of any
       length and the stack is not suitable.
     * A fixed size of allocated memory can be used for items that don't
       change often. This is used for option values, for example. It
       doesn't waste memory and the overhead of malloc()/free() is not
       important.
     * For strings that change often, calling malloc()/free() often
       consumes a lot of time. This is especially true for strings and
       lists that can grow in size. In Vim a dynamically allocated piece
       of memory is used for this - a "struct growarray". The memory
       allocation is made in large steps, so that a certain number of
       items can be appended without reallocating.
       
   I have ended up creating quite a few data structures in Vim. The
   disadvantage is that this has taken a lot of time, it adds to the
   complexity and makes maintenance more work. The advantage is that Vim
   works very fast and efficient. Compare this with Emacs, where the main
   complaint is that it's slow and consumes a lot of resources. I have
   the impression that Emacs uses generic constructs and depends on the
   computer to be fast. In my opinion I have made the right choice. A
   text editor is used all day by many people; it is worth investing time
   and effort in making it work well.
   
  The future
  
   In November 1998 an survey was held to allow Vim users to vote for
   changes to Vim. This resulted in a good overview of what users wanted
   to be added. This is the top six:
 1. Add folding (display only a selected part of the text)
 2. Vertically split windows (side-by-side)
 3. Add configurable auto-indenting for many languages (like 'cindent')
 4. Fix all problems, big and small; make Vim more robust
 5. Add Perl compatible search pattern
 6. Search patterns that cross line boundaries

   The first three and the sixth items have been implemented in Vim 6.0.
   Fixing problems continues as usual! Search patterns have been extended
   to include all the Perl features, but they are not directly Perl
   compatible. That would cause problems with backwards compatibility.
   
    Folding
    
   One of the first features that was added to Vim 6.0 is folding. This
   is a mechanism to hide part of the text, so that the overall structure
   can be seen. It looks like folding paper. This is a big change,
   impacting on all parts of the code - and it's not very clear how a
   user would use folding for different kinds of files. I have already
   spent quite a bit of time trying to come up with a good user interface
   for folding. Since different users have different demands, I ended up
   with several folding modes:
  manual     The range of lines that forms a fold is selected manually.
  indent     The indent of a line is used to decide if it will be folded.
             For example, all lines which have an indent of more than 8 spaces
             can be folded.
  expr       Like "indent", but use an expression to decide about the fold
             level of a line.
  syntax     The syntax highlighting items specify fold regions.
  marker     Markers in the text are used to specify the start and end of a
             fold region.

   Most methods allow folding without changing the file. That is required
   for most people who are just viewing a file, or are sharing files with
   other people. But it doesn't allow specifying exactly where the folds
   need to be. Therefore the "marker" method has been added. This allows
   you to insert markers exactly where a fold should start. For example,
   if you have a few comments just before a function definition, you can
   put the marker just above the comment that should be in the same fold
   as the function. You can also include some text to explain the
   contents of the fold, for example "local declarations" or the name of
   a function and a short note about it. The disadvantage is that the
   file needs to be modified to add the markers.
   [fold.jpg]
   
   This picture shows C source code with folds. The 'foldlevel' is set to
   show one fold line for each function. These folds have been defined
   with markers in the text, like "{{{1", with a short text to explain
   the contents of the fold. The fold for the "lineFolded()" function has
   been opened to be able to view it.
   
    Multi-line search patterns
    
   As mentioned above, I have added the ability to match a line-break
   with a regular expression pattern. The changes to the documentation
   for this new feature are quite short; it's easy to explain the new
   patterns which can be used to match a line-break. The changes to the
   source code were more extensive. The main issue is that only one line
   of text from a buffer is guaranteed to remain in memory. When getting
   another line, the previous one may be freed to make room. This limits
   the amount of memory used. The code that was calling the pattern
   matching function often assumed that its pointers would remain valid,
   but now that this function may need to retrieve other lines in the
   buffer to check for a match, this is no longer the case. The solution
   is to use locations in the file - line number and column - instead of
   pointers
   
   I thought that including the patch for multi-line patterns would be a
   day of work, but it turned out a week's worth of changes. It's easy to
   underestimate the impact of a change. The true scale often becomes
   clear only during the implementation and testing of the new
   functionality. If this happens in commercial software development you
   miss your deadline, go over the budget and are generally in trouble;
   thus it is necessary to carefully inspect the impact of a change and
   spend time on making a plan. Fortunately, for free software
   development this isn't all that important. You just end up spending
   some more time and the release is ready a bit later. Therefore it is
   often not worth incurring the overhead of planning. That's a big
   advantage of free software development over commercial software!
   
    UTF-8
    
   Another item that has made it into version 6.0 is internationalisation
   (i18n). Vim already supports editing double-byte files. This is called
   multi-byte support, but it actually doesn't work for more than two
   bytes. Since most of the world appears to be moving to Unicode, that
   is what I have added. Historically Vim has internally worked with
   bytes; therefore it's a logical step to use UTF-8 encoding (Unicode
   characters encoded as a sequence of 8-bit bytes). An alternative would
   be to use wide characters (16 or 32 bit), but that would require all
   code that handles text to be changed. Since most text would still be
   7-bit ASCII, it would also be inefficient.
   
   When adding support for UTF-8 there are a few important decisions to
   be made. Will all text inside Vim be encoded with UTF-8, or is there
   an option that switches between ASCII, UTF-8 and perhaps double-byte
   encoding? Doing everything in UTF-8 has the advantage that most of the
   code will be simpler, since it only has to deal with one type of
   encoding. But typing characters, displaying text and copy/paste with
   other applications may use another encoding, requiring conversion
   between UTF-8 and other formats. This could cause overhead and even
   changes in the text. The latter is anacceptable and must be avoided.
   If you read a file with Vim and write it without making changes, you
   should get exactly the same file.
   
   Thus the main question is whether the text should be converted to
   UTF-8 when it is read into Vim, or kept in its original encoding:
   
     Keep all text internally in UTF-8 format.
   
   When reading a file, convert it to UTF-8, when writing it convert it
   to the file format. Typed text also needs to be converted, if the
   input method does not supply UTF-8 format. If the screen uses another
   format, it will also require conversion.
   
     Keep the text in the original format of the file.
   
   The typed text and characters sent to the screen only need to be
   converted when they use another format. All functions that handle text
   must be aware of the possible formats. Conversion needs to be done
   when yanking text in one window and putting it in another if the files
   use a different encoding.
   
   Both alternatives have their advantages and disadvantages. After
   thinking about this for a while, I decided to implement a mix of both.
   The internal format can be selected. It can be ASCII, UTF-8, a
   double-byte encoding or something else. Usually this should match the
   environment (the current locale). When possible, conversion is done
   between the file format and the internal format. When the conversion
   is not possible (it is unavailable or there are illegal characters) it
   is skipped This possibly results in wrongly displayed text, but at
   least the text isn't modified when it is written back.
   
  Conclusion
  
   Vim has grown to become a big open-source program. Not only is it
   useful for many people, it can also be used as an example of how
   open-source programs are developed. I hope Vim inspires and helps
   other open-source authors with their work.
   
   If you want to use Vim, you can find it in many places: if you have a
   Linux, FreeBSD or Solaris 8 distribution, look for a Vim package which
   you can install easily. For Linux the Vi command may start a minimal
   version of Vim, install a package if you want to use more features.
   
   For information about downloading Vim use this URL:
   http://vim.sf.net/download.php.
   
   A lot of information about Vim can be found on its web site:
   http://www.vim.org. Fortunately Sven Guckes volunteered to maintain
   the Vim web site, allowing me to concentrate on developing Vim. Vim
   tips and plugins can be found on http://vim.sf.net. These are created
   and uploaded by Vim users.
   
   If you have questions, bug reports, or want to help with Vim
   development, join one of the Vim mailing lists. See
   http://www.vim.org/mail.html.
   
   [kop_small.jpg] Bram Moolenaar studied electrical engineering at the
   Technical University of Delft and graduated in 1985 on a
   multi-processor Unix architecture. Although now mainly working on
   software, he still knows how to use a soldering iron. He was born in
   the Netherlands, in the area where tulips are grown, and now lives in
   Venlo, in the east of that country. He worked many years for Oce to
   design their first digital photocopying machines; he is currently
   doing freelance work and spends a lot of time on the free, open-source
   text editor Vim (http://www.vim.org), of which he is the main author.
   This involves handling e-mail from users and co-developers, fixing
   bugs and implementing new features. Bram founded the ICCF Holland
   foundation (http://iccf-holland.org) to support the Kibaale Children's
   Centre in Uganda. This project helps AIDS victims by providing
   schooling, medical attention and care made to measure. His home page
   is http://www.moolenaar.net, and his email address is
   address@hidden
     _________________________________________________________________
   
   Never forget your freedom! All of web contents in this site is created
   by free software. Send your comments or questions about this web page
   to our webmaster.  Back to Home Page
   Last modified: Jan 02, 2002
   [zope_ball] [powered_by_freebsd] [gimp] [powered_by_GNU_Emacs] [css]
   [xhtml] 
   
   © 1995-2001 RON's Datacom Co., Ltd.
   All Rights Reserved.

-- 
Petition contre les brevets logiciels      http://petition.eurolinux.org/
Frederic Couchet                     Tel: 06 60 68 89 31 / 01 49 22 67 89
APRIL                                               http://www.april.org/
Free Software Foundation Europe                 http://www.fsfeurope.org/