One of the Common Lisp packages that I find very useful is f2cl, originally written by Kevin A. Broughan and Diane M. K. Willcock [1], with recent contributions by Richard J. Fateman and Raymond Toy [2]. It can convert a lot of the Fortran 77 codes out there fairly reliably into quite fast Common Lisp code. The lack of a fancy home page or even a separate project (it is currently part of CLOCC) makes it somewhat hard to find, but should not be taken as a sign of abandon. A more prominent project that uses f2cl is the open source CAS system Maxima.

The advantages of using f2cl over a foreign function interface are, among other things, that the generated Common Lisp code has far better error handling, and is much more convenient to use (for a Common Lisp programmer, at least).

I think it would be nice to have ASDF system definitions for the Fortran 77 packages bundled with f2cl. This would complement the mk:defsystem definitions already available. A small difficulty is that the f2cl compiler has a lot of options, and it is desirable to want to set them globally, eventually overriding them on a file by file basis. At least, this is what is done in some of the .system files. I managed to convince ASDF to do this, but I am not sure if the way i did it is the best solution. From the point of view of OOP this can be (and will be) improved, of course. I’m referring more to the need of things like accessing ASDF symbols that are not exported. In any case, having global options that can be overriden on a component basis looks like a feature one would like to have every now and then.

A typical f2cl .system file looks like this (for example):

(mk:define-language :f2cl
    :compiler #'f2cl:f2cl-compile
    :source-extension "f")

(mk:defsystem minpack
    :source-pathname "clocc:src;f2cl;packages;minpack"
    :components
    ((:file "minpack"
		      :language :lisp)
     (:module "minpack"
	      :source-pathname ""
	      :source-extension "f"
	      :package :minpack
	      :language :f2cl
	      :compiler-options (:include-comments t
				 :keep-lisp-file t
				 :relaxed-array-decls nil
				 :array-type :array
				 :array-slicing t
				 :package :minpack)
;; etc

Note that :file components can also have a :compiler-options, although I don’t know how the overriding behavior really is.

To convince ASDF to behave in a similar way, I subclassed the asdf:module class, adding an f2cl-options slot.

(defclass f2cl-module (module)
  ((f2cl-options :initform nil
		 :initarg :f2cl-options
		 :accessor f2cl-options)))

To have an f2cl source file type with corresponding compilation, one does

(defclass f77-source-file (cl-source-file)
  ((f2cl-options :initform nil
		 :initarg :f2cl-options)))

(defmethod source-file-type ((c f77-source-file) (s module)) "f")

(defmethod perform ((o compile-op) (f f77-source-file))
  (apply #'f2cl:f2cl-compile
	 (cons (make-pathname :name (component-name f)
			      :type "f"
			      :defaults (component-pathname f))
	       (compiler-options f))))

But here is the catch. Compiler options have to be computed from the module *and* the file. I did it like this

(defmethod f2cl-options ((c f77-source-file))
  (let* ((c-c-o (slot-value c 'f2cl-options))
	 (p-c-o (when (typep (parent c) 'f2cl-module)
		  (f2cl-options (parent c)))))

    ;; override global options.
    (append c-c-o p-c-o)))

where

(defmethod parent ((c f77-source-file))
  (slot-value c 'asdf::parent)) ;; Oops

Anyway, it works. Now I am wondering if I overlooked the apropriate feature in ASDF, or another way of doing this. Accessing ‘asdf::parent is good enough for me, but before attempting to contribute .asd files to f2cl, I would prefer to have a more solid solution.

P.S: The rest of the .asd file looks like this

(defsystem minpack
  :components
  ((:f2cl-module :minpack
	    :f2cl-options (:include-comments t
			       :keep-lisp-file t
			       :relaxed-array-decls nil
			       :array-type :array
			       :array-slicing t
			       :package :minpack)
	    :components
	    ((:file "minpack")
	     (:f77-source-file "dpmpar" :depends-on ("minpack"))
	     (:f77-source-file "enorm" :depends-on ("minpack"))
	     (:f77-source-file "fdjac1" :depends-on ("dpmpar"))
	     (:f77-source-file "fdjac2" :depends-on ("dpmpar"))
	     (:f77-source-file "qrsolv" :depends-on ("minpack"))
	     (:f77-source-file "lmpar" :depends-on ("dpmpar" "enorm" "qrsolv"))
	     (:f77-source-file "qrfac" :depends-on ("dpmpar" "enorm"))
	     (:f77-source-file "lmdif" :depends-on ("dpmpar" "enorm" "fdjac2" "lmpar" "qrfac"))
	     (:f77-source-file "lmdif1" :depends-on ("lmdif"))
	     (:f77-source-file "lmder" :depends-on ("dpmpar" "enorm" "lmpar" "qrfac"))
	     (:f77-source-file "lmder1" :depends-on ("lmder"))
	     (:f77-source-file "dogleg" :depends-on ("dpmpar" "enorm"))
	     (:f77-source-file "qform" :depends-on ("minpack"))
	     (:f77-source-file "r1mpyq" :depends-on ("minpack"))
	     (:f77-source-file "r1updt" :depends-on ("dpmpar"))
	     (:f77-source-file "hybrd" :depends-on ("dogleg" "dpmpar" "enorm" "fdjac1"
						  "qform" "qrfac" "r1mpyq" "r1updt"))
	     (:f77-source-file "hybrd1" :depends-on ("hybrd"))
	     (:f77-source-file "hybrj" :depends-on ("dogleg" "dpmpar" "enorm" "qform" "qrfac"
						  "r1mpyq" "r1updt"))
	     (:f77-source-file "hybrj1" :depends-on ("hybrj"))
	     ))))

Footnotes

[1] K. A. Broughan and D. M. K. Willcock, Fortran to Lisp translation using f2cl, Software Practice & Experience, 26(10) pp. 1127 – 1139 (October 1996)

[2] R. J. Fateman and R. Toy, Converting call-by-reference to call-by-value: Fortran and Lisp coexisting, Proceedings of the 2003 international symposium on Symbolic and algebraic computation, pp. 95 – 102, 2003.

Note: If you want to comment, please take into account that the wordpress comment box is not friendly to code.

The Nelder-Mead algorithm is a rather popular algorithm for (low dimensional) nonlinear programming. The original version (from 1965, see [1]) has known and varied failure modes, but never really had to fear for its popularity. It doesn’t need derivatives, which can be quite convenient, and has a reputation to work well even with noisy and rough functions.

Recently, a few provably convergent variants have been devised (see [2], [3], and [4]). The one by A. Bürmen et al [4], the “Grid Restrained Nelder-Mead Algorithm” (GRNMA) is notable because it does not impose a “sufficient descent” condition at each iteration, which makes it closer to the original algorithm.

A reliable, simple-to-use derivative free minimization routine is a handy little tool to have around. It turns out that implementing the GRNMA is also a pleasant programming project.

Here you can download a rather faithful implementation of the GRNMA (in Common Lisp). I also included the classical Nelder Mead algorithm for completeness.

The performance of these improved methods (judging from the numerical experiments in [4]) seems to be about equal in terms of number of function evaluations. But as a side effect of the additional reliability, they tend to be more efficient than the original algorithm even when it does not fail.

The code is under the MIT licence, with no warranty of any sort, etc.

References:

[1] J.A. Nelder and R. Mead, “A simplex method for function
minimization,” The Computer Journal, vol. 7, pp. 308-313, 1965.

[2] P. Tseng, “Fortified-descent simplical search method: A general
approach,” SIAM Journal on Optimization, vol. 10, pp. 269-288,
1999.

[3] C.J. Price, I.D. Coope, and D. Byatt, “A convergent variant of the
Nelder-Mead algorithm,” Journal of Optimization Theory and
Applications, vol. 113, pp. 5-19, 2002.

[4] A. Bürmen, J. Puhan and T. Tuma, “Grid Restrained Nelder-Mead
Algorithm”, Computational Optimization and Applications, vol.
34, no. 3, pp. 359 – 375, 2006.

It turns out that, as pointed out by Jens Axel Søgaard, the method I wrote about a few days ago is the alias method by Walker, as found in D. Knuth’s TAOCP, but with a modification by R. A. Kronmal and A. V. Peterson. Their contribution, it seems, was the algorithm to rearrange the intervals in O(N) operations.

In any case, there is a very good and complete textbook, nowadays available freely on the net, where all these things are explained, disected, and thoroughly overengineered in every good sense of the word. The book is “Non-Uniform Random Variate Generation”, by Luc Devroye (thanks to Brad Lucier for mailing me this great link).

Below you’ll find some additional notes about the method, prompted by reading part of the book, and a link to an updated version of the CL code. (For simplicity, I will write as if in the context of my first post).

The first thing I found interesting is that, while in theory one random number is enough, in practice this doesn’t always hold, because one is using the higher bits to choose the slot, and the lower bits to compute the alternative. The more you need for the former, the less you have for the latter. This means that the simulated variable will produce the values with slightly different probabilities than those specified. However, I think it is unlikely for this deviation to be really important unless the number of possible values of the variable becomes large, but it is something to think about.

The other interesting point is that the method can be made even faster. The trick to speed up the generation of values of the random variable is to add more possible values but with probability zero. What happens then is that one has a variety of slots for whom only the alternaive needs to be considered, and thus one can avoid the second call to the random number generator. The more of these phantom values, the less calls are needed. And if these ‘dead’ slots are rearanged in the correct order, which btw is very easy, it is also possible to reduce the number of array accesses.

In old days that seems to have been very important. I doubt that an array access will make that much difference nowadays, but being so easy to implement, it doesn’t cost you much either. Economizing calls to the random number generator, in particular if it is an expensive one, seems to me a bigger advantage.

I have updated my implementation, and included the option :phantom-values to specify the number of extra values with zero probability to consider. This version goes the safe route, using a second call to RANDOM should it be necessary, but also avoids array accesses when possible. You can get it here.

A new version of slime has been released. See here, or download it directly as a gzipped tar archive or as a zip file.

Some days ago, there was some discussion in #lisp about how to program a random variable with nontrivial probability distribution. That is, given values x₁,…,x_n, and probabilities p₁,…,p_n, how to write a function that randomly returns one of those values with the corresponding probability each time it is called.

It turns out this problem has a beautiful, simple, and efficient solution called the alias method (R. A. Kronmal and A. V. Peterson. On the alias method for generating random variables from a discrete distribution. The American Statistician, 33:214–218, 1979.), that requires only one call to the random number generator. While I haven’t read any complete explanation of it (only the one in R. A. Kronmal, A. V. Peterson, The alias and alias-rejection-mixture methods for generating random variables from probability distributions. Proc. 1979 Winter Simulation Conference (ed. H.J. Highland et al.) 269–280, 1979, which isn’t complete), I think it is highly unlikely that it is different from the method I describe below.

An Common Lisp implementation can be found here.

The alias method

If you choose one real number between 0 and 1 at random with a uniform distribution, the probability that it lies between a and b, with
0≤a<b≤1, is b-a. So the obvious solution to our problem is to put intervals I_i in [0,1) of length p_i next to each other. Each time we want a value of our random variable, we choose a value v in [0,1) at random with a uniform distribution, and see in which interval it falls. If it falls in I_k, the value is x_k. The problem is that finding the interval is costly.

The fundamental idea behind the alias method is that this approach still works if each I_i was subdivided in nonoverlaping, not necessarily contiguous intervals J_i¹, J_i²,…,J_i^k_i in [0,1) such that the sum of their lengths is p_i. This can be used to rearange the intervals in such a way that it is easy to know in what interval a given point lies, and to which value the interval corresponds. To do this, one divides [0,1) in n intervals L_k of equal length, and then packs each with at most two intervals belonging to different x_k. Thus, finding out in what interval a point falls amounts to finding out, first, in which L_k it falls, which can be obtained by division and truncation, and then finding out which one of the two possibilities that are left holds.

To get this redistribution right, pack L₁ with an interval whose length is less or equal than 1/n, and, if necessary, take away what is needed to fill up L₁ from one whose length is greater than 1/n. Now, filling up the rest of the L_i is very close to the initial problem, but with one value (and one interval) less.