Saturday, November 03, 2007

I get irritated because too many people in the "computing" business don't understand the word "lambda". Its just irritating because at the heart of it, its just a simple fundamental idea, like the notion of natural numbers or counting. So just spend a few minutes, understand what it means and stay educated. In a few years (if not already), almost every language is going to expose something like a lambda to its programmers. So just pick up the idea ok?

This is a working description, it wont make you an expert on the Lambda Calculus. It might give you some intuitions about it though. Lets get started.

So assuming you are familiar with a C-like language, how do you write a function that takes an integer and returns it?

int foo(int x) {
    return x;
}

Now lets look at what this is saying, it says there is a function called foo, it takes an int which it names x and it returns an int. the body of the function has a single statement called return x. Great! Now, this function is a very famous one and is called the identity function (usually goes by the nick-name id). Also when we talk about functional programming we dont really care very much for the C-style types. So let's rename the function and drop the types.

id(x) {
      return x;
}

Don't worry too much about the types. Typed functional languages usually have a different notion of types from the C like languages. The types will come back, but in a bigger and better form. We however wont discuss any type theory in this article.

So now, what it is saying? It says here is a function called id, it takes some value that it calls x and it returns x. Ok, now lets take a look at this closely. In the syntax that we have here, the name of the function is part of the syntax. We would like to say - here is a function and then assign it the name id. So that we are able to look at the function and its name as two different things. Lets do that by introducing the keyword called "function". Javascript, for example, has such a keyword.

id = function(x) {
      return x;
}

Ok. Now lets make one more simplification. Lets say that the language is a little like Ruby or so where that last statement in a functions body is its return value. Did you get that? That means that we no longer need the keyword return. We can simply make the last statement x and by that we know that the return value from the function will be x.

id = function(x) {
      x;
}

Great, lets rewrite this a bit:

id = function(x) { x; }

Now for one more simplification (really C like languages are so complex!) - lets assume that we can write only one statement inside each function. Just one. If that's the case it can be much like a if with just one statement - we don't need the curly braces anymore. We also don't really need the semicolon.

id = function(x) x

Great! Congratulations ladies an gentlemen, you are looking at a lambda. Simply rename the keyword function and call it lambda and we are all set.

id = lambda(x) x

If we drop the name assignment and look at it we have:

lambda(x) x

This is a function that has no name. It take one argument and return it. Various functional languages use slightly different syntax for writing out their lambdas. The lambda calculus, the underlying calculus of these systems, usually uses a \ instead of the verbose name lambda. It also uses a dot to separate the arguments from the body of the function. A little like this:

\x.x

The functional language Haskell uses an arrow to separate the body from the arguments.

\x->x

The language(s) ML, OCaml, SML, (maybe F#) etc. use the name fun to stand for lambda.

fun x -> x

The language Scheme uses the name lambda and lots of parenthesis.

(lambda (x) x)

Ruby lets you write something very similar to a lambda like this:

lambda {|x| x}

Most of these languages give you some mechanism by which you can give a name to your lambdas (otherwise most of the time you are a little hard pressed to refer to them).

Haskell

id = \x -> x

ML

let id = fun x -> x

Scheme

(define id (lambda (x) x))

There are some alternate syntaxes for writing named lambdas in most languages:

Haskell

id x = x

ML

let id x = x

Scheme

(define (id x) x)

So what is a lambda? It is usually a name used to define a function. The function is a value that can be assigned to a variable name. Just like any other value in your language.

A lambda is an abstraction form. Normally in a program you can read the value of a variable to which you have not assigned a value. In other words is you write (x + 1), it does not have any meaning if x has not been assigned a value already. One way to look at what lambda says is that it abstracts the value of the x. So \x.(x+1) means - give me a value for x and then I will give you the value for x+1. The value of x is abstracted in the body of the lambda. This description comes very close to our informal notion of a function in a C like language.

The lambda calculus usually allows only for lambdas that take one argument. Hence to write functions that take multiple arguments we nest lambdas. We write \x.(\y.(x+y)) for a function that adds two numbers. Most functional languages support multi-argument lambdas. In Haskell this would be \x y -> (x+y).

So what's the big deal? Nothing much and quiet a lot. Nothing much because the notion of functional abstraction is something we learn from high-school math which we port over to programming languages. This is an old and familiar notion.

That said, in the functional programming community, it has long been realized (it is a large community with varying beliefs, secret cults, traditions and practices) that lambdas are the only abstraction form required for a lot of things.  One can do away with loops and conditionals (if, switch) and assignments and numbers and object oriented programming and pretty much everything you can think of (and some things you cant) - they can all be encoded using lambdas. Hence lambdas form a sort of foundational building block for a large number of programming paradigms. At the heart of all of this is the notion of a function - hence the name functional programming.

Saturday, November 03, 2007 6:03:52 PM (Eastern Standard Time, UTC-05:00)  #    Comments [2]  | 

One of my long standing irritations with Haskell is that there is no real way to generate fresh names if your code is non-monadic. Haskell is a fairly "pure" side-effect free language. Hence when you write interpreters and such in it, and you need a new name for something (say a new variable name), there is no way to generate one.

The normal (and correct) solution for this is to write your code in a State monad of some sort and pass a counter around. Every time you need a fresh name you retrieve the counter, increment it, update it and such. This is all very well, but it brings the baggage of the monadic syntax with it. It would be nice to be able to retain direct-style syntax and have fresh names.

Often when you want to test an idea, the clutter of fresh name generation simply gets in the way. Generating fresh names is essentially an effectful operation, which is essentially why Haskell does not allow it. All effects should be made explicit in the types of terms in Haskell.

Using the hack for fresh names below you side step this requirement. The direct consequence of this is that your code suddenly becomes effectful. It is no longer pure. Writing code that relies on fresh names does not automatically mean that code has a purely functional representation in the obvious way.

The other down side is that code written using fresh name generation may not even work correctly in GHC most of time. This is because GHC assumes that there are no side effects in your code. Hence it freely moves code around and does common sub-expression elimination. To clearly see what this means, consider a pure function that computes the 100th prime number. The function takes no arguments and returns a prime number to the caller. If the function is called from 5 different parts of your program, the sharing semantics of lazy evaluation ensures that it is only computed once (if at all). Subsequent calls, simply use the previously computed value.

Now imagine you write a function that returns a fresh name. If GHC decides that the function does not need to be re-evaluated, in every context where the function is called the original computed value will be returned. This means that you will get only one name ever and this is not what you want. In other words adding fresh names breaks "referential transparency".

Hence writing code that uses fresh names, as defined here, without any direct support from the compiler, is inherently at risk from various compiler optimizations. Use it based on your judgement of things. (Don't use it for the the next missile guidance system even if you think you know what you are doing. )

module FreshName(Opaque, freshName) where
-- Generate FreshNames hack
-- Roshan James, Nov 2007
 

import Data.IORef
import Debug.Trace
import Foreign

data Opaque = Opaque Int
              deriving (Eq, Show)

freshCounter :: IORef Int
freshCounter = unsafePerformIO $ newIORef 0

freshName :: (Opaque -> a) -> a
freshName f = unsafePerformIO $ do c <- readIORef freshCounter
                                   writeIORef freshCounter (c + 1)
                                   return (f (Opaque c))

In the above module, Opaque is the type of a fresh name. The Opaque data type cannot be inspected by consumers of the module. The only operation that's defined on Opaque is the equality test i.e. Opaques can be compared to each other, but cannot be used for anything else. For convenience  I have defined a Show on Opaque as well for pretty printing, in the absence of which one would need to have a lookup table to correctly display Opaques.

In some senses freshness is a strange effect. It is an effect that breaks (programs change their results) in the presence of sharing or code motion. It is completely impervious to order-of-evaluation unlike state, control operations, jumps, exceptions etc - we can switch the order of reductions any way we like and the semantics of the program is preserved.

The general rule, to my best understanding, is this: If the type of a term exposes the type of the fresh name then the term cannot be freely shared. It is not a referentially transparent term. When I say the type exposes the Opaque type, I mean that some sub-term of the type is the Opaque type or is defined in terms of the Opaque type. Terms that do not expose the Opaque type, but potentially uses them internally, are referentially transparent. Maybe this insight can form a good intuitive guideline for writing code in GHC using fresh names. 

Here is an sample usage of the fresh names module. I define an alpha conversion function using it:

substitute :: Var -> Var -> Exp -> Exp
makeVariable :: Opaque -> Var

alphaConvert :: Exp -> Exp
alphaConvert (Lam x e)  = freshName $ \op ->
                         let x' = makeVariable op
                         in (Lam x' (substitute x x' e))

Drop me a note if you find this useful or if you have a better technique.

Saturday, November 03, 2007 1:18:43 PM (Eastern Standard Time, UTC-05:00)  #    Comments [0]  | 
"We spend our whole lives in unconscious exercise of the art of expressing our thoughts with the help of words."
          -- Vincent Van Gogh
Saturday, November 03, 2007 9:49:18 AM (Eastern Standard Time, UTC-05:00)  #    Comments [0]  | 
 Thursday, November 01, 2007

Today Dr Sabry mentioned the Oral Qualifiers to me - maybe I should start putting together a reading list and selecting a committee. I thought I should put down a first list based on what comes to mind first without caring too much about what people really want to deal with. So this list will have to undergo many alterations - I thought it would be nice to preserve it. I don't know how large or small PhD Orals reading lists are - maybe I should ask around. Maybe this one is too small.

Oral Qualifiers
===============

Computational/Process Calculi
-----------------------------
Functions as Processes, Pi calculus - Milner
Join Calculus, Reflexive CHAM - Gonthier, Fournet
STM - Tim Harris, Simon PJ, Herlihy

Foundations: Logic and Mathematics
----------------------------------
Basic Category Theory - Pierce
Basic Mathematical Logic - ?
Natural Deduction and Sequent Calculus - Dag Prawitz
B*, K* logics, Linear/Affine type systems - Dunn, Walker
Lattices, Gaggle theory - Dunn, Bimbo

Types & Semantics
-----------------
Lambda Cube (Fw, Dep. types) - Barendegt, Aspinwall, Hofman
Typed Operational Reasoning, Equational Correspondence - Sabry, Harper, Felleisen
Something about Pure Type Systems - ?
Basic Model Theory, Scott Models, Denotational Semantics - ?
Calculus of Constructions - Coquand
Lambda~ mu mu~ Calculus, CBV CBN duality - Curien, Herberlin, Wadler
Type theory behind Control operators - Makarov, Sabry, Danvy, Filinski, Kiselyov, Felleisen
Theory of Objects - Cardelli, Abadi

As I look at this list I wonder, what can I possibly do in the *real* world after I get out of here? Maybe I should be writing a web server or some such thing. That way when I am out of here maybe someone will say "He knows all this Category theory nonsense. Really! I looked it up on Wikipedia, it really is called the generalized-abstract-nonsense. But he's also written a web server, maybe we should interview him?"

 

If you are wondering what PhD Oral Quals are like, maybe these serve as a good explanation -

More...

Thursday, November 01, 2007 12:06:29 AM (Eastern Standard Time, UTC-05:00)  #    Comments [3]  | 
 Wednesday, October 31, 2007

Aziz has released his shiny new Ikarus Scheme compiler. This is the first compiler specifically targeted at  R6RS Scheme. R6RS is the latest Scheme standard - as a matter of fact it is so new that the standard itself is still under development. Ikarus implements almost all of the interesting parts of the standard. It is supports a superset of R5RS, the current standing Scheme spec.

Ikarus is also a fast optimizing compiler that generates x86 opcode as of today and it runs on Windows, Mac and Linux systems. Ikarus has its own unicode engine, garbage collector, loader, linker, cross platform binary format, stack management, etc etc. Pretty much all of the things you would expect to find in a JVM or a .Net. All the of the compiler is written by Aziz, and a lot of it is based on the work of his advisor, Kent Dybvig, on the Chez Scheme  system.

http://www.cs.indiana.edu/~aghuloum/ikarus/index.html:

Ikarus is a free optimizing incremental native-code compiler for R6RS Scheme.

Ikarus is free to download, to distribute, to modify, and to redistribute. The complete source is available according to the GNU General Public License (GPL3).

Ikarus is an optimizing compiler, so your Scheme code will run fast without the need to port hot spots to C "for performance". With an incremental compiler, you don't need a separate compilation step to make your program run fast. The best part is that the compiler itself is fast, capable of compiling thousands of lines of code per second.

Finally, Ikarus is an R6RS compiler. R6RS is the latest revision of the Scheme standard. The preliminary release of Ikarus supports over 80% of the most important features R6RS, and later releases will bring Ikarus closer to full R6RS conformance. R6RS libraries, scripts, record types, condition system, exception handling, unicode strings, bytevectors, hashtable, and enumerations are among the supported features.

Ikarus is probably also useful as a target language for compiling experimental functional languages as opposed to targeting C or going through the pain of compiling to ASM, .Net or JVM yourself.

Yay! I think I will be moving away from Petite Chez Scheme and start using Ikarus as my primary Scheme system (despite the fact that its under the non-free unethical GNU license which does actual material harm to the society). If functional languages and Scheme are your cup of tea, try out Ikarus!

Wednesday, October 31, 2007 8:21:50 PM (Eastern Standard Time, UTC-05:00)  #    Comments [4]  | 
 Tuesday, October 30, 2007

I have been thinking about what it means to understand something. In computer science we tend to claim that we understand something, often in a way that is truly confusing. That is because often in CS one can can get by with little knowledge of something, without developing a truly deep understanding of the nature of what we are dealing with. This is probably true of real life as well and hence this blog entry.

I think it was several years into programming with C++, and other similar languages, when it struck me that I truly don't understand object orientation and OO programming. Knowing the syntax of the class construct or writing some programs that use classes does not mean that the program is written in an OO style. What my understanding lacked here was a good enough insight to use the philosophy of OO design to write my programs as communicating objects.

Sometimes, however, the level of non-understanding is much shallower than this. Very often we use a name or a pronoun as an explanation. I was reading Richard Feynman the other day, in one of this articles he talks about his childhood and his father:

He had taught me to notice things and one day when I was playing with what we call an express wagon, which is a little wagon which has a railing around it for children to play with that they can pull around. It had a ball in it -- I remember this -- it had a ball in it, and I pulled the wagon and I noticed something about the way the ball moved, so I went to my father and I said, "Say, Pop, I noticed something: When I pull the wagon the ball rolls to the back of the wagon, and when I'm pulling it along and I suddenly stop, the ball rolls to the front of the wagon," and I says, "why is that?" And he said, "That nobody knows," he said. "The general principle is that things that are moving try to keep on moving and things that are standing still tend to stand still unless you push on them hard." And he says, "This tendency is called inertia but nobody knows why it's true." Now that's a deep understanding -- he doesn't give me a name, he knew the difference between knowing the name of something and knowing something, which I learnt very early. ... So that's the way I was educated by my father, with those kinds of examples and discussions, no pressure, just lovely interesting discussions.

I myself had developed wariness of pronoun-explanations early on, and hence my mis-explanation-beasts wear a more convincing sheep-skin. I have been reading about type systems this weekend. A couple of brilliant talks given by Kyle Ross and Michael Adams got me seriously thinking about the magnitude of my ignorance. I had a working knowledge of type systems, I knew enough tricks to impress people at juggling tricks and mimicry with respect to types, but I never truly understood them. Today I was commenting to someone that my understanding of existential quantification is just a bunch of random properties about what it stands for.  I know how to look at existentials as abstract data type encodings or as modules or such - but all those explanations seem like different ways to looking at it without actually taking the bull-by-the-horns.

My understanding of existentials reminds me of an old story about blind men seeing and elephant. Back in India, I think everyone has heard this tale at some point or the other. In the US most people don't seem to get the reference to the blind men seeing the elephant. This is an interesting and fairly good allegory for explaining  a limited or half-baked understanding.

http://www.kheper.net/topics/blind_men_and_elephant/Buddhist.html

http://en.wikipedia.org/wiki/Blind_Men_and_an_Elephant :

In various versions of the tale, a group of blind men (or men in the dark) touch an elephant to learn what it is like. Each one touches a different part, but only one part, such as the side or the tusk. They then compare notes on what they felt, and learn they are in complete disagreement. The story is used to indicate that reality may be viewed differently depending upon one's perspective, showing how absolute truths may be relative; the deceptive world of half-truths.

Coming back to computer science, the other day I was talking to my advisor, Prof Amr Sabry. We were talking about macro systems and staged evaluation and such. I believe we were talking about the KFFD algorithm or so when the topic came up. We had some algorithm that sort of works but was cluttered with a lot of details. Sometimes I tend to be a little lax about how I look at things and say, "If I can implement it, I understand it". This seemed to be a good measure of the understanding - if I can build it myself from scratch, I "understand" it.

At this point he said "Let me show you the difference between understanding something and knowing it well enough to be able to implement it. Consider the following encoding:

encoding

Now suppose I tell you that my phone number is 855-3127, can you convert it into the encoding? Well, sure you can. Can you implement it? Yes you can. You can write a little table and look up the values and such. But do you understand it?"

"You understand stand it when you are able to see this."

This put me back on my toes with respect to what I think I understand. A bit of much needed sharpening of the old intellectual-honesty-knife.

Tuesday, October 30, 2007 1:50:07 AM (Eastern Standard Time, UTC-05:00)  #    Comments [1]  | 
 Sunday, October 28, 2007

This is a great slide deck from Benjamin Pierce, Professor Computer Science at UPenn, that shows what the world of programming languages research is doing especially in the context of types.

http://www.cis.upenn.edu/~bcpierce/papers/tng-lics2003-slides.pdf

Also from Pierce is a really good introduction (the best I have found to date) about polymorphic lambda calculi and actually programming in some of those languages. This describes the hierarchy starting with the simply typed lambda calculus (F1) and describes F2, F3, etc and Fw (F-omega). He also talks about encodings of various sorts of types and a whole lot of neat stuff. All of that written in a very readable fashion. I spent my weekend on this and it was time well spent.

http://www.cis.upenn.edu/~bcpierce/papers/leap.pdf

(For some reason this pdf seems to be back to front, but its just fine when printed)

Pierce's webpage is a gold mine of information for the PL student, including neat things like a 'Great Works in Programming Languages' list. He is also a pretty good photographer. I did see him in person when I was at MSRC in summer of 2006. I believe he was visiting those days. I now regret not having walked up to him and saying hi.

Sunday, October 28, 2007 10:25:11 PM (Eastern Standard Time, UTC-05:00)  #    Comments [0]  | 
 Friday, October 26, 2007

I cannot help it if my paintings do not sell.
But the time will come when people realize that they are worth more than the cost of the paint.
     -- Vincent Van Gogh

 

Vincent
Don McLean,
1971

Starry Starry Night
Vincent Van Gogh
Saint-Rémy, France: June, 1889

 

Moonrise
Ansel Adams
Hernandez, NewMexico, 1941

Friday, October 26, 2007 9:06:48 PM (Eastern Standard Time, UTC-05:00)  #    Comments [1]  |