India has a odd situation with respect to politics and government. Most well educated people don't want to have anything to do with it. These fields are not considered 'good' or 'respectable' career options. The people who run the administration and drive the policy making are the ones who were the bottom of your high school or college class. No one who has the skills to be anything better considers politics to be a respectable enough life goal. While I was in India I thought this was normal and maybe even the right thing.

Why is this? In large part because we see the ugly side of our politics too often: the crudeness, the outright dishonesty, the corruption and the incompetence. "I don't want to be in there, fighting the pig in the mud".

Case in point: Growing up in Kerala I have heard this man  make too many inappropriate and distasteful comments on the state TV channels. He is the sort of textbook politician whose uncouth manner paints such a low picture of politics in my state that it deters most well to do people from having anything to do with him and his ilk.

Last week we had the terrorist situation in Mumbai that left everyone saddened and apprehensive of the future in the region. Mr Achuthanandan, who is currently the Chief Minister of the State of Kerala, shows up at the home of one of the commandos, Major Sandeep Unnikrishnan, who died fighting the terrorists. Why? To use a rather blunt metaphor, its a bit like a dog pissing on a pole to mark its territory - the obligated visited to show your solidarity and to establish your political presence.

The deceased Major's father decides that he does not want to entertain any such display in his house and he tells Mr Achuthanandan to leave. What would someone who truly felt for their loss do? What would someone who is touched by the situation in Mumbai and for those who lost their lives there do? And what does Achuthanandan do?

 

Major Unnikrishnan's father refuses to meet Kerala CM

Kerala CM insults slain Major's dad.

After talking to Pooja, I felt it necessary to mention the great mathematician G. H. Hardy on my blog. Hardy is most famous outside of mathematics for his "A Mathematician's Apology". The book, written in later in life by Hardy, talks among aother things about how mathematics is young man's game. He is also somewhat know for his association with Ramanujan and for being the person responsible for bringing him to Cambridge where his greatest mathematics unfolded.

Quoting one of the mathematicians, C Snow, that Hardy worked with:
http://www-groups.dcs.st-and.ac.uk/~history/Biographies/Hardy.html

A mathematicians apology is, if read with the textual attention it deserves, a book of haunting sadness. Yes, it is witty and sharp with intellectual high spirits: yes, the crystalline clarity and candour are still there: yes, it is the testament of a creative artist. But it is also, in an understated stoical fashion, a passionate lament for creative powers that used to be and that will never come again. I know nothing like it in the language: partly because most people with the literary gift to express such a lament don't come to feel it: it is very rare for a writer to realise, with the finality of truth, that he is absolutely finished.

Hardy was a sort of purist mathematician, one who did his mathematics not for the sake of its applicability to anything, but for the sake of doing great mathematics. Hardy, along with Littlewood and Ramanujan,  is also mention in Apostolos Doxiadis' "Uncle Petros and the Goldbach Conjecture". The link above gives a short summary on his life.

Some quotes:

Asked if he believes in one God, a mathematician answered: "Yes, up to isomorphism".

http://en.wikipedia.org/wiki/G._H._Hardy:

It is never worth a first class man's time to express a majority opinion. By definition, there are plenty of others to do that.

A mathematician, like a painter or a poet, is a maker of patterns. If his patterns are more permanent than theirs, it is because they are made with ideas.

Many years back I worked full time at Microsoft on one of the many projects related to Vista (or what was then called Longhorn). Last December I was visiting friends in the Redmond area when one of the jokingly observed how the code I had written for Microsoft still hadn't seen light of day, while two and a half years later, I had moved on and completed my Masters and had started on my PhD.

This wasn't a sort of accidental slip, but I remember we were told at that time that if we had any ideas to suggest for Vista, they better be ideas that will be new and interesting 3-4 years later when the OS actually ships. This is a tall order for any sort of idea, much less for the volatile world of "software project" ideas.

While this seemed like very ironic humor at that time, over the past month or so, once in a while I thought about what that meant. After all, I had spent a year writing all that code. A very busy year at that - I had spent 15+ hours a day, I daresay, that was my average day, struggling with the turmoil of the massive engineering effort that was Vista.

I wasn't a *great* programmer by many standards, but I'd like to think that I was better than many I had encountered. 15+ hours of my time for about a year, took a lot of out of my life and it didn't seem to have amounted to anything! Sure I was being paid a handsomely, but one one like to think that one's efforts contribute to the world in some way as well. After all that year was full of deadlines and things being rushed to be completed and such. What came of all of it? If it came to nothing, what a waste of life that was...

A few weeks back, I got a call from Steve who works with Google (who has a fascinating blog btw), about coming back and working with them on some stuff. I casually asked what happened about the last thing I had worked on, when I was there last summer. I had made some extensions to the Rhino compiler, the largest part of which was adding the yield control operator to it. Steve said that they were using it. Somewhere in the back of my mind I said "What?!".

Maybe my programming has matured over the years. Maybe the ~6 hrs a day I spent in office as an intern produced production quality code. I somehow assumed that it wouldn't see any real world use. Was it really that special that it was ready for real world use? Don't get me wrong, it wasn't bad code. But it was code that only written, not "baked" for years.

Maybe there was another reason. Maybe, it wasn't a property of the code at all, but of the fact that there was something radically different about the outlooks of both these companies. There are many thing one can say about this "difference in outlook", positive and negative things things about both. But a shift that causes developers to feel effective by default as opposed to feeling ineffective by default, is an empowering thought. "If I build it for you, what will become of it?"

Maybe mine was an isolated case of wasted engineering effort and this is nitpicking. If that's so, I'll be happy for it.

This Sunday, Kyle, Michael and I were enjoying a little late morning brunch at the lovely Le Petit Cafe and some light banter about polymorphic lambda calculi. Patrick would occasionally swing with his strange stories, narrated with his uniquely French mannerism. Joie de vivre!

Thus passed a nice Sunday morning when I see a little girl walk up to me. She is short, about 3 foot tall, not quiet opaque and is wearing a bright red cape and hood. She stands by my side, next to our table, and listens to us for a bit. Kyle and Michael were chatting about Kinds and Sorts. Then she says to me "What strange conversations you have Grandma". I say, "The better to understand them with, my dear". I expect her to scamper away, doing whatever it is that she needs to do, but instead she stands there, her large luminous eyes locked in mine. All this starts to get very irritating and I'd wish she would just buzz off when Kyle asks me something. I turn to reply mumbling that maybe I should eat her or something, that way she'd go away. Kyle asks me what I mean. I explain. The two of them suddenly get unnecessarily excited. "You hold him down. I will run for help". Groan...

Thinking back about it, I wonder if the wolf ate her simply because she was so annoying. After all, ignoring rumors, he was trying to get some sleep and there comes this pest asking all these inane questions.

After being relatively upset about Parzania, I found myself at the RSS community page on Orkut. Needless to say, that was even more disturbing. It was without a certain amount of morbid horror that I looked at their polls, their organized report-as-fraud activities, their cyber-lynchings and such.

I usually don't like to deal with this sort of thing. I am complacent, cowardly and comfortable in my relative safety. It bothered me to think that if I had no fear it would be dishonest, or just plain stupid.

I happened to look at my blog and saw Star Trek and the Q next to Parzania, it struck me what the Q might say about India and the RSS. "Your species is always suffering and dying", he would say in his How-entertaining--What-do-you-expect-anyway attitude.

And indeed, what do I expect? The world has always been a barbaric place. We express surprise and shock at the slaughter of Delhi by Nadir Shah, the slaughter of the Sikhs after the assassination of Indira Ghandhi, the India-Pakistan partition riots, the Bombay bomb blasts, the Ghodhra riots, (and hardly just India) the Khmer Rouge regime of Cambodia, the Rape of Nanking, the Hutu-Tutsi conflict, the occupation of Iraq, the massacre of the Algerians... and on and on and on. Just too many to list. In fact so many, that one should ask, what's the big deal? This is, after all, human nature.

People have been always mercilessly wiping out entire populations of "others" based on some definition of identity. Why are we so surprised by this? Why do we expect that in the times of our generation the world should have magically become a "civilized place"? Hardly, if you look at the evidence.

It will be a long time before we turn into a species that is not always suffering and dying. Something truly deep and fundamental has to change about us for that.

In the short term however, in a few hundred years or maybe in our own lifetimes, the vices of our day shall blow away to be replaced by other ones. To make any of them to go away, something fundamental needs to change that invalidates their cause of existence. A little like the way Frown Power and Superman destroyed the Ku Klux Klan, might there be an Indian equivalent? Prosperity? Accountability? Can we motivate such a change?

Q: Let us pray, for understanding and for compassion.
Capt. Picard: Let us do no such damn thing! What is this need of yours for costumes, Q? Have you no identity of your own?
Q: I come in search of the truth.
Capt. Picard: You come in search of what humanity is!
Q: I *forgive* your blasphemy!

For a while now there is a fly living in my apartment. I dont know how it got in there - I usually keep the windows closed (and even otherwise they have screens on them). Anyway, I was cooking the other day and there comes this fly zooming into the kitchen. Thats when I saw it first. I was making pasta, nothing too funny.

I dont quiet know what to do about it. Its one of those fast ones with a not-easy-to-ignore buzzing sound. And it just wont still, it insists on flying around. I have tried, unsuccessfully, to swat at it and such. I am not sure what to do ... spraying it with something like my deo doesnt feel like a nice thing to do. Also, I dont see how it can possibly find its way out.

I finally found somethng that gives me some peace, I can shut it in the bathroom - it zooms in there to annoy me when I go in. I let it out when I go in and then try trap it back in there again. Lets see how long this game continues.

Srikanth Chunduri visited me this weekend. The last time we met was in Hyd 3 years back. These days  Srikanth is up to living the life of a fast paced New Yorker in the financial district. He introduced me to some music from XLRI... enjoy.

 

If you don't understand Hindi, I would advice that you don't go about singing this without having checked the meaning of the lyrics with a friend. Enough said.

Here is another piece, which I would rather not embed on my blog:

http://www.youtube.com/watch?v=y_KT3sMR4CY

Enjoy.

I asked my soul, "What is Delhi?"
She replies, "The world is the body, Delhi its life"
                                                                                  - Ghalib

I spent the past two days recovering from a trip to New York. Recovering from the stress of meeting an old friend in whose company, the both of us inevitable live larger than life.

 

Abhijit and Shweta drove me down to NY – a 16 hour drive from Bloomington, Indiana which we covered in two days on the way there.

 

Bad weather

 

 

and the general monotone of a long drive in a closed car on a US highway were the dominant aspects.

 

 

  

Anand is doing his PhD in Social Anthropology at Columbia University.
New York, with Anand, was great city to visit

 

 

with sky scrapers

 

 

and statues,

 

 

and seagulls,

 

 

and interesting cuisine,

 

 

and public transport,

 

 

and strange dancing light creatures,

 

 

and buildings with character.

 

 

 

Christmas night was spent over poker with Jose and Shweta, Marilyn and Ashok and Shweta’s sister and her husband. The day after Christmas was spent at a Jazz bar in Harlem which had great music – partly prompted by the fact that the godfather (James Brown) has passed away. All of this punctuated by fairly insane amounts of walking, drinking, late nights running into early mornings and several other things that one can do insane amounts of, in three days.

 

All in all, much to recover from.

 

Here's linking to Anand who has more intense things to say.

I have been getting my brain fried the past few days trying to relate call-by-need and delimited continuations. Staring vacantly into space I noticed my little flying certificate from Cornwall. Happy memories, hence the post –

 

 

 

Granite cliffs around Land’s End, Cornwall taken from a Cessna 172 that I was flying.

(Yes it’s safe to let go of the controls of a small plane like that for a short while)

 

 

On foot this time.

 

Since many of my blog entries have ceased being blog entries and have become more beacons of the fact that I am trudging through life, here is another life update.

 

I am back in Bloomington, Indiana after a great summer. After many years have I had a summer that left me with so many memories. I spent the summer in Cambridge, UK and like I was telling one of my friends here at IU (Indiana University) that Cambridge is like a distillation of the essence of several Bloomingtons put together, and then matured over several hundred years – a little like comparing a fine Scottish whisky to beer – maybe a 16year old Glenmorangie in burgundy wood to a Guinness.

 

I made it a point to travel as much as I could this summer – as much as time and finances would allow. And it was great. I look at some slight embarrassment at the rant about Cornwall below. But yes I had a great time at Cornwall. Cornwall aside, took several trips to London, went to Norfolk – Hunstanton, Cromer, Great Yarmouth. London’s a pretty amazing city and I haven’t been to most of the major touristy spots. Spent some time gallivanting around Cambridge – went to Peterborough and Ely – saw the famous cathedrals. Went to Granchester – the little village near Cambridge.

 

However most of the really great travel came towards the end of summer. I had a chance to finally go to Scotland. The trip to Scotland was primarily motivated by a long standing wish to go see Skye – the island of Skye – like in the famous Robert Stevenson poem “over the sea to Skye”. Scotland was amazing in many ways – firstly it was beautiful. But that aspect of it didn’t impress me much initially – in my mind I kept comparing it to the highranges in Kerala and not seeing much more there. But after a while you realize how much grander Scotland is – the deep lochs and the high lands and the people. I got a chance to spend two night at the bank of the Lochness in a little town called Fort Augustus. At Fort Augustus I picked up a liking for good single malt whisky. I never had a liking for hard liquor before – but this is great. Favourites? Hmm… I like a Glenmorangie (esp in Burgundy wood), The Macallan, Dalwhinnie; I like this really interesting whisky liquor called the Glayva…. Scotland was interesting in other ways too – here was a land that was under conflict with the English for many hundred years and they too, much like India tries to hold on fiercely to their traditions. Scotland of today is not much like that, but an Scotsman you meet will be only happy to make clear to you how they are not Englishmen. It was alsoa funny parallel that very often tartan patterns on kilts were like the checked patterns on mundu back in Kerala.

 

After Scotland the next major travel was to Paris and then to Rome. Paris was good, Rome was great. I think aside from a love for whisky another take away from summer is the need to be Italian and to speak Italian – I am so amazed by that language – I have taken an instinctive liking to it. It also partly has to do with a good Italian friend I made this summer (who also happens to be a logician in his free time). More on all that later. Honestly these days there is too much to write about, and so many things to say about every little thing – I need to sort out my time for blogging a little more carefully.

 

Hairy Coo!

After several weeks (running into months) of blog silence, here is a post.

 

I have been busy. Been teaching myself several new things. Life in Bloomington has been good on the whole.

 

Several good things have happened in life. I am on the verge of finishing my first formal paper in computer science. I have been teaching myself how think in formal terms, not just as a programmer, but like a computer scientist. It is an interesting process.

 

My first paper should be about the yield operator – it turned out that what was idle tinkering and curiosity over the past few years actually seems to have given rise to something of theoretical value. The paper has some formalizations of things, reference implementations, comparison with existing operators and such. Looks like I just may have enough material for a follow up paper as well.

 

 

I have an interesting summer job. I am going to be (finally) at Microsoft Research in Cambridge. I am going to be an intern working with Simon Marlow and at this time it looks like I will be working on a parallel garbage collector for Haskell. It should be good fun to get back to writing systems code for a change.

 

I have seen so many things in code that I am unsure of code that I write these days. I keep seeing demons in my code. Most of the time I don’t code. When I do, and I intend to use it, it is usually in some extreme language – ruby being one extreme and Haskell being the other. I am afraid of what code I will actually write, when I get down to it and if I actually think about the code. There is an interesting Phil Wadler paper titled “Imperative Functional Programming” – I am afraid that if you give me C these days code I write maybe better described as Functional Imperative Programming. Maybe not that bad though.. at heart I think I am still an imperative programmer – a little shaken, but I can still see the machine execute in my head.

 

 

I also spent sometime writing a little book. It is incomplete, but I offer it for download here. If you are the sort, do read it for the fun of it and give me feedback. My intention is to add for text and add more machines and make it a more complete “little” book.

 

It is a book about abstract machines – machines that will teach you the essence of programming languages and constructs. It shows you how several constructs work in a concise formal way to the extend that you can take the definitions in the book, apply the syntax of your favorite language to it and create your own interpreters and potentially your own programming languages.

 

The Little Machines

 

Thats all for now. More when time allows.

 

ps. If you have any suggestions about places to visit in the UK and such let me know. I am looking for interesting things to do in the time I can take away from work.

Some quotes and thoughtlets collected over the years. Some are my own, though I don’t know which ones exactly – over time it has all got mixed up in my head.

 

From Roshan to language designers of the future, on the topic of programming language types:

If it looks like a duck, and quacks like a duck, we have at least to consider the possibility that we have a small aquatic bird of the family anatidae on our hands.

(adapted from Douglas Adams).

 

I just fixed the last bug!

 

Any sufficiently advanced bug, is indistinguishable from a feature.

-- quoting Prof Andrew Lumsdaine, Advanced Operating Systems (P536)

 

Marvin: "I am at a rough estimate thirty billion times more intelligent than you. Let me give you an example. Think of a number, any number."

Zem: "Er, five."

Marvin: "Wrong. You see?"

The mattress was much impressed by this and realized that it was in the presence of a not unremarkable mind.

-- Douglas Adams, Life, the Universe and Everything

 

Life! Don't talk to me about life.

 

Ok, so what the answer?

Its 5,1,1,3,2…

No, no, no, I don’t like it..

You don’t like it?

It doesn’t mean anything, what does it mean?

Now that’s a different question.

-- solving the universe selection problem, Quantum Programming

 

When all you have is a hammer, every problem looks like a nail.

 

Would you like tea or coffee?

Mathematician: Yes.

-- quoting Michel Salim

 

What do you mean its third party fault? I can’t get my work done and you are partying?

 

It’s an idea so simple, that understanding it messes your mind.

-- adapted from Dan Friedman, Principles of Programming Languages

 

Creating a great language doesn’t involve assuming that your users are less smart than you.

 

The language that you use defines what you can most easily think of. Languages instill patterns of thought. Certain languages make difficult the understanding of certain ideas.

 

A novice was trying to fix a broken lisp machine by turning the power off and on.

Knight, seeing what the student was doing spoke sternly- "You can not fix a machine by just power-cycling it with no understanding of what is going wrong."

Knight turned the machine off and on.

The machine worked.

-- AI Koan

 

Lambda the ultimate.

-- Dan Friedman

 

The only law in physics that we know of that has a direction with respect to time is that of entropy.

-- QP class, B629 Computer Science, Indiana University

 

Accept it. We are Labor.

 

A style makes explicit what a language makes implicit.

-- Dan Friedman

 

What I was coming to is that its something that cant be expressed in the lambda calculus.

But that’s obvious.

-- quoting Amr Sabry

 

Science is a way of trying not to fool yourself. The first principle is that you must not fool yourself, and you are the easiest person to fool.

-- Richard Feynman

 

This is not even wrong.

-- Amr Sabry

 

There exists no formal method to convert an informal argument into a formal one.

-- quoting Amr Sabry

 

There exists no reversible classical function that coverts a quantum superposition into a classical state.

-- above paraphrased by Roshan

 

Misguided rambling from Roshan: Computer Science to Physics and Back Again

There exists no formal method to convert an informal argument into a formal one. This is roughly equivalent to the second law of thermodynamics - the total entropy of any isolated thermodynamic system tends to increase over time, approaching a maximum value. This is also our only formal notion of the quantity called time. Here the law is rephrased as follows and things brings out its directional property with respect time, more clearly - It is not possible for heat to flow from a colder body to a warmer body without any work having been done to accomplish this flow. Energy will not flow spontaneously from a low temperature object to a higher temperature object. This is roughly equivalent to saying that there is not no notion of a partial computation without a notion of sequencing with respect to time – this is the ‘.’ (dot) sequencing operator of the pi calculus. The lambda calculus does not define sequencing. Are all closed systems pi-systems?

 

Don’t worry, we are just playing games.

 

Summary of the known laws of the fictitious universe –

- There exists at least one notion of fundamental duality. Using this all other forms of duality can be derived.

- There exists at least one notion of self referential quantification. Using this all forms of self referential quantification can be derived.

- There exists an order of relationship of things among themselves. There exists an order of relationship of events among themselves. In other words there exists at least one notion of ordering or sequencing.

  

The Tao that can be described in words is not the true Tao

The Name that can be named is not the true Name.  

From non-existence were called Heaven and Earth

From existence all things were born.

In being without desires, you experience the wonder

But by having desires, you experience the journey.

Yet both spring from the same source and differ mostly in name.

This source is called "Mystery"

Mystery upon Mystery,

The womb giving birth to all of being. (1)

- Tao Te Ching, as translated by John R Mabry

 

All consistent axiomatic formulations of number theory include undecidable propositions ...

Gödel showed that provability is a weaker notion than truth, no matter what axiom system is involved ...

- Gödel Escher Bach, Douglas Hofstadter

 

Thirty spokes join together at one hub,

But it is the hole in the center that makes it operable.

Clay is molded into a pot,

But it is the emptiness inside that makes it useful.

Doors and windows are cut to make a room,

It is the empty spaces that we use.

Therefore, existence is what we have,

But non-existence is what we use. (11)

- Tao Te Ching, as translated by John R Mabry

 

The impossible did not bother him unduly. If it could not possibly be done, then it must be done impossibly. The question was how?

-- Long Dark Tea-time of the Soul, Dirk Gently.

 

((lambda (x) (x x)) (lambda (x) (x x)))

 

If you're going to tell a lie, tell a big one (then nobody will believe it's a lie.)

-- Joseph Goebbels

 

Is there a name that describes a situation where all parties involved, despite understanding fully or partly the true nature of the situation, choose to play a role until an outcome of the situation presents itself in such a way that it cannot be held the sole responsibility of any of the parties involved?

 

Why can’t we all just get along?

 

The average celebrity meets, in one year, ten times the amount of people that the average person meets in his entire life.

-- Jack Nicholson.

 

Deep down, I'm pretty superficial.

-- Ava Gardner

 

 “One day an evil magician flew over his house and – “

“Just a minute, “ interrupted the king (who was very practical). ‘I didn’t know that magicians could fly!”

“Most of them don’t,” she replied, “but this one did.”

“But how could he?” asked the king. 

“Because he was a flying magician,“ she replied.

“Oh, that explains it,” he said. “Go on!”

-- Raymond Smullyan

 

I agree with you, its just that I am not willing to admit it.

 

I searched for one of my favorite quotes and found this page. I laughed my guts out for sometime - http://www.corsinet.com/braincandy/explain.html

 

Fortune has me well in hand, armies 'wait my command

My gold lies in a foreign land buried deep beneath the sand

The angels guide my ev'ry tread, my enemies are sick or dead

But all the victories I've led haven't brought you to my bed

You see, everybody loves me, baby, what's the matter with you?

Won'tcha tell me what did I do to offend you?

-- Don McLean, “Everybody Loves me, Baby”

 

Ph.D. Haskell programmer - ate so many bananas that his eyes bugged out, now he needs new lenses!

-- Evolution of a Haskell Programmer

(I nearly died laughing on this one – these days I have been trying to understand monads in the context of computational effects and CPS. If you don’t get the reference look here, Erik Meijer’s classic on the ‘Point Free Style’ - http://citeseer.ist.psu.edu/meijer91functional.html)

 

This is the sort of thing I have avoided writing about and thought I would not be, for a long time to come – I was afraid to too, mostly because I felt I would be able to or would get too involved in warding of the slack I would get for saying this. But reading this article that Sriram had linked to in his blog brought a mountain of stuff tumbling down. (I don’t know why I am so much into this software-ethical-stereotype thing these days, I have work to do you know). Before you go ahead and read the article linked above, here is what I have to say. This is not a long story, but one of those minor incidents that have stuck on to gnaw at my phsyche.

 

Rewind several years back to when I was in my second year of undergraduate studies. I had just got my first computer (yes) and Pooja had got hers. We didn’t really have internet connections those times, though we did have the modem and required hardware. We eventually figured that if we needed to talk to each other, we could use one of those direct modem-to-modem dialup software. After a lot of tinkering with dialup software and modem settings, we had something going that mostly worked for us. I had a copy of some old dos based software – I cant seem to recall its name – that we used to dialup each others computers and basically ‘chat’.

 

The software was the most reasonable one I could find for our purpose, but was however seriously limiting and got rather frustrating after a few weeks of use. God-geek-in-the-making that I believed myself to be those days, I figured that the only way I would get things to happen my way is by writing a modem communication software of my own. (I know what most of you might be thinking, but those days it wasn’t too surprising to see how often this particular approach was the panacea to all my ills).

 

Those days I also got a chance to talk to one of the few men who seemed to know everything, or atleast had a damn good idea about how to get to know the few things that he might have missed. I got a chance to talk to Jayakrishnan K about the modem’s idea that I had on my mind. JK would know about these things – he should because he had written Kerala’s first BBS (bulletin board service) by hand (as a couple of COM files on a DOS machine doing all the multitasking and all). A short discussion later, I had my head full of X-Modem, Y-modem, Z-modem etc and I was reasonably charged up. The error control would be hard, he had warned me, but that was ok.

 

So I got home and start off, eventually I figured that I probably wasn’t as smart as I thought I was. I was using intervue (Ralf Brown’s fabled Interrupt List (Ralf Brown is now Professor at CMU)) for reference. Somehow some things didn’t seem to make too much sense and I needed more information on how to initialize the modem correctly.

 

This is where GPL stepped in. I turned that in the pile of commonly shared source and software in the college students circles I had a piece of software that did modems communication. I wasn’t really sure what the full intent of the software was, but It has what I wanted – it had assembly code to initialize a modem and set all the settings and such. And being assembly code that manages a hardware device there weren’t too many ways about – you had to load some registers, write to some ports, call some interrupts etc in a certain fixed sequence of ways. Well if you are creative, you can think of several ways of accomplishing the equivalent of

Mov AX, 10

But that by itself didn’t mean that the code to initialize the modem would be written very differently by me, once I had seen how it is to be done. It also turned out that the software (as you would be expecting by now) was GPLed.

 

After the excitement of reading the ASM snippet I had a look at ‘COPYING’, the GPL text file. It seemed to me that if I were to use this piece of code from here to write my modems software to talk to Pooja the entirety of the code that I write would have to fall under the GPL license. It would not be my code anymore. Which is to say, that I would not be free to decide what I do with it – weather I sell it or hide it or share it – that choice would not be mine anymore. That was really disturbing for a while, because here I was wanting to write that piece of modems communications software to talk to a friend and I had a solution in hand, but there was nothing I could do about it – without subjecting myself to legal risk, had I written the software and shared it around without giving away my code. That was my first introduction to the viral nature of the GPL, though at that time the only word I could think of was ‘unfair’.

 

Then I figured that I probably needn’t tell anyone and I could write my software anyway. But then I wondered what would happen (with the notion of fame) that I had this famous piece of software and then one day someone realizes that I actually did have this piece of software on my hard-disk and then they would say that I stole this code illegally. This bothered me a lot because I like to think of myself as an ethical person and tend to get reasonably bothered by the converse. Immature as it may seem, I also wondered if no one could ever write modem’s communication software again.

 

And in time the feeling came to pass, and my code never got written and we used the same old piece of software until we bought ourselves internet connections and things blew over. I still wanted to know if there was a way out.

 

Years later when I had my first chance to meet Richard Stallman, founder of the Free Software Foundation and the GNU project and author of the GNU GPL, I wanted to know if there was a way for me. But in the passing time many other things were beginning to affect my thinking and I was beginning to be more bothered about the small programming communities and their spirit of unbounded sharing and coexistence with the commercial world that the rising tide of GPL acceptance seemed to be wiping out. I feared for those communities and I felt I was part of them.

 

When Stallman finally did come, he was visiting model Engineering College to talk about the danger of software patents – on his first ever visit to India. And then there were people there at the talk, who were asking Stallman how they could detect if someone has used GPLed code in their own software, maybe after modifying it. How they could track down these people and bring them to court. And there I was, wanting to ask the exact opposite – about how I could be protected for using GPLed code because there was no other obvious way to do something esp once you have seen one approach that solves the problem. The question about who protects the programmers.

 

My question never got asked.

 

People around me told me that my concern was unfounded. Things like that don’t happen. Nobody is bothered if you take from GPL code they said. Its just there to be a moral standard they said.

 

Last year, for some uncalled for reason, the topic came back again. This time over train ride home to Cochin from Bangalore when I was traveling with Sidharth. He didn’t see the problem and I was going on raving from the Karnataka border to Ernakulam railway station that its getting dangerous that there is so much GPLed code out there just lying around on the web, crawled by search engines and popped up at you without your ever intending. And if you took from these – copy paste from your browser that smart little three line snippet – you are strictly speaking, subjecting yourself to illegal usage of GPL code.

 

Over the years also I have developed this habit in programming when I am stuck I usually search the web for a solution. Maybe a smart little hack or a piece of advice on how to do something and I am chugging along smoothly again. After joining Microsoft two months back a few colleagues here advised me against doing this – you never know the legal implications of what you are doing they said. It seemed a little odd then, but then maybe I had been forgetting where I was coming from. I do use the web a lot – but now with this additional fear factor and layer of caution – and I wished I lived in a world where ‘freedom’ was a different, more real sort of freedom of choice.

 

Then I found this article : http://www.icsharpcode.net/pub/relations/amatterofinspiration.aspx

I do not know who is right and who is wrong in this debate, but I can empathize with the factors that caused it to originate. This is from the group that creates the rather well know .Net IDE called #develop – an alternative to VS and commercial IDEs and is often recommended to students…

 

I had been procrastinating making this post for some days now. Esp Chris Pratley’s words ring so close to my fears “I've been a little gun-shy of blogging about Word for fear of being inundated by what are as far as I can tell a gang of "net thugs" who roam the net making...”. Hopefully this will be the last thing I need to have to say about this entire topic for a while. My blog is getting increasingly dedicated to writing stuff that I get dragged into rather than stuff I like getting into.

 

DO NOT hold this in perspective of my employer or any previous employer or any other context - these are purely my OWN OPINIONS done on my own steam and is not nor was ever was part of any job description of mine.

So quiet frankly, I

1)       am a fraud

2)       am trying to deceive people

3)       am not ‘worth the trouble’

4)        - to say that I am not ‘worth the trouble’ is misleading, because it implies that there is some potential good to be achieved otherwise

5)       am upto doing bad things, pure and simple

6)       am starting phony open source groups

7)       should have evidence collected against me

8)       – you should be organized against me

9)       should be denounced as fraud

10)   should have protests staged against each of my meetings

11)   should have articles written about my deceptions

12)   should be treated as a liar

13)   should not be treated as respected or legitimate

14)   should have universities close down programs because I am a liar

15)   should have this campaign against me renewed each year

 

If I missed something in the above set, I apologize.

You can read it all and more here -

http://mm.gnu.org.in/pipermail/fsf-friends/2004-October/002484.html

 

The reason this applies to me is because I often talk to my juniors at college and to students as well as professional developers about .Net. I do talk about it as an open platform and I believe that they should adopt it, learn from it, use it and develop on it. (A few days back I had come across this)

 

Sriram, now you know why (like Jack Nicholson) says ‘we can’t all get along’. I have tried and fell flat on my face, several times.

 

Noufal, now you know why I don’t think that MIT’s AI labs success has nothing to do with the GPL and think that the FSF is a political agitation/movement more than anything about technology or software. Unfortunately their sphere of influence is around technology.

 

If the things I do warrant that the above apply to me, then I am not ashamed of any of them.

 

 

 

Kids, if any of you in college are reading this, then here is a piece of mindspace – you are in college/university to be learning, not to be religious or have political ideals. Learn the goods and bads of every system – but more fundamentally try to learn about as much as you can about the art of computer science – that is so rare and there seem to be so many fewer people who know about it.

 

 

 

Now, here are some basics, you just need to be able to read and think to understand these. If you know how to use a web search engine, then you could have

Standards

The open/free standard - http://msdn.microsoft.com/net/ecma/

I intentionally use the words above, because there is No definition of the English language use of the above words that is violated in the way they are used above. If word x means y in a certain political ideal, in my personal sense of ethics, followers of y cannot call those who understand x as x to be non-conformant. Your mileage on that may vary.

 

The standards document – the Common Language Infrastructure – which is to say the way the runtime should work.

Its architecture.

Its file formats

Its API

Its opcode / IL / bytecode – whatever you call it

The C# language.

 

The above are also ISO standards under ISO/IEC 23270 (C#), ISO/IEC 23271 (CLI) and ISO/IEC 23272 (CLI TR).

 

Microsoft does not own or run ECMA or ISO.

 

Patents

About the patents, here is a mail by Miguel of the Mono project.

http://www.mail-archive.com/ilugd@lists.linux-delhi.org/msg05784.html

And look at this

http://samgentile.com/blog/archive/2003/02/19/2647.aspx

 

Implementations

 

Microsoft offered the SSCLI source code

http://msdn.microsoft.com/net/sscli/

Works for BSD, Mac and Windows

The BSD bit patched for Linux is here - http://www.macadamian.com/products/sscli/download.html

These are governed by a non-viral not-for-commercial-use license

http://msdn.microsoft.com/MSDN-FILES/027/002/097/ShSourceCLILicense.htm (print it out, it fits on a page)

 

Mono

http://www.mono-project.com/about/index.html

http://www.mono-project.com/about/faq.html read the faq, before you ‘decipher’

 

 

Maybe I am a fraud who is lying about all of the above. Do you love the .Net technology; do you talk to others about it? Do you write commercial/proprietary software? Maybe you are a liar-fraud too.

 

Btw DO NOT hold this in perspective of my employer or any previous employer or any other context – these are purely my OWN OPINIONS done on my own steam and is not nor ever was, part of any job description of mine. Maybe I am writing this hastily, if I am wrong, I will correct myself. Personally, I have an objection to being called unethical.

 

I got to meet vice presidents S Somasegar and Eric Rudder yesterday. They were at IDC as part of an exec visit to India. Eric struck me as a thoughtful and intelligent person. He was technical advisor to BillG for a while, so that’s saying a lot.

 

While a lot of things cannot be discussed in a public blog, I do like the way Eric responded to one of the questions during the open forum. This question was related to losing market place in the very small business and personal use scenarios due to the often posed ‘good enough’ argument against commercial software. Among other things, Eric said that if ‘good enough’ is good enough, then we deserve to lose.

That’s taking on a much larger commitment that just saying that we will make good products and sell them, I think that that’s saying that in time the quality of what we can do will change what good enough means. That’s a measure of a company and will unlike most others ‘big’ guys in this line.

 

I also spend part of yesterday thinking about the ‘ethical’ argument with which non-commercial software tends to propagate and justify itself. I remember receiving lots of feedback on my talk about the commercial software model at the first anniversary of the Bangalore .net user group.

 

Not a lot, but a significant amount has been said from the perspective of commercial software.

http://www.microsoft.com/resources/sharedsource/Initiative/speeches/mundie_model.mspx

In general look here from time to time, some of it might seem to be a revelation -

http://www.microsoft.com/resources/sharedsource/

This is also an interesting book, with names like Lawrence Lessig an Bradford Smith in there -

http://www.aei.brookings.org/publications/abstract.php?pid=296

 

Also I realized something else, it’s probably an obvious little thing – but a lot of what you hear when you are outside Microsoft, sounds like science fiction: Avalon, Indigo, WinFS … ta da da. I now have having a weird feeling of ‘coming in touch’ right now, because I am downloading one of the builds of one of these pieces to try on my machine – the sort of thing I would not have seen for years otherwise, and it feels good. It also feels like a ghost stepped out of the shadows and then you realize that he was real all along.

 

I know this blog entry is all pithy stuff with no enumerations of ‘facts’; but that’s ok, for this one. I know it will not stand one of my ‘where are your facts?’ rants. But again, that’s ok for this one.

 

I have been putting more of my free time into digesting Scheme. I am now working on XEmacs. After several stops and starts with emacs in the past, I am finally using it (might I add - productively) to write scheme code. I figured that even if emacs is of no use to me for most of my work, I could atleast use it to write scheme code. It should be good for that for sure.

 

Don’t misunderstand me when I say this, because you may have different opinion, especially if you have lived in Unix for a long time – which I haven’t. XEmacs still leaves me feeling like I am taking piano classes. I use Scite and Visual Studio for most of my day to day work.  Of course, if I am writing a document, then it is Word. I also have a love-hate relationship with Rob Pike’s Sam editor.

 

For a long time I used the old Turbo C++ IDE in DOS (yes I am not ashamed to confess) and it served me rather well. After which it was Visual Studio 5... it was a pain to use in some cases, but with this plug-in called ‘Whole Tomato’ that I used, it was a killer.

 

When I got started on Linux, editing was a real pain – it was like I had to learn to type all over again. That was major put-off. As a matter of fact I figured that I could not suffer the ‘miserable’ editors they have there and so decided to write my own editor. (Yes, I was crazy enough to try and write an editor instead of learning arcane key combinations…) Of course, since I could edit code on Linux, I decided to write my editor in a portable way in DOS and then figure out how to take it to Linux.

 

As most DOS code was written those days, I wrote interesting text mode menu libraries and such other things that assumed that I had access to VGA text buffer to access. Finally when I actually had something going, I decided to make the big switch – Prot to Linux. Almost immediately, I hit a blank wall.

 

The entire style of programming, with direct access to video memory that was used to write large Text-UI based apps on DOS, was simply non-existent in Linux. That, I still remember, was such a major put off that I was going around cursing Linux to anyone who would listen to me. The only alternatives that I was left with, was to use some editors that did old Word-Star like CTRL-K based key combinations, which was rather irritating. I also spent half a day looking at the ncurses lib that time, hoping for a way out. All I got was buggy behavior and one buggy lib after the other and numerous dependencies on some .so or the other which I had to go and find on the net .

 

Finally my entire work on the editor on Linux got scratched and I decided to turn it into a DOS only thing – which I used for much after that as file viewer. The thing was distributed as w.exe to my friends and the default config file used to display multicolor menu and the title “The Zen Masters Workbench”. In w.exe was viewer that could be given C like structures and could read data out of binary files to match the structures. That became the original basis for the implementation of my DDL language.

 

 

 

I don’t know why I typed all that down… anyway here is the little bit of scheme code that implements a for-loop as a macro. I know it is not much, but its good fun.

 

(require (lib "defmacro.ss"))

 

(define-macro for

  (lambda (init  cond  step  code)

    (let ((loop (gensym)))

      `(let ,loop ,init

             (if ,cond

                 (begin

                   ,code

                   (,loop ,step)))))))

 

;see nested for loops

(for ((i 1)) (< i 10) (+ i 1)

     (begin

       (for ((j i)) (< j 10) (+ j 1)

                (begin

                  (display i)

                  (display j)

                  (display " ")))

       (newline)))

 

Of course, written entirely in xemacs. One reason to use xemacs over Scite is that it gives me this really nice syntax driven indentation. The syntax high-lighting, for what I have seen, is pretty corny though. But on the whole, good enough to write scheme code for now.

 

If you are starting out on Scheme and already know how to program, I recommend Dorai Sitaram’s “Teach Yourself Scheme in Fixnum Days” as a starter. The SICP is a lovely text as well and full of ideas, but if you don’t have lots or free time on your hands and are not the sort who likes letting an idea go without giving it its due thought, then I wouldn’t recommend SICP. It will be a while before you get down to scheme-ing. Also Prof Kent Dybvig’s book on Scheme is good, but again if you are the impatient sort, the DS book is the better choice.

 

In a few weeks I should get my copy of the Little Schemer, I intend to be ready for it by then

(The following article was originally written for the .Net Developer Journal where it featured in the Nov 2003 issue. This was co-written by Pooja and me and basically serves as an intoroduction to our DDL language. The original language was co-written by us and two of our friends Rakesh Nandakumar and Tony Sakariya as our final year college project. The DDL in its present .Net avatar is work by Pooja and me. The copyright perios of this article has expired, hence this post. You can find the article hosted on the magazine website here: http://www.sys-con.com/dotnet/article.cfm?id=460. I hope Pooja will be mirroring this article too)

 

Enter the Data Definition Language: a developer perspective

Roshan James & Pooja Malpani

(Authors of the DDL)

 

Greetings, In this short write-up we are going to discuss our new language for the .Net platform called the Data Definition Language (DDL). We hope that by the time you have finished reading this, you will want to log onto the internet, download the DDL and try it for yourself (and drop us mail about it).

 

The DDL is a unique language that aims at solving a common problem. We all would have faced situations where we needed to read data from a file of some given format and there are no real library routines for this purpose, other than the standard file handling libraries. In such cases, we would have to develop code that will calculate the addresses of various values that we want to retrieve from the file and write file handling code that will retrieve the data. While this is probably feasible for simple file types, the need to support a complex file type, or multiple file types or a changing file type can easily become a development bottleneck.

 

Once, we had to work with a development team that was creating an application for analyzing data collected in aircraft black boxes. Data dumps from black boxes have very complex formats. The team needed to read values of various aircraft parameters such as left-aileron-angle, right-engine-exhaust-temperature etc, which were scattered across various bits that seemed to have no respect for word boundaries or any such thing. To add to the fray there was no apparent standard between black box formats of the same manufacturer, version of the same aircraft or any such thing. The team needed to have their product support multiple black box formats without a huge development cycle between these. A little thought made it apparent that there was no real solution to supporting data retrieval from arbitrary file formats. This is the problem that the DDL was designed to solve.

 

How does the DDL understand the format of the file I want to read data from? The DDL is a language. It is designed to be simple and intuitive for expressing data formats. To develop a solution with the DDL a developer needs to first write a DDL script that represents the file format. Once the script is developed, you can run the DDL engine/interpreter on the script, provide the engine with your data file and you are ready to start reading information out of the file.

 

So how can I use the DDL in developing an application? The DDL engine is designed to be an interpreter that be hosted by any .Net application – which means that your VB.Net or C# programs can act as hosts for the DDL. A typical DDL solution would consist of your parent application that does all the UI and has all the business logic. This would host the DDL engine with which you can programmatically interact. The DDL engine simply acts as a substitute for your complex file handling code; it does not dictate what you do with the data that you have retrieved.

 

Hosting the DDL.

 

Once you have hosted the DDL engine, you tell it to do tasks such as ‘load this ddl script file’, to make it understand your file format. Then you tell it to ‘use this data file’ so that it can apply your ddl script (which defines the data format) of the actual data file. Then you tell it ‘give me the value of left-aileron-angle’ and the DDL engine looks at the script, understands where in the data file ‘left-aileron-angle’ would exist and reads out the value for your host application.

 

What are the DDL script files like? The DDL language provides you with primitive types that represent bits – these are available from single bit definitions to 32 bit definitions. These are named i1 i2, i3 etc to i32. Variables can be declared to represent any of these types.

            i16 width

This represents a 16 bit value called width.

 

Now declarations like this should be grouped into DDL structures. Structures would look like this:

 

struct Window

{

i16 width

i16 height
}

 

A DDL script file can contain multiple structures like the one above. Similar to the declaration of primitive bit types you could also define a structure to contain a member of another structure type.

 

struct Bitmap

{

      i8 BitsPerPixel

      Window w

}

 

The DDL script requires that you mark one structure in the script file as the ‘init’ structure. The DDL uses the “init” structure as the first structure of your data file – the init structure is mapped to offset zero of your data file. The init structures and its members (which maybe instances of some of the other structures declared in the script file) are expected to represent the entire data file.

 

The tree-like structure formed by structures in a DDL script file.

 

The DDL structures are different in concept from structures in languages such as C. The first difference is that a DDL structure can contain members depending on conditions. Secondly, a member in a DDL structure simply represents a region in the data file; an i8 type member would represent a byte-sized region. Members in a structure have their offset addresses from the base address of the structure automatically calculated or can have explicit addresses provided. This little script demonstrates both of these.

 

 

 

struct EmployeeData

{

      i32  empNumber

      i1   fPhoneNumberProvided

      i7   fUndefined

      when( fPhoneNumberProvided == 1 )

      {

            i8[10]      phoneNumberString

      };

      @ 0,0 i24 empSerialNumber

      i8 empDesigCode

}

 

This simple script demonstrates some interesting things. An instance of the EmployeeData structure will contain a member called phoneNumberString of 10 bytes, only if the fPhoneNumberProvided flag bit is set. Similarly the notation “@ 0,0” is an explicit address specification that causes the address of the declaration immediately following it to fall at an offset of 0 bits from the start of the structure. Thus, with respect to little-endian Intel architecture, the lower 3 bytes of empNumber are the empSerialNumber and the most significant byte stands for the employee’s designation code.

 

Layout of the structure with respect to the bits of the data file.

 

The script also demonstrates a simple array of i8 (or byte) type of ten elements. Unlike C, the size of an array can be specified via an expression. To illustrate the power of arrays consider the following snippet that uses the above employee structure.

 

struct EmployeeHeader : init

{

      i16 empCount

      EmployeeData [empCount] emps

}

 

The first member decides the number of employees whose data is provided and ‘emps’ represents as array of EmployeeData types. You can now ask the DDL engine for the designation code of the 5^th employee and DDL engine sets about to determine the location of the 5^th employee and retrieve its ‘empDesigCode’ value for you. Now remember that the EmployeeData had a member that would occur conditionally – this means that the size of each EmployeeData instance can be different. The DDL engine internally determines that there is a dependency and checks the flag in each of the preceding EmployeeData instances to determine the actual location of the 5^th instance that you requested.

 

The ddl script provides only a minimal set of programming constructs whose purpose is centered on being able to define data formats. The present ddl language is rich enough to support a wide variety of common file formats. There may however be some format types that may not be expressible in the ddl or are hard to express.

 

The DDL language has constructs for representing address specs, size specs, conditional dependencies, different kinds of array constructs etc. This is a just a brief description of the language, the complete language description document is available from the homepage.

 

Hosting the DDL Interpreter: This is probably the simplest part. Imagine that EmployeeHeader and EmployeeData together formed a ddl file called employee.ddl and that we had a data file in this format called empinfo.bin. The following C# snippet is all you will need to start off using the DDL.

 

using System;

using System.DDL;

 

class DDLTestClass

{

       static void Main()

       {

              //initialise the DDL

              ManagedDDLEngine ddl = new ManagedDDLEngine();

              ddl.LoadSourceFile("employee.ddl");

              ddl.OpenDataFile("empinfo.bin");

              ddl.InterpretData(); //this call is needed to map the scoure to the data

             

              Console.WriteLine("No of employees = {0}",

                     ddl.GetValue("empCount")); // GetValue() read the value of a variable

             

              //contents of emps[0]

              ddl.Seek(".emps:0"); //seek changes path into a member

              Console.WriteLine("Data of 0th Employee \n\t "+

                     "empNumber={0}",

                     ddl.GetValue("empNumber"));

             

              //contents of emps[5]

              ddl.Seek(".emps:5");

              Console.WriteLine("Data of 5th Employee \n\t "+

                     "empNumber={0} \n\t "+

                     "empDesigCode={1}, \n\t "+

                     "empSerialNumber={2}",

                     ddl.GetValue("empNumber"),

                     ddl.GetValue("empDesigCode"),

                     ddl.GetValue("empSerialNumber"));

 

              ddl.Dispose(); //clean up

 

       }     

}

 

This little snippet loads the ddl with the script file and a data file and reads values from it. This snippet does not show any of the error handling code that would be required in a production quality application. One concept you need to be familiar with is that of path.

 

The init structure is represented as “.” (dot). Any member under it is represented using its member name. Any child of that member is separated by a dot, and so on. If there is an array in the path, then the array instance is separate by a “:” (colon).

Ex

init.emp[0] will be represented as “.emps:0”

 

At any point the GetValue() call will return the values on any of the variables in the current path. The Seek() is used to set the current path to another location, subsequent GetValue() calls will read values from that location. Bit values that are read are treated as unsigned integer types. All values are returned as ‘double’ type.

 

A document describing the API exposed by the DDL is available on the homepage for details.

 

What is available? The DDL engine itself is currently for download as System.DDL.dll. This is a mixed mode .net assembly developed in Managed C++ and can be hosted in any .net application. The entire source code of the DDL is also available for download. The DDL is offered for use free of cost and is currently not under any licensing or royalty restrictions. We however expect that in return we get feedback that can help improve the DDL. If the DDL is used for commercial purposes we hope that the authors drop us a note and possibly give credit where credit is due – this would help in popularizing the DDL. You are however not required to do any of these and are free to use the DDL without any acknowledgements at all.

 

Also a console program is available which can be used to test-run DDL scripts that you are developing. This is also available in source form as an example of hosting the DDL. The site also has tutorial material about the DDL console as well as documentation about the language, API, internal algorithms and such.

 

Present and Future: The DDL in its .Net avatar is presently in Beta 1 status and is the work of two people. We believe that the idea of a generally useful DDL system has substance and are hoping to work towards it.

 

For future development we are hoping to strengthen to the DDL language so that the language can be used to express data formats that are currently not expressible or are hard to express. Also plans are underway for a compiler for the DDL language. The compiler is expected to take a .ddl file as input and generate a .net assembly as output that will be code streamlined for your ddl script, rather than be a general purpose ddl interpreter.

 

We are hoping to build a community around the DDL and would like to invite you to join the DDL development project. Inputs for future design aspects, known issues etc would be appreciated.

 

The DDL project homepage is http://ddl.sscli.net .

Contact us: spark at sscli dot net, dolly at sscli dot net.

 

(Download the article and related code sample here

The homepage for the DDL project remains http://ddl.sscli.net

You may also want to visit: http://www.thinkingms.com/pensieve/homepage/work/system.ddl/index.htm

And http://www34.brinkster.com/mpooja/work/Other/ddl/index.htm

A complete description of the language is available at http://www34.brinkster.com/mpooja/work/Other/ddl/html/DDL-Language.htm)

 

Please give us feedback about the language and what you would like to see implemented. One of the plans we have for the future is a compiler for the DDL – this has been postponed because various other projects kept getting in the way.

 

A few days back I found what seemed to be a book about Ruby. This was being discussed on the Ruby mailing list. It’s called “A Little Ruby” or more precisely “A Little Ruby. A Lot of Objects”. You can find it here:

http://web.archive.org/web/20030618203059/visibleworkings.com/little-ruby/

(Someday it will be available here: http://www.visibleworkings.com/little-ruby/ )

 

Instead of writing the whole thing myself or copy paste it, I ask you to simply go read the book. That is my blog entry for the day.

 

The “Little Ruby” book is a conversation between two people where some sublime ideas about the design philosophy of the Ruby language are discussed. The book itself is a pleasure to read and more importantly, to think about. (It is an incomplete book, only 3 chapters – the author Brain Marick said on the Ruby list that he hopes to complete it sometime).

 

Reading “Little Ruby” put in a phrase in my thinking – “Model of Computation”, I don’t know if this sounds sober, but I think this is what I am really looking for.

In all my tinkering around languages, compilers, runtimes and other things – I am looking for a Model of Computation, a fundamental set of programmatic thought abstractions that are beautiful and can encompass various forms of programming.

 

The Little Ruby book talks about a model of computation where all computation is simply built around the idea of passing messages to objects. It is a simple concrete idea with which the rest of the Ruby world is built (apart of syntactic sugar). I don’t know if you are used to thinking in this way – but it is a powerful form of thought.

 

Let me quote from one of the conversation toward the end of the third chapter (the last chapter that is written so far):

 

“A language that provides lots of features

will always be missing that one feature you

need.”

 

“But a language that chooses the right

simple rules for you to combine lets you

build the features you need.”

 

This is the basic idea of composition – small integral units that compose to produce powerful behavioral entities. Have you ever thought why a unix command shell guy never really thought much of a Win/Dos user – because somewhere the way the shell forces you to thinking terms of composition of small do-one-thing-well tools and create powerful meta-tools, is a greater thought pattern.

 

You might have heard this being said about tools in the old unix culture (I say ‘old’ because I have different opinions of ‘unix’ culture as it is now)

 

"This is the Unix philosophy. Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface."

--Doug McIlroy

 

The “Little Ruby” book is inspired by the old book “The Little LISPer”. Something that is now on my reading list – I can’t seem to get a copy of this anywhere. The present edition of the book is called “The Little Schemer”. The book is co written by Prof Daniel P Friedman of Indiana University and Prof Matthias Felleisen of Rice University. The Little Schemer discusses a different model of computation from what the “Little Ruby” describes.

 

I did not know this then, but sometime last year I was in email correspondence with Prof Friedman. That time, had I known that he is author of a respected LISP text book, I might have been frightened off the prospect of asking this -  but in one of the mails I had asked “why Lisp?”

 

Roshan,

 

The most fundamental building block of computation is composition. If the language does not support composition in a trivial way, then I have no use for it.  ML, Haskell, LISP, and Scheme each give a kind of composition.  Composition is the building block of Category Theory, which is a unifying tool that helps clarify much of mathematics. and logic.  So, thinking that it would be okay to use a language that does not support composition is impossible for me.

 

(I quote this here presently without his permission, I believe he would be ok though).

I didn’t understand him then. But now after a year, I think I am closer to understanding him.

 

What would a unified model of computation be? Can such a thing exist? Can we think of all computation using a set of minimal and powerful abstraction such that every other form of computation can be built out of it. Can this be one that is easy and fun to use that we could interact with this force on a day to day basis.

 

And what forms the underlying foundation for computation then might also form the underlying basis for other systems of organized thought as well. This is like the dream of Grand Unified Field Theory in physics. Can something like that exist in the computational systems as well?

 

I don’t know enough to guess. But however I believe that as long we keep pursuing computing in a way that is fun and simple, we are probably on the right track.

 

 

To end this entry I want to quote from the preface of the little ruby:

 

Welcome to my little book. In it, my goal is to teach you a way to think about computation, to show you how far you can take a simple idea: that all computation consists of sending messages to objects. Object-oriented programming is no longer unusual, but taking it to the extreme - making everything an object - is still supported by only a few programming languages.

 

Can I justify this book in practical terms? Will reading it make you a better programmer, even if you never use "call with current continuation" or indulge in "metaclass hackery"? I think it might, but perhaps only if you're the sort of person who would read this sort of book even if it had no practical value.

 

The real reason for reading this book is that the ideas in it are neat. There's an intellectual heritage here, a history of people building idea upon idea. It's an academic heritage, but not in the fussy sense. It's more a joyous heritage of tinkerers, of people buttonholing their friends and saying, "You know, if I take that and think about it like this, look what I can do!"

 

As a closing note, sometime last year I was looking to do research under someone working with the SSCLI code base and work on virtual machines and runtimes. I wanted to do my Masters.

 

At that time the best way I could describe what I wanted to do was to say that I was looking runtimes and virtual machines research with a specific interest in SSCLI. Now, maybe I can describe myself a little better.

 

The only way I could think of doing this that time was to ask around in online forums and mailing lists about universities doing work with Rotor. That accompanied by a barrage of mails to everyone who I thought might know, or point me in the right direction. One name that came up was of Prof Ralf Johnson of UIUC. Right now I was looking for Brian Marick (author of little ruby) on Google, Brian is research student doing his PhD under Prof. Johnson.

 

How many of us recognize the name of this man?

 

Mitch Kapor was the founder of the Lotus corporation. He was the man who designed the Lotus 1-2-3. If you know you history, Lotus was one of the only large applications companies that was a serious challenger for Microsoft in its early years. There were years spent over the battle for the spreadsheet that was fought on both the old Mac as well as the old DOS machines.

 

Microsoft’s offering those days were called Multiplan. Multiplan was fairly beat by Lotus 123 in almost all fronts. Microsoft eventually thought through their faults and strengths and eventually released Excel – the spreadsheet battle was over.

 

Mitch Kapor himself, is one person I think of as being fairly amazing.

 

He was co founder of the EFF, the Electronic Frontier Foundation along with John Perry Barlow. The EFF was the organization that for the first time stood up for hackers rights and digital rights. This was of significant and epic proportions in the early 90s when hacker arrests and crackdowns were gaining a witch-hunt like momentum.

 

“The EFF is a non-profit civil liberties organization working in the public interest to protect privacy, free expression, and access to public resources and information online, as well as to promote responsibility in new media.”

 

The EFF was the organization that for the first time took the American Secret Service to court over the ruling and prosecution of the ‘hacker’ Knight Lightning. The EFF won and it literally brought the end of an era about how people of ‘hackers’ and the rules for information security.

 

I highly recommend reading this book called the Hacker Crackdown by Bruce Sterling. The book reflects the ethos of a time when the parameters of information security were very different from how we think of them now. Considering the license of the book, what it intends to convey and what I hope it may change about your thinking, I would recommend downloading a softcopy of the book.

 

Today I happened to come across Mitchell Kapor’s website and blog.

Website: http://www.kei.com/homepages/mkapor/

Blog: http://blogs.osafoundation.org/mitch/

 

I found this entry, right on top and I couldn’t help smiling:

 

May 09, 2004

Now I'm Mad

 

Some idiot Atkins Diet spammer just posted 53 bogus comments in this blog. I'm disabling comments (globally) shortly and figuring out if there's any recourse.

 

They don't know it yet, but they picked the wrong person to do this to.

 

 

 

Trashbin finally has patch.

 

Before trashbin was written I used to wonder a bit out this mythical entity called metadata that is often talked about in the .Net framework. Metadata was mentioned on almost every discourse on .Net and libraries like reflection libs were based squarely on it. Metadata formed the central pillar of design of the self describing type/component system that is called the .Net framework.

 

However, I could find almost nothing that showed me actual metadata, in its physical form. As a consequence I decided to try and understand what metadata was all about. Thus trashbin was written. Trashbin first saw light of day when announced in a Bangalore User Group post sent in the wee hours of morning.

 

From: spark

Subject: metadata viewer: trashbin v0.1, src+bin release

Date: Mon, 09 Jun 2003 13:28:37 -0700

-----------------------------------------------------------

 

hi group, 

i had been spending some of my odd freetime and sometimes lost sleep into exploring the .net exe/dll format and peeking into metadata glue.

well, the glue is rather interesting to look into and gives you a small insight into where information for things like the class loader and the reflection api get their data. trashbin is small viewer for metadata info that i am releasing with src. this is the first version anywhere and so is expected to be buggy - do mail. if internals interests you, take a look:

http://www.thinkingms.com/pensieve/homepage/work/trashbin/trashbin.htm

cheers :)

rosh

 

Since its release, trashbin has more or less worked fine for me and I have been using it for about a year now.
In essence, this is what trashbin does:

 

>trashbin

Spark (?)  Managed(.Net)/Native PE-COFF file viewer. Version 0.2

May 2003, contact: rosh@mvps.org

Last update: May 2004

 

usage: trashbin [options]

 

        portable executable info:

        /dos     display dos header

        /sig     display the file signature

        /coff    display coff header

        /pe      display pe/optional header

        /dd      display data directories in pe header

        /sec     display section headers

        /exp     display export table

        /imp     display import table

        /reloc   display relocation information

        /tls     display Thread Local Storage information

 

        managed info:

        /corhdr          display the common language runtime header

        /mdhdr           display metadata headers

        /md:Strings      display metadata stream #Strings

        /md:Blob         display metadata stream #Blob

        /md:US           display metadata stream #US (user strings)

        /md:GUID         display metadata stream #GUID

        /md:#~           display optimised metadata tables stream-header

        /mdtab           display optimised metadata tables

 

        other:

        /type    indicates the type of the PE file

        /csv     enable excel compatible, CSV output

 

        ps. The name trashbin is 'inspired' from dumpbin :)

 

Since most people who are reading this entry might be interested in what metadata is and what the PE file format is like, here goes:

 

The PE file format is Microsoft’s Portable Executable File Format. Essentially most exes and dlls that you will see on a windows system have this file format. Yes Exes and DLLs have the same format. The difference largely lies in the fact that a DLL file does not necessarily have an entry point defined. Here is a little about the Exe/Dll format:

 

Once upon a time, there used to be old DOS exes that came with what was the DOS exe header. Microsoft retained the DOS exe header in all subsequent exe formats so that the executables would be compatible across their operating systems. Which is why, you can run any windows or .Net exe on any Microsoft operating system (including dos) and see it run. Of course these programs would not do anything in the dos environment other than display a message saying that the exe would run under windows. The point however is that the exe did validly execute on a 15 or twenty year old system that was built for processors that did not have a concept of memory beyond 1Mb.

 

The DOS header is known for it special signature bytes MZ. Open any exe file and notice that the very first two bytes are MZ. MZ stands for Mark Zbikowski, the person who developed the DOS exe file format. Prior to that the executable format was called the Com file format. Those of you who have had the chance to work on DOS would not have forgotten what a pleasure some of those COM files used to be. The COM format belonged to the then popular CP/M operating system of Digital of the great Gary Kildall. Gary Kildall was pioneer in a way few people were.. anyway that is story for later.

 

The following is a dump of the initial few bytes of an exe file showing the then new MZ DOS header:

 

>hexv HelloWorld.exe

 

0000:0000│ 4D 5A 90 00 03 00 00 00 │ 04 00 00 00 FF FF 00 00 │ MZÉ▒♥▒▒▒│♦▒▒▒  ▒▒

0000:0010│ B8 00 00 00 00 00 00 00 │ 40 00 00 00 00 00 00 00 │ ╕▒▒▒▒▒▒▒│@▒▒▒▒▒▒▒

0000:0020│ 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00 │ ▒▒▒▒▒▒▒▒│▒▒▒▒▒▒▒▒

 

In a sense Mr Zbikowsky has the most popular initials on the planet. The DOS exe header itself is available as a structure that is defined in the winnt.h header file that is available on almost every windows based c/cpp dev environment.

 

Now the DOS exe header did not suffice to hold a lot of the new information that the exe had to present to the operating system, when windows came along. So new structures were introduced which were the akin to the old unix based Common Object File Format (COFF). There is plenty of literature available about this on the net.

 

The PE file is denoted by the signature bytes “PE  ". If you download trashbin the source code has some embedded urls that give you information about the PE file format itself. Those may prove valuable for your understanding of the actual exe file format.

 

Just to connect all that I have been talking about to trashbin and how you can actually examine an exe file with it, these are the relevant switches.

 

        /dos     display dos header

        /sig     display the file signature

        /coff    display coff header

        /pe      display pe/optional header

 

Now that we have covered that ground, lets move on. The PE file has a data structure called the Data Directory which is displayed through the  /dd option.

 

        /dd      display data directories in pe header

 

The data directory basically contains pointers to various data structures inside the PE file. The DD has 16 entries and a dump of the DD looks like this:

 

C:\WINNT\system32>trashbin tracert.exe /dd

_IMAGE_DATA_DIRECTORY

0       VirtualAddress = 0

        Size = 0

1       VirtualAddress = 0x19ac

        Size = 0x78

2       VirtualAddress = 0x3000

        Size = 0x11b8

3       VirtualAddress = 0

        Size = 0

4       VirtualAddress = 0

        Size = 0

5       VirtualAddress = 0

        Size = 0

6       VirtualAddress = 0x1090

        Size = 0x1c

7       VirtualAddress = 0

        Size = 0

8       VirtualAddress = 0

        Size = 0

9       VirtualAddress = 0

        Size = 0

10      VirtualAddress = 0

        Size = 0

11      VirtualAddress = 0x240

        Size = 0x7c

12      VirtualAddress = 0x1000

        Size = 0x88

13      VirtualAddress = 0

        Size = 0

14      VirtualAddress = 0

        Size = 0

15      VirtualAddress = 0

        Size = 0

 

 

This is trashbin examining tracert program that comes with windows. The tracert program is a native exe (not a .net exe). Entries in the DD have a predefined meaning, these are defined in winnt.h as follows:

 

#define IMAGE_DIRECTORY_ENTRY_EXPORT          0   // Export Directory

#define IMAGE_DIRECTORY_ENTRY_IMPORT          1   // Import Directory

#define IMAGE_DIRECTORY_ENTRY_RESOURCE        2   // Resource Directory

#define IMAGE_DIRECTORY_ENTRY_EXCEPTION       3   // Exception Directory

#define IMAGE_DIRECTORY_ENTRY_SECURITY        4   // Security Directory

#define IMAGE_DIRECTORY_ENTRY_BASERELOC       5   // Base Relocation Table

#define IMAGE_DIRECTORY_ENTRY_DEBUG           6   // Debug Directory

//      IMAGE_DIRECTORY_ENTRY_COPYRIGHT       7   // (X86 usage)

#define IMAGE_DIRECTORY_ENTRY_ARCHITECTURE    7   // Architecture Specific Data

#define IMAGE_DIRECTORY_ENTRY_GLOBALPTR       8   // RVA of GP

#define IMAGE_DIRECTORY_ENTRY_TLS             9   // TLS Directory

#define IMAGE_DIRECTORY_ENTRY_LOAD_CONFIG    10   // Load Configuration Directory

#define IMAGE_DIRECTORY_ENTRY_BOUND_IMPORT   11   // Bound Import Directory in headers

#define IMAGE_DIRECTORY_ENTRY_IAT            12   // Import Address Table

#define IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT   13   // Delay Load Import Descriptors

#define IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR 14   // COM Runtime descriptor

 

You can compare these against what is present in the tracert program dump.

Now lets do a DD dump of a .net exe

 

>trashbin HelloWorld.exe /dd

_IMAGE_DATA_DIRECTORY

0       VirtualAddress = 0

        Size = 0

1       VirtualAddress = 0x2370

        Size = 0x4b

2       VirtualAddress = 0x4000

        Size = 0x340

3       VirtualAddress = 0

        Size = 0

4       VirtualAddress = 0

        Size = 0

5       VirtualAddress = 0x6000

        Size = 0xc

6       VirtualAddress = 0

        Size = 0

7       VirtualAddress = 0

        Size = 0

8       VirtualAddress = 0

        Size = 0

9       VirtualAddress = 0

        Size = 0

10      VirtualAddress = 0

        Size = 0

11      VirtualAddress = 0

        Size = 0

12      VirtualAddress = 0x2000

        Size = 0x8

13      VirtualAddress = 0

        Size = 0

14      VirtualAddress = 0x2008

        Size = 0x48

15      VirtualAddress = 0

        Size = 0

 

The interesting thing to note is that in a managed exe, the 15^th entry is non zero. The 14^th entry point to the Common Runtime Header or the CorHdr. The CorHdr structure is defined in corhdr.h.

 

The CorHdr is where (so to speak) the managed world starts. So .Net exe files are regular PE files which have all the .Net specific content in a particular offset in the exe file. The idea was that .Net was designed to be platform neutral. So Microsoft could assume that the facilities provided by the exe file format would be available in file formats of other operating systems where .Net would one day run. For this sake .net specific content assumes that the whatever the parent file format that is enclosing it, the .Net specific content can be made laid out in a large well defined binary blob.

 

A description of the CorHdr, Metadata layout and other intricacies of the underlying system can be found described (in varying detail) in the ECMA specification of the Common Language Infrastructure. Partition 2 that describes metadata is the one that you would want to look it, in this regard.

http://msdn.microsoft.com/net/ecma/

 

Another interesting resource for the technical mind is Serge Lidin’s book “Inside the .Net IL Assembler” . The book is available as an Indian India edition also, so it is affordable.

 

Trashbin lets you view this part of the managed exe file.

These are the relevant switches:

 

        /corhdr          display the common language runtime header

        /mdhdr           display metadata headers

        /md:Strings      display metadata stream #Strings

        /md:Blob         display metadata stream #Blob

        /md:US           display metadata stream #US (user strings)

        /md:GUID         display metadata stream #GUID

        /md:#~           display optimised metadata tables stream-header

        /mdtab           display optimised metadata tables

 

Trashbin is not a dissembler – it simply lets you view the other data that is present in the exe/dll file. I figured I don’t need to do a dissembler because ILdasm and several other tools do the job so well. ILdasm btw is written by Serge Lidin.

 

The metadata in the managed file is divided into a set of streams. A stream is like an area of memory reserved for content of a specific type. These are namely –

#Strings

#US

#Blob

#GUID

#~ and #-

 

The #Strings stream keeps all the strings in the application that include things like names of classes, methods, parameters, namespaces, assemblies etc. So this stream basically contains all kind of strings that are part of the source code itself – these are where the names that are used by the reflection library are available.

 

The #US stream is the one for user defined strings. So when you say Console.WriteLine(“Hello World”) in your program, the “Hello World” goes into #US and the Console.WriteLine goes into #Strings. The strings present in the #US set are Unicode strings – so each character is two bytes.

 

Here are some nice and friendly hex dumps of these regions from a exe file followed by the corresponding information being ripped by trashbin:

 

#Strings

 

0000:0460│ 00 00 00 00 00 00 00 00 │ 00 3C 4D 6F 64 75 6C 65 │ ▒▒▒▒▒▒▒▒│▒

0000:0470│ 3E 00 48 65 6C 6C 6F 57 │ 6F 72 6C 64 2E 65 78 65 │ >▒HelloW│orld.exe

0000:0480│ 00 6D 73 63 6F 72 6C 69 │ 62 00 53 79 73 74 65 6D │ ▒mscorli│b▒System

0000:0490│ 00 4F 62 6A 65 63 74 00 │ 43 61 6C 63 00 48 65 6C │ ▒Object▒│Calc▒Hel

0000:04A0│ 6C 6F 57 6F 72 6C 64 53 │ 61 6D 70 6C 65 00 48 65 │ loWorldS│ample▒He

0000:04B0│ 6C 6C 6F 57 6F 72 6C 64 │ 00 41 64 64 00 2E 63 74 │ lloWorld│▒Add▒.ct

0000:04C0│ 6F 72 00 4D 61 69 6E 00 │ 53 79 73 74 65 6D 2E 44 │ or▒Main▒│System.D

0000:04D0│ 69 61 67 6E 6F 73 74 69 │ 63 73 00 44 65 62 75 67 │ iagnosti│cs▒Debug

0000:04E0│ 67 61 62 6C 65 41 74 74 │ 72 69 62 75 74 65 00 61 │ gableAtt│ribute▒a

0000:04F0│ 00 62 00 61 72 67 73 00 │ 43 6F 6E 73 6F 6C 65 00 │ ▒b▒args▒│Console▒

0000:0500│ 57 72 69 74 65 4C 69 6E │ 65 00 45 78 63 65 70 74 │ WriteLin│e▒Except

0000:0510│ 69 6F 6E 00 67 65 74 5F │ 4D 65 73 73 61 67 65 00 │ ion▒get_│Message▒

 

>trashbin HelloWorld.exe /md:Strings

METADATA STREAM #Strings

        Offset : "String"

        0x1    : ""

        0xA    : "HelloWorld.exe"

        0x19   : "mscorlib"

        0x22   : "System"

        0x29   : "Object"

        0x30   : "Calc"

        0x35   : "HelloWorldSample"

        0x46   : "HelloWorld"

        0x51   : "Add"

        0x55   : ".ctor"

        0x5B   : "Main"

        0x60   : "System.Diagnostics"

        0x73   : "DebuggableAttribute"

        0x87   : "a"

        0x89   : "b"

        0x8B   : "args"

        0x90   : "Console"

        0x98   : "WriteLine"

        0xA2   : "Exception"

        0xAC   : "get_Message"

 

#US

 

0000:04B0│ 65 00 00 00 00 17 48 00 │ 65 00 6C 00 6C 00 6F 00 │ e▒▒▒▒↨H▒│e▒l▒l▒o▒

0000:04C0│ 20 00 57 00 6F 00 72 00 │ 6C 00 64 00 00 00 00 00 │  ▒W▒o▒r▒│l▒d▒▒▒▒▒

 

>trashbin test.exe /md:US

METADATA STREAM #US

0x1, (23 bytes)

    Txt: H.e.l.l.o...W.o.r.l.d..

    Hex: 48 00 65 00 6c 00 6c 00 6f 00 20 00 57 00 6f 00 72 00 6c 00 64 00 00

 

I will just skip over the #GUID and #Blob streams for now. The #~ stream is the interesting one that is the stream that actually contains the metadata tables. These is an alternate stream thaty can be present which is the #- stream. The #- stream again contains metadata tables but these are called the un-optimized tables because certain sort orders are not maintained in these tables. The Microsoft compiles always emit optimized tables and since I have not been using any other compilers (Mono too seems to emit optimized tables) I don’t have support for #- in trashbin.

 

Lets focus on #~. The #~ is the real metadata, if you would like to think of it that way, It is actually a small relational database that is compressed down to the last bit. There are a large number of tables (with predefined schemas) that can occur here. These tables provide information about the exe or dll (or precisely the assembly) that they are trying to describe.

 

These tables cross reference each other as well as reference entries in the other streams, such as the #GUID and #Strings streams. The trashbin option /md:#~ gives you the header of #~ stream, which is a kind of summary view of what the stream contains:

 

>trashbin HelloWorld.exe /md:#~

METADATA STREAM #~

        TABLES HEADER

                MajorVersion = 1

                MinorVersion = 0

                HeapSizes = 0

                        #String Index = 2 bytes wide

                        #GUID Index = 2 bytes wide

                        #Blob Index = 2 bytes wide

                Valid  = 0x00000900021547

                Sorted = 0x0002003301fa00

 

        METADATA Tables

                RID.               TableName    [No of Rows]

                0.                    Module    [1]     Row=10 bytes

                1.                   TypeRef    [4]     Row=6 bytes

                2.                   TypeDef    [3]     Row=14 bytes

                6.                    Method    [4]     Row=14 bytes

                8.                     Param    [3]     Row=6 bytes

                10.                MemberRef    [5]     Row=6 bytes

                12.          CustomAttribute    [1]     Row=6 bytes

                17.            StandAloneSig    [2]     Row=2 bytes

                32.                 Assembly    [1]     Row=22 bytes

                35.              AssemblyRef    [1]     Row=20 bytes

        Table Count = 10

 

The above listing says that helloworld.exe contains 10 tables in its metadata headers.

To give you a taste of what I am talking about lets look at some of these tables:

 

To look at the metadata tables, use the option

 

        /mdtab           display optimised metadata tables

 

This is the table called TypeDef that has a listing of all the types defined in this assembly.

 

[RID=2] Table TypeDef

     [  DATA]Flags    [STRING]Name     [STRING]Namespace [CI:64 ]Extends  [RID: 4]FieldList [RID: 6]MethodList

   1.[  0x  ]0        [  0x  ]1        [  0x  ]0         [RID: 2] 0       [RID: 4] 1        [RID: 6] 1        

   2.[  0x  ]100001   [  0x  ]30       [  0x  ]35        [RID: 1] 1       [RID: 4] 1        [RID: 6] 1        

   3.[  0x  ]100001   [  0x  ]46       [  0x  ]35        [RID: 1] 1       [RID: 4] 1        [RID: 6] 3        

 

The table has a field called name – which is the name of the type. The value of the names field are actually offsets into the #Strings stream. So if you care to compare (I have provided a listing of the #Strings stream earlier), what the table is saying is that this assembly has 3 types which are namely , Calc and HelloWorld. At this point let me drop the source code of the HelloWorld.cs file from which this assembly was compiled:

 

//HelloWorld.cs

namespace HelloWorldSample

{

        using System;

       

        public class Calc

        {

                public int Add(int a, int b)

                {

                        return a+b;

                }

        }

       

        public class HelloWorld

        {

                public static void Main(string[] args)

                {

                        try

                        {

                                Calc c = new Calc();

                                Console.WriteLine(c.Add(10,20));

                        }

                        catch(Exception e)

                        {

                                Console.WriteLine(e.Message);

                        }

                }

        }

}

 

As you can see the source does define two classes called Calc and HelloWorld. A type in .Net terms is a very strict entity that encompasses  value types and reference types and type entities like classes, structures, enums etc.

 

Here is a table that shows me what types are being referenced in this assembly:

 

[RID=1] Table TypeRef

     [CI:75 ]ResolutionScope [STRING]Name     [STRING]Namespace

   1.[RID:35] 1              [  0x  ]29       [  0x  ]22       

   2.[RID:35] 1              [  0x  ]73       [  0x  ]60       

   3.[RID:35] 1              [  0x  ]90       [  0x  ]22       

   4.[RID:35] 1              [  0x  ]A2       [  0x  ]22       

 

The types are Console, Object, Exception etc

 

Here is a list of all the methods that are defined within this assembly;

 

[RID=6] Table Method

     [  DATA]RVA      [  DATA]ImplFlags [  DATA]Flags    [STRING]Name     [  BLOB]Signature [RID: 8]ParamList

   1.[  0x  ]2050     [  0x  ]0         [  0x  ]86       [  0x  ]51       [  0x  ]A         [RID: 8] 1       

   2.[  0x  ]2064     [  0x  ]0         [  0x  ]1886     [  0x  ]55       [  0x  ]10        [RID: 8] 3       

   3.[  0x  ]2078     [  0x  ]0         [  0x  ]96       [  0x  ]5B       [  0x  ]14        [RID: 8] 3       

   4.[  0x  ]20C8     [  0x  ]0         [  0x  ]1886     [  0x  ]55       [  0x  ]10        [RID: 8] 4       

 

You can see the method name add, Main etc here, as well as auto-generated methods such as .ctor (the constructor). 

 

In short if you examine the metadata headers carefully you get an idea about the extent of meta information being stored about your program and what can be supported by any reflection based API. A complete listing of the metadata tables are provided in the ECMA specs, so that’s your best reference, Mr Lidin’s book also does a good job of this and I used both of these while I was writing trashbin.

 

So there – I have said what I wanted to say about trashbin. This might be a good starting point to explore the exe and dll file formats.

 

Before I stop, I must add, do take a look at the switches

        /exp     display export table

        /imp     display import table

 

these show the methods that are exported from an exe/dll (so that it maybe dynamically loaded and invoked from another process) and shows what method calls it invokes. This is kind of the ‘metadata’ of the old unmanaged world.

 

Typing trashbin ntdll.dll /exp in the system32 folder will give you a listing of the native NT api, which might make for good reading again (if that sort of stuff gets your interest).

 

Most of the content of this blog entry was presented at the Bangalore .Net User group as one of the UG meetings talks.

 

 

 

All that said, here is the real reason I started out on this blog entry: Recently Kaushik Srenevasan pointed out a bug in trashbin’s #Strings parsing routine. Kaushik is a ‘MS Student Ambassador’ and has a blog here: http://dotnetjunkies.com/WebLog/kaushik/

 

I had, for some reason, assumed the presence of padding 0s when I had written the original code. That more or less always worked, because the #Strings stream was expected to be padded with zeros till is a 4 byte boundary.

 

Its has been patched according to Kaushik’s suggestion, so Thank you Kaushik.

 

 

 

Trashbin can be downloaded from its homepage here:

http://www.thinkingms.com/pensieve/homepage/work/trashbin/trashbin.htm

 

 

As foot note I must say that the Microsoft Dumpbin program that ships with Visual Studio does most of the things that trashbin does and a few additional things. Dumpbin however does not display metadata information from managed PE files yet.

 

 

I thought of putting together some of my old mails and user group posting as blog entries, with some patches so that they are’ blogopatible’. They would have ended up on my blog, if I had a blog when I made these posts. I feel some of these are of lasting importance, at least with respect to the impressions they had on me.

 

All of these are personal opinions, probably more relevant in the context that they were originally written.

 

-----Original Message-----
From: James, Roshan
Sent: Tuesday, December 09, 2003 11:57 PM
 Subject: XAML and markups: Introducing WFML - WinForms Markup Language

Hi Folk,

 

Ever since Linux Bangalore I have been thinking about something that Miguel de Icaza had demoed on stage. He was showing Mono’s UI, Glade, and showed how Glade generates UI from XML markup. They (Miguel and Nat) were also joking as to how XAML is an idea that they had thought of 6 years back – the idea of using XML markup for representing UI and separating 'business logic' from UI.

 

            He also went on to show how Glade is used to generate GTK# UI in linux. A '.glade' file is simply an xml file that contains tags that correspond to the properties of the controls (or widgets as they call them).

 

(I pulled glade file off the net to give you an idea of what it looks like

 

< widget class="GtkWindow" id="window2">

  < property name="visible">True< /property>

  < property name="title" translatable="yes">window2< /property>

  < property name="type">GTK_WINDOW_TOPLEVEL< /property>

  < property name="window_position">GTK_WIN_POS_NONE< /property>

  < property name="modal">False< /property>

  < property name="resizable">True< /property>

  < property name="destroy_with_parent">False< /property>

 

  < child>

    < widget class="GtkButton" id="button1">

      < property name="border_width">10< /property>

      < property name="visible">True< /property>

      < property name="can_focus">True< /property>

      < property name="label" translatable="yes">button1< /property>

      < property name="use_underline">True

      < property name="relief">GTK_RELIEF_NORMAL< /property>

    < /widget>

  < /child>

< /widget>

 

)

 

The strange thing was how they used glade files - in XAML you compile the xml file to generate a class (or what MS calls a partial class, one that can be extended elsewhere). In glade they did no such thing, they simple wrote a cs file and called one Glade API function passing it the name of '.glade' file. That returned some kind of object and presto they had their UI. Now the claim was that this really separates the UI from the code and as a matter of fact the XML can be chnaged after the exe file has been compiled. Changes to the XML will reflect in the UI without recompilation of anything. They also mentioned that Microsoft hasn’t figured out how to do this yet.

 

(Here is fragment of c# code that uses the .glade

 

                using Gtk;

                using Gnome;

                using Glade;

                using GtkSharp;

 

        public class GladeTest

                {

                                /* If you want to access the glade objects you have to "import" them.

                                 * This is not required, but else you can only work with the

                                 * pre-defined signal handlers */

                                [Glade.Widget]     

                                Button button1;

 

                                public GladeTest (string[] args)

                                {

                                                Application.Init();

 

                                                /* This loads the glade file glade.glade,

                                                 * selects window2 and connects it to the current object,

                                                 * which is the class GladeTest here. */

                                                Glade.XML gxml = new Glade.XML ("file.glade", "window2", null);

                                                gxml.Autoconnect (this);

 

                                                button1.BorderWidth=10;

 

                                                Application.Run();

                                }

)

 

            Now this got me thinking about how they implemented this dynamic behavior - as result of which WFML was born. It was rather surprising how easy this was to do. I am sure Miggy's code will be better and more optimized, but I think my general direction is correct.

 

WFML, in short provides (or at least hopes to provide) a markup language for UI that simply does not have to compiled (in the conventional sense). WFML UI is actually WinForms based UI, WFML by itself cannot add any features that are not present in WinForms. What it does is that it lets you write very clean looking code that does not have a clutter for UI itself. The entire UI of the program is expressed in an external XML file, called a .wfml file. Now if you have ever tried writing a Win forms application by hand without studio, you will remember how hard it is to remember so many of the house-keeping things required - take a look at some WFML based code:

 

//win.cs

using System;

using System.Windows.Forms;

using System.Windows.Forms.Markup;

 

 

class CMain

{

        static void Main()

        {

                IAttachable win = new Wfml().CreateUserInterface("win.wfml","MainWindow");

                Application.Run((Form)win);

        }

}

 

Simple ? This is a C# file that uses the WFML library. The markup lib being a pure .Net assembly can be used by any .Net language, - C#, VB, C++, Jscript, VJ# etc. Now what you need is a WFML file that specifies the UI. And you simply write the .wfml file like this (win.wfml):

 

 

        < Window

                Name="MainWindow"

                Height="200"

                Width="400"

        />

 

As you can see, the WFML is simply an XML file with certain tags that the WFML library understands. The advantage of using something like WFML is that you can completely change this XML at will and the UI of your program can be completely changed.

 

Now compile the cs:

   

csc /t:winexe /r:System.Windows.Forms.Markup.dll win.cs

 

Run win.exe and you will see an empty window

 

The following .wfml code displays a button and a text box for the same win.exe program. The program need not be edited or recompiled, only the .wfml file needs to be changed.

 

< Window

        Name="MainWindow"

        Height="200"

        Width="400"

       

        Text="Main Window"

>

        < Button Name="mybutton"

                Top="10"

                Text="Click Me"

        />

        < TextBox

                Name="tbox"

                Top="40"

                Left="5"

                Width="200"

        />

 

(What tags and attributes are allowed within a WFML? Every attribute that you set in a tag has to be a property of the corresponding WinForms type. That is to say that if you want to put a attribute called ‘Text’ in the ‘Window’ tag, then the class System.Windows.Form.Window should support a property called Text. WFML being a very simple system is case sensitive and provides no error feedback if something is wrong.)

 

And you can even attach event handlers and write back to the UI:

 

using System.Windows.Forms;

using System.Windows.Forms.Markup;

using System;

 

class CMain

{

        public TextBox tbox;

         

        public void clicked(object sender, EventArgs e)

        {

                tbox.Text+="+ ";

        }

        

        static void Main()

        {

                IAttachable win = new Wfml().CreateUserInterface("win.wfml","MainWindow");

                CMain cm = new CMain();

                win.AttachConsumer(cm);

               

                Application.Run((Form)win);

        }

}

 

Do try some WFML, the supported types are Button, TextBox and Label - but I guess the rest can be added easily.  Now I don't think this was anywhere as hard as they were making it out to be on stage.

 

Roshan James

------------------------------

 

 

You can download the WFML.zip file here. This also contains the demos that I had for this UG post.  While I never did continue on the WFML, folk did mail me back asking about the WFML assuming it was part of a larger UI framework.

 

Of course, capital negatives of the approach are that there is no IDE to ease development of WFML and such, but the idea itself is rather interesting.

 

How WFML actually works, as you might have guess is that it uses reflection API to dynamically generate a System.Windows.Form type that is customized according to the WFML file. This runtime generated type also runs reflection based code on its caller and appropriately latches up event handlers dynamically. Also it uses reflection to assign references of the caller to classes to the actual objects created in the generated Form class.

 

You however need not compare XAML to WFML or like approaches because XAML is a whole different beast. Differences start with be67ing vectored based don’t stop any time soon.

I thought of putting together some of my old mails and user group posting as blog entries, with some patches so that they are’ blogopatible’. They would have ended up on my blog, if I had a blog when I made these posts. I feel some of these are of lasting importance, at least with respect to the impressions they had on me.

 

All of these are personal opinions, probably more relevant in the context that they were originally written.

 

-----Original Message-----

From: James, Roshan

Sent: Friday, December 05, 2003 5:37 PM

To: MVP Mailing List

Subject: An Audience with Miguel

 

 

written in a hurry:

 

An Audience with Miguel

 

Hi, yesterday Pooja and I got to catch a part of Linux Bangalore, the annual Linux convention, and I thought that it would be nice to write to mvplist and share our experiences. We had missed the dates for the event and I was rather shocked at having missed a chance to see Miguel De Icaza in person.

 

Miguel, for those who don't know about him, is the creator of mc (the Midnight commander), Gnome (the rather popular open source desktop) and in recent days the lead for the Mono project. The Mono project is the only other major (when I say major here, it is not that I don’t know of dotGNU and other attempts, it is because I personally feel that Mono is more complete than those) .Net implementation outside of the Microsoft world. Mono runs on Windows as well as on Linux and probably other Unix flavors. This guy has been famous/notorious in the open source community for writing papers like 'Lets make Unix not suck' and has a rather 'misfit' personality for the typical religious ramblings of the free software types. In short this man has written his own windowing systems, his own .Net, his own enterprise servers and was running his own company called Ximian.

 

Ximian has been recently bought by Novell making Novell a major player in the open source world. Novell has also bought Suse - major Linux flavour and Novell seems to be on the path of becoming a significantly important open source company, along the lines of maybe Red Hat and such. However unlike Redhat and many of the 'Linux companies', Novell has a focus on delivering products rather than making Linux distributions and delivering them dirt cheap or free if they have to, to get into the market. So in short Miguel is as much a demi-god as our own Anders Hejlsberg, or Don Box.

 

The above paragraph was to set things in perspective. This is the mono website (http://www.go-mono.com/). Having been rather disappointed to have missed Linux Bangalore dates and missed Miguel, I happened to check their talk schedules by sheer accident around 11.30 yesterday. As fate would have it, there was a talk by Miguel, his last one, scheduled at 12.00. After a ~12km drive and some conversation with the registration counter we were at the IISC Bangalore venue – the last time I was here, I was attending the Microsoft Tech Ed.

 

One thing you notice up front as you enter is a big banner of Abdul Kalam, our president. The poster quotes him saying that it is probably not good to have important national software depending on proprietary solutions as proprietary solutions and highly dependant on the market that the vendors cater to and that sort of unreliability is probably not a good thing. And also that free software would really help a poorer country like ours as prices of commercial software are rather high. At least that was the message in spirit - don't think I have got any of his words right.

 

The crowd, as far as I could say was probably the same caliber as the technical crowd I usually get to interact with at UGs and various .Net technology events. Probably not as good in some respects - but there was this thing in the air that they were all up to 'something important'. Also one other thing that was noticeable was the set of demo computers setup, where people could sit down and try out many of the software that was being talked about in the talks.

 

Miguel's talk was at the main hall of the IISC venue (those who know the place will know what I mean). He was accompanied by Nat Friedman of Novell, a fellow mono-ist. What happened at the talk was something I wasn't prepared for. For those of you who have a mental image of Miguel by now, this guy is young, in his early twenties. He was carrying a digital SLR camera with a hefty lens and flash addons and was dressed in baggy jeans and black t-shirt. So was the other guy Nat. Now these guys hop on stage (literally), sit down on the floor - one of them rolls out a length of cable that he had wrapped around his neck the whole while. They pull out two laptops and they get a network setup between their systems and the presentation starts up.

 

The presentation showed of some parts of C# , web services, GTK# for windowing, GTK + for generating XML markup for the UI (and some jokes about how XAML is a copy of their own 6 year old idea) and more. These guys did the whole presentation sitting or lying down on the floor of the stage, sometimes editing the presentation right there in front of the audience, pulling jokes on each other and writing code the whole time for a full hour - and the whole thing was on their own Mono. Awesome.

 

After the talk Pooja and I met up with Miguel and introduced ourselves as being from the ‘dark side’ (He asked me if I was an ASP.Net developer because of all the questions I was asking from what otherwise seemed to be a relatively .Net ignorant audience. He was interested in the fact that I was an MVP.) and asked him if we could meet him sometime later, maybe over dinner or so. He was fine with that – that however was not to happen as his schedule did not allow him to. Various activities were planned for him for the whole of the next day (today, Friday) and he was going to flying on Saturday. He did promise to talk to our .Net user group the next time he is in India (expected to be in the second quarter of next year). I had to get back to office so I had to leave then; with the intention of returning for Jani's talk a little later in the evening.

 

I managed to sneak out of office again to attend the talk by Mr. Janakiram (I hope you know him - he heads Microsoft India's academic/university relationship program). Along with Jani was Mr. Gaurav Daga (he is a Program Manager of the famous Services for Unix team at Hyderabad; SFU won last years best open source software of the year award). Their venue didn't do any justice to their talk. Their talk was scheduled at one of the smaller halls and the crowd was so packed for the talk that I couldn't get close to the door of the hall. Jani and Gaurav as usual pulled a great show. They were talking about the new 'Unix' being built in windows ;) (Their demo was rather awesome: they took a Unix app ran it on windows literally, they wrote .Net code and exposed it as a web service, they built a proxy around the service and consumed the proxy as a COM component which was used by excel which was used to dump – did I miss anything?)  

 

All the while, from when he finished his talk, Miguel and his gang were there in the main lobby, showing off code or sitting or taking photographs or ready to talk to anyone at all about anything. When he was not doing that he would be sitting around in some corner with Nat typically typing away at a piece of code he was working on.

 

Since I couldn't get to listen to Jani at all due to the crowd and having driven the ~12km stretch in Bangalore traffic for the third time that day, I was walking around looking rather moody. Miguel then walks up to me, 'Microsoft dude', and gets our picture taken together. He kept calling me 'Microsoft dude' and he himself wanted to be called 'The Dude'; it was interesting talking to him.

 

I wanted to introduce Jani and Gaurav to Miguel and Nat after their talk. It was funny because on meeting Jani and Gaurav, Miguel seemed to freeze up a little bit - I think it's that Microsoft-effect. But on the whole they were nice folk. And Jani and Gaurav were good too. Jani mentioned how he wrote a wrapper around the 'Tk' widget library and called it a windows forms assembly and got to run some regular winforms code on Linux’

 

In retrospect, if I wanted to pick faults with the event I could and I could say that the organizing was bad, because the projector was shaky and the stage wasn't setup properly and the food was bad and all that and be complacent about the whole thing. But looking at the good side, you see guys who are probably the gods of their community actually sitting around with the developers showing off their code and laughing and talking rather that running off after a talk or sitting in a separate area. These are probably something's that we could learn from these folk. The number of people who Miguel touched that day and the number of people who will remember him are much more that those who will remember any of the speakers at technology forums where I have had the opportunity to speak or attend.

 

Miguel was different from my vision of the open source advocate. Probably because he was less of an advocate and more of a real programmer.

 

Miguel said that no they don't say that they are going to beat Microsoft or that Microsoft is going away or anything - which is normally common talk for OSI folk I have previously met. There is place for both he said. He said he has friends at Microsoft. He said that Dave Stutz is a good friend. We talked a little about Stallman and Free software. I asked him if he would be joining MS like Don Box offered and he said probably not. He would rather be doing his thing like this and be 'helping out the poorer countries'. He says they will keep on writing software, take good ideas wherever they find them and give it out as dirt cheap or free. There was this time when he said that 'they took out the GC and put in a toy GC in rotor and took out the JIT and put in a toy JIT in rotor - but the GC in Mono is your GC as much as its my GC' and he said that to Jani - 'its your as much as its mine and I would like to keep doing that'. There is this certain element of real sincerity which I find so missing in my work place and often at our technical seminars.

 

Free wheeling aside and sorry for all the typos and bad language in my writing, one thing that I probably miss from speakers and from many of our communities is that people hardly seem to be doing all that for themselves as much as for the community - I haven't seen anyone sit down and give all their time and energy to the community they are trying to foster. I don't see anyone on our side actually be there with the people and spread that sense of what they are doing - most of the audiences at our talks see us as speakers on podiums, rarely as people, we don't usually give them the room for that. I wish we could do that. And I wish that when we work on our user groups we can do it for the sense of community, rather than for the sakes of meeting numbers and budgets and revenue targets and stuff. I wish we all do this because we like our technology first - the Microsoft communities have never been able to do that the way these guys have. (This was probably rant, but reading this months later I feel that a lot of human touch is still missing in the communities. Somewhere the communities are built around a carrot culture and people who do things for carrots. Miguel has his carrots, but the way I saw his carrots were from a kind of passion that I personally feel in speakers I have known, myself being equally at fault. Something about what happened there that day felt like the spirit of those things I have thought about so much – hackerdom, the free hackers, the hacker ethic, mentor’s manifesto. Something is missing here and I am sure we can fix it because we have some extremely smart people who are so passionate about their technology)

 

I hope I have not tipped off any one by writing all this; this is probably something that could use some thought.

 

Cheers

Roshan

 

(Left to Right: Natt Friedman (Ximian/Novell of the Mono Project, cofounder of Ximian), Me, Miguel De Icaza (Ximian/Novell - author of Mono and other great feats of hackerdom, cofounder of Ximian), Gaurav Daga (Program Manager, Microsoft - Services for Unix Team), Pooja Malpani (CTS - programmer, Microsoft MVP .Net). This was taken at Linux Bangalore 2004, the annual Linux convention.)

 

 

(President’s quote put upon a hoarding at Linux Bangalore 2004)

 

 

 

This was recently posted on Miguel’s blog (there is more - go read the entry):
http://primates.ximian.com/~miguel/archive/2004/Apr-24.html

 

Jeff seems to like Cringley's statement of "The central point was that paying too much attention to Microsoft simply allows Microsoft to define the game. And when Microsoft gets to define the game, they ALWAYS win."

 

A nice statement, but nothing more than a nice statement, other than that, its all incorrect.

 

Microsoft has won in the past due to many factors, and none of them related to `Let them define the game', a couple from a list of many:

 

·         They leveraged their monopoly to break into new markets. The most discussed one is when they used brute force and anti-competitive strategies to get their products into new markets, but in some other cases they got fairly good adoption of their products with little or no effort: just bundle it with Windows: MSN messenger, Media Player.

 

·         Competitors were outmaneuvered or were incompetent (See HIgh Stakes No Prisoners).

 

·         People were sleeping at the wheel.

In 1993-1994, Linux had the promise of becoming the best desktop system. We had real multi-tasking, real 32-bit OS. Client and Server in the same system: Linux could be used as a server (file sharing, web serving), we could run DOS applications with dosemu. We had X11: could run applications remotely on a large server, and display on small machine. Linux quickly became a vibrant innovative community, and with virtual-desktops in our window managers, we could do things ten times as fast as Windows users! TeX was of course `much better than Windows, since it focuses on the content and the logical layout' and for those who did not like that, there was always the "Andrew" word processor. Tcl/Tk was as good as building apps with QuickBasic.

And then Microsoft released Windows 95.

 

·         A few years later, everyone is talking components: Netscape is putting IIOP on their client and server (ahead of their time, this later became popular as web-services on the browser); Xerox ILU; Bonobo; KParts; the Borland sponsored event to build a small component system that everyone agrees with; language bindings are at their top.

The concensus at that time? Whatever Microsoft is doing is just a thin layer on top of COM/DCOM/Windows DNA which to most of us means `same old, same old, we are innovating!'.

And then Microsoft comes up with .NET.

 

 

Maybe, sometimes, rarely, one man can change the world – or at least make a significant dent.