Spark's Pensieve - User groups

PL-Wonks: Programming Languages mailing list, Indiana University, CS Dept

Looking for the PL group at IU? Go here to subscribe for talk schedules and other updates:

http://mailman.cs.indiana.edu/mailman/listinfo/pl-wonks

MLP (Mindless Link Propagation) | User groups

Friday, February 01, 2008 1:46:41 AM (Eastern Standard Time, UTC-05:00)

Comments [0] |

Trashbin - the Portable Executable / Metadata viewer

Trashbin finally has patch.

Before trashbin was written I used to wonder a bit out this mythical entity called metadata that is often talked about in the .Net framework. Metadata was mentioned on almost every discourse on .Net and libraries like reflection libs were based squarely on it. Metadata formed the central pillar of design of the self describing type/component system that is called the .Net framework.

However, I could find almost nothing that showed me actual metadata, in its physical form. As a consequence I decided to try and understand what metadata was all about. Thus trashbin was written. Trashbin first saw light of day when announced in a Bangalore User Group post sent in the wee hours of morning.

From: spark

Subject: metadata viewer: trashbin v0.1, src+bin release

Date: Mon, 09 Jun 2003 13:28:37 -0700

-----------------------------------------------------------

hi group,

i had been spending some of my odd freetime and sometimes lost sleep into exploring the .net exe/dll format and peeking into metadata glue.

well, the glue is rather interesting to look into and gives you a small insight into where information for things like the class loader and the reflection api get their data. trashbin is small viewer for metadata info that i am releasing with src. this is the first version anywhere and so is expected to be buggy - do mail. if internals interests you, take a look:

http://www.thinkingms.com/pensieve/homepage/work/trashbin/trashbin.htm

cheers :)

rosh

Since its release, trashbin has more or less worked fine for me and I have been using it for about a year now.
In essence, this is what trashbin does:

>trashbin

Spark (?) Managed(.Net)/Native PE-COFF file viewer. Version 0.2

May 2003, contact: rosh@mvps.org

Last update: May 2004

usage: trashbin [options]

portable executable info:

/dos display dos header

/sig display the file signature

/coff display coff header

/pe display pe/optional header

/dd display data directories in pe header

/sec display section headers

/exp display export table

/imp display import table

/reloc display relocation information

/tls display Thread Local Storage information

managed info:

/corhdr display the common language runtime header

/mdhdr display metadata headers

/md:Strings display metadata stream #Strings

/md:Blob display metadata stream #Blob

/md:US display metadata stream #US (user strings)

/md:GUID display metadata stream #GUID

/md:#~ display optimised metadata tables stream-header

/mdtab display optimised metadata tables

other:

/type indicates the type of the PE file

/csv enable excel compatible, CSV output

ps. The name trashbin is 'inspired' from dumpbin :)

Since most people who are reading this entry might be interested in what metadata is and what the PE file format is like, here goes:

The PE file format is Microsoft’s Portable Executable File Format. Essentially most exes and dlls that you will see on a windows system have this file format. Yes Exes and DLLs have the same format. The difference largely lies in the fact that a DLL file does not necessarily have an entry point defined. Here is a little about the Exe/Dll format:

Once upon a time, there used to be old DOS exes that came with what was the DOS exe header. Microsoft retained the DOS exe header in all subsequent exe formats so that the executables would be compatible across their operating systems. Which is why, you can run any windows or .Net exe on any Microsoft operating system (including dos) and see it run. Of course these programs would not do anything in the dos environment other than display a message saying that the exe would run under windows. The point however is that the exe did validly execute on a 15 or twenty year old system that was built for processors that did not have a concept of memory beyond 1Mb.

The DOS header is known for it special signature bytes MZ. Open any exe file and notice that the very first two bytes are MZ. MZ stands for Mark Zbikowski, the person who developed the DOS exe file format. Prior to that the executable format was called the Com file format. Those of you who have had the chance to work on DOS would not have forgotten what a pleasure some of those COM files used to be. The COM format belonged to the then popular CP/M operating system of Digital of the great Gary Kildall. Gary Kildall was pioneer in a way few people were.. anyway that is story for later.

The following is a dump of the initial few bytes of an exe file showing the then new MZ DOS header:

>hexv HelloWorld.exe

0000:0000│ 4D 5A 90 00 03 00 00 00 │ 04 00 00 00 FF FF 00 00 │ MZÉ▒♥▒▒▒│♦▒▒▒ ▒▒

0000:0010│ B8 00 00 00 00 00 00 00 │ 40 00 00 00 00 00 00 00 │ ╕▒▒▒▒▒▒▒│@▒▒▒▒▒▒▒

0000:0020│ 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00 │ ▒▒▒▒▒▒▒▒│▒▒▒▒▒▒▒▒

In a sense Mr Zbikowsky has the most popular initials on the planet. The DOS exe header itself is available as a structure that is defined in the winnt.h header file that is available on almost every windows based c/cpp dev environment.

Now the DOS exe header did not suffice to hold a lot of the new information that the exe had to present to the operating system, when windows came along. So new structures were introduced which were the akin to the old unix based Common Object File Format (COFF). There is plenty of literature available about this on the net.

The PE file is denoted by the signature bytes “PE ". If you download trashbin the source code has some embedded urls that give you information about the PE file format itself. Those may prove valuable for your understanding of the actual exe file format.

Just to connect all that I have been talking about to trashbin and how you can actually examine an exe file with it, these are the relevant switches.

/dos display dos header

/sig display the file signature

/coff display coff header

/pe display pe/optional header

Now that we have covered that ground, lets move on. The PE file has a data structure called the Data Directory which is displayed through the /dd option.

/dd display data directories in pe header

The data directory basically contains pointers to various data structures inside the PE file. The DD has 16 entries and a dump of the DD looks like this:

C:\WINNT\system32>trashbin tracert.exe /dd

_IMAGE_DATA_DIRECTORY

0 VirtualAddress = 0

Size = 0

1 VirtualAddress = 0x19ac

Size = 0x78

2 VirtualAddress = 0x3000

Size = 0x11b8

3 VirtualAddress = 0

Size = 0

4 VirtualAddress = 0

Size = 0

5 VirtualAddress = 0

Size = 0

6 VirtualAddress = 0x1090

Size = 0x1c

7 VirtualAddress = 0

Size = 0

8 VirtualAddress = 0

Size = 0

9 VirtualAddress = 0

Size = 0

10 VirtualAddress = 0

Size = 0

11 VirtualAddress = 0x240

Size = 0x7c

12 VirtualAddress = 0x1000

Size = 0x88

13 VirtualAddress = 0

Size = 0

14 VirtualAddress = 0

Size = 0

15 VirtualAddress = 0

Size = 0

This is trashbin examining tracert program that comes with windows. The tracert program is a native exe (not a .net exe). Entries in the DD have a predefined meaning, these are defined in winnt.h as follows:

#define IMAGE_DIRECTORY_ENTRY_EXPORT 0 // Export Directory

#define IMAGE_DIRECTORY_ENTRY_IMPORT 1 // Import Directory

#define IMAGE_DIRECTORY_ENTRY_RESOURCE 2 // Resource Directory

#define IMAGE_DIRECTORY_ENTRY_EXCEPTION 3 // Exception Directory

#define IMAGE_DIRECTORY_ENTRY_SECURITY 4 // Security Directory

#define IMAGE_DIRECTORY_ENTRY_BASERELOC 5 // Base Relocation Table

#define IMAGE_DIRECTORY_ENTRY_DEBUG 6 // Debug Directory

// IMAGE_DIRECTORY_ENTRY_COPYRIGHT 7 // (X86 usage)

#define IMAGE_DIRECTORY_ENTRY_ARCHITECTURE 7 // Architecture Specific Data

#define IMAGE_DIRECTORY_ENTRY_GLOBALPTR 8 // RVA of GP

#define IMAGE_DIRECTORY_ENTRY_TLS 9 // TLS Directory

#define IMAGE_DIRECTORY_ENTRY_LOAD_CONFIG 10 // Load Configuration Directory

#define IMAGE_DIRECTORY_ENTRY_BOUND_IMPORT 11 // Bound Import Directory in headers

#define IMAGE_DIRECTORY_ENTRY_IAT 12 // Import Address Table

#define IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT 13 // Delay Load Import Descriptors

#define IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR 14 // COM Runtime descriptor

You can compare these against what is present in the tracert program dump.

Now lets do a DD dump of a .net exe

>trashbin HelloWorld.exe /dd

_IMAGE_DATA_DIRECTORY

0 VirtualAddress = 0

Size = 0

1 VirtualAddress = 0x2370

Size = 0x4b

2 VirtualAddress = 0x4000

Size = 0x340

3 VirtualAddress = 0

Size = 0

4 VirtualAddress = 0

Size = 0

5 VirtualAddress = 0x6000

Size = 0xc

6 VirtualAddress = 0

Size = 0

7 VirtualAddress = 0

Size = 0

8 VirtualAddress = 0

Size = 0

9 VirtualAddress = 0

Size = 0

10 VirtualAddress = 0

Size = 0

11 VirtualAddress = 0

Size = 0

12 VirtualAddress = 0x2000

Size = 0x8

13 VirtualAddress = 0

Size = 0

14 VirtualAddress = 0x2008

Size = 0x48

15 VirtualAddress = 0

Size = 0

The interesting thing to note is that in a managed exe, the 15^th entry is non zero. The 14^th entry point to the Common Runtime Header or the CorHdr. The CorHdr structure is defined in corhdr.h.

The CorHdr is where (so to speak) the managed world starts. So .Net exe files are regular PE files which have all the .Net specific content in a particular offset in the exe file. The idea was that .Net was designed to be platform neutral. So Microsoft could assume that the facilities provided by the exe file format would be available in file formats of other operating systems where .Net would one day run. For this sake .net specific content assumes that the whatever the parent file format that is enclosing it, the .Net specific content can be made laid out in a large well defined binary blob.

A description of the CorHdr, Metadata layout and other intricacies of the underlying system can be found described (in varying detail) in the ECMA specification of the Common Language Infrastructure. Partition 2 that describes metadata is the one that you would want to look it, in this regard.

http://msdn.microsoft.com/net/ecma/

Another interesting resource for the technical mind is Serge Lidin’s book “Inside the .Net IL Assembler” . The book is available as an Indian India edition also, so it is affordable.

Trashbin lets you view this part of the managed exe file.

These are the relevant switches:

/corhdr display the common language runtime header

/mdhdr display metadata headers

/md:Strings display metadata stream #Strings

/md:Blob display metadata stream #Blob

/md:US display metadata stream #US (user strings)

/md:GUID display metadata stream #GUID

/md:#~ display optimised metadata tables stream-header

/mdtab display optimised metadata tables

Trashbin is not a dissembler – it simply lets you view the other data that is present in the exe/dll file. I figured I don’t need to do a dissembler because ILdasm and several other tools do the job so well. ILdasm btw is written by Serge Lidin.

The metadata in the managed file is divided into a set of streams. A stream is like an area of memory reserved for content of a specific type. These are namely –

#Strings

#US

#Blob

#GUID

#~ and #-

The #Strings stream keeps all the strings in the application that include things like names of classes, methods, parameters, namespaces, assemblies etc. So this stream basically contains all kind of strings that are part of the source code itself – these are where the names that are used by the reflection library are available.

The #US stream is the one for user defined strings. So when you say Console.WriteLine(“Hello World”) in your program, the “Hello World” goes into #US and the Console.WriteLine goes into #Strings. The strings present in the #US set are Unicode strings – so each character is two bytes.

Here are some nice and friendly hex dumps of these regions from a exe file followed by the corresponding information being ripped by trashbin:

#Strings

0000:0460│ 00 00 00 00 00 00 00 00 │ 00 3C 4D 6F 64 75 6C 65 │ ▒▒▒▒▒▒▒▒│▒

0000:0470│ 3E 00 48 65 6C 6C 6F 57 │ 6F 72 6C 64 2E 65 78 65 │ >▒HelloW│orld.exe

0000:0480│ 00 6D 73 63 6F 72 6C 69 │ 62 00 53 79 73 74 65 6D │ ▒mscorli│b▒System

0000:0490│ 00 4F 62 6A 65 63 74 00 │ 43 61 6C 63 00 48 65 6C │ ▒Object▒│Calc▒Hel

0000:04A0│ 6C 6F 57 6F 72 6C 64 53 │ 61 6D 70 6C 65 00 48 65 │ loWorldS│ample▒He

0000:04B0│ 6C 6C 6F 57 6F 72 6C 64 │ 00 41 64 64 00 2E 63 74 │ lloWorld│▒Add▒.ct

0000:04C0│ 6F 72 00 4D 61 69 6E 00 │ 53 79 73 74 65 6D 2E 44 │ or▒Main▒│System.D

0000:04D0│ 69 61 67 6E 6F 73 74 69 │ 63 73 00 44 65 62 75 67 │ iagnosti│cs▒Debug

0000:04E0│ 67 61 62 6C 65 41 74 74 │ 72 69 62 75 74 65 00 61 │ gableAtt│ribute▒a

0000:04F0│ 00 62 00 61 72 67 73 00 │ 43 6F 6E 73 6F 6C 65 00 │ ▒b▒args▒│Console▒

0000:0500│ 57 72 69 74 65 4C 69 6E │ 65 00 45 78 63 65 70 74 │ WriteLin│e▒Except

0000:0510│ 69 6F 6E 00 67 65 74 5F │ 4D 65 73 73 61 67 65 00 │ ion▒get_│Message▒

>trashbin HelloWorld.exe /md:Strings

METADATA STREAM #Strings

Offset : "String"

0x1 : ""

0xA : "HelloWorld.exe"

0x19 : "mscorlib"

0x22 : "System"

0x29 : "Object"

0x30 : "Calc"

0x35 : "HelloWorldSample"

0x46 : "HelloWorld"

0x51 : "Add"

0x55 : ".ctor"

0x5B : "Main"

0x60 : "System.Diagnostics"

0x73 : "DebuggableAttribute"

0x87 : "a"

0x89 : "b"

0x8B : "args"

0x90 : "Console"

0x98 : "WriteLine"

0xA2 : "Exception"

0xAC : "get_Message"

#US

0000:04B0│ 65 00 00 00 00 17 48 00 │ 65 00 6C 00 6C 00 6F 00 │ e▒▒▒▒↨H▒│e▒l▒l▒o▒

0000:04C0│ 20 00 57 00 6F 00 72 00 │ 6C 00 64 00 00 00 00 00 │ ▒W▒o▒r▒│l▒d▒▒▒▒▒

>trashbin test.exe /md:US

METADATA STREAM #US

0x1, (23 bytes)

Txt: H.e.l.l.o...W.o.r.l.d..

Hex: 48 00 65 00 6c 00 6c 00 6f 00 20 00 57 00 6f 00 72 00 6c 00 64 00 00

I will just skip over the #GUID and #Blob streams for now. The #~ stream is the interesting one that is the stream that actually contains the metadata tables. These is an alternate stream thaty can be present which is the #- stream. The #- stream again contains metadata tables but these are called the un-optimized tables because certain sort orders are not maintained in these tables. The Microsoft compiles always emit optimized tables and since I have not been using any other compilers (Mono too seems to emit optimized tables) I don’t have support for #- in trashbin.

Lets focus on #~. The #~ is the real metadata, if you would like to think of it that way, It is actually a small relational database that is compressed down to the last bit. There are a large number of tables (with predefined schemas) that can occur here. These tables provide information about the exe or dll (or precisely the assembly) that they are trying to describe.

These tables cross reference each other as well as reference entries in the other streams, such as the #GUID and #Strings streams. The trashbin option /md:#~ gives you the header of #~ stream, which is a kind of summary view of what the stream contains:

>trashbin HelloWorld.exe /md:#~

METADATA STREAM #~

TABLES HEADER

MajorVersion = 1

MinorVersion = 0

HeapSizes = 0

#String Index = 2 bytes wide

#GUID Index = 2 bytes wide

#Blob Index = 2 bytes wide

Valid = 0x00000900021547

Sorted = 0x0002003301fa00

METADATA Tables

RID. TableName [No of Rows]

0. Module [1] Row=10 bytes

1. TypeRef [4] Row=6 bytes

2. TypeDef [3] Row=14 bytes

6. Method [4] Row=14 bytes

8. Param [3] Row=6 bytes

10. MemberRef [5] Row=6 bytes

12. CustomAttribute [1] Row=6 bytes

17. StandAloneSig [2] Row=2 bytes

32. Assembly [1] Row=22 bytes

35. AssemblyRef [1] Row=20 bytes

Table Count = 10

The above listing says that helloworld.exe contains 10 tables in its metadata headers.

To give you a taste of what I am talking about lets look at some of these tables:

To look at the metadata tables, use the option

/mdtab display optimised metadata tables

This is the table called TypeDef that has a listing of all the types defined in this assembly.

[RID=2] Table TypeDef

[ DATA]Flags [STRING]Name [STRING]Namespace [CI:64 ]Extends [RID: 4]FieldList [RID: 6]MethodList

1.[ 0x ]0 [ 0x ]1 [ 0x ]0 [RID: 2] 0 [RID: 4] 1 [RID: 6] 1

2.[ 0x ]100001 [ 0x ]30 [ 0x ]35 [RID: 1] 1 [RID: 4] 1 [RID: 6] 1

3.[ 0x ]100001 [ 0x ]46 [ 0x ]35 [RID: 1] 1 [RID: 4] 1 [RID: 6] 3

The table has a field called name – which is the name of the type. The value of the names field are actually offsets into the #Strings stream. So if you care to compare (I have provided a listing of the #Strings stream earlier), what the table is saying is that this assembly has 3 types which are namely , Calc and HelloWorld. At this point let me drop the source code of the HelloWorld.cs file from which this assembly was compiled:

//HelloWorld.cs

namespace HelloWorldSample

{

using System;

public class Calc

{

public int Add(int a, int b)

{

return a+b;

}

public class HelloWorld

{

public static void Main(string[] args)

{

try

{

Calc c = new Calc();

Console.WriteLine(c.Add(10,20));

}

catch(Exception e)

{

Console.WriteLine(e.Message);

}

As you can see the source does define two classes called Calc and HelloWorld. A type in .Net terms is a very strict entity that encompasses value types and reference types and type entities like classes, structures, enums etc.

Here is a table that shows me what types are being referenced in this assembly:

[RID=1] Table TypeRef

[CI:75 ]ResolutionScope [STRING]Name [STRING]Namespace

1.[RID:35] 1 [ 0x ]29 [ 0x ]22

2.[RID:35] 1 [ 0x ]73 [ 0x ]60

3.[RID:35] 1 [ 0x ]90 [ 0x ]22

4.[RID:35] 1 [ 0x ]A2 [ 0x ]22

The types are Console, Object, Exception etc

Here is a list of all the methods that are defined within this assembly;

[RID=6] Table Method

[ DATA]RVA [ DATA]ImplFlags [ DATA]Flags [STRING]Name [ BLOB]Signature [RID: 8]ParamList

1.[ 0x ]2050 [ 0x ]0 [ 0x ]86 [ 0x ]51 [ 0x ]A [RID: 8] 1

2.[ 0x ]2064 [ 0x ]0 [ 0x ]1886 [ 0x ]55 [ 0x ]10 [RID: 8] 3

3.[ 0x ]2078 [ 0x ]0 [ 0x ]96 [ 0x ]5B [ 0x ]14 [RID: 8] 3

4.[ 0x ]20C8 [ 0x ]0 [ 0x ]1886 [ 0x ]55 [ 0x ]10 [RID: 8] 4

You can see the method name add, Main etc here, as well as auto-generated methods such as .ctor (the constructor).

In short if you examine the metadata headers carefully you get an idea about the extent of meta information being stored about your program and what can be supported by any reflection based API. A complete listing of the metadata tables are provided in the ECMA specs, so that’s your best reference, Mr Lidin’s book also does a good job of this and I used both of these while I was writing trashbin.

So there – I have said what I wanted to say about trashbin. This might be a good starting point to explore the exe and dll file formats.

Before I stop, I must add, do take a look at the switches

/exp display export table

/imp display import table

these show the methods that are exported from an exe/dll (so that it maybe dynamically loaded and invoked from another process) and shows what method calls it invokes. This is kind of the ‘metadata’ of the old unmanaged world.

Typing trashbin ntdll.dll /exp in the system32 folder will give you a listing of the native NT api, which might make for good reading again (if that sort of stuff gets your interest).

Most of the content of this blog entry was presented at the Bangalore .Net User group as one of the UG meetings talks.

All that said, here is the real reason I started out on this blog entry: Recently Kaushik Srenevasan pointed out a bug in trashbin’s #Strings parsing routine. Kaushik is a ‘MS Student Ambassador’ and has a blog here: http://dotnetjunkies.com/WebLog/kaushik/

I had, for some reason, assumed the presence of padding 0s when I had written the original code. That more or less always worked, because the #Strings stream was expected to be padded with zeros till is a 4 byte boundary.

Its has been patched according to Kaushik’s suggestion, so Thank you Kaushik.

Trashbin can be downloaded from its homepage here:

http://www.thinkingms.com/pensieve/homepage/work/trashbin/trashbin.htm

As foot note I must say that the Microsoft Dumpbin program that ships with Visual Studio does most of the things that trashbin does and a few additional things. Dumpbin however does not display metadata information from managed PE files yet.

Programming | Talks | User groups | Zeitgeist

Tuesday, May 18, 2004 7:20:34 AM (Eastern Standard Time, UTC-05:00)

Comments [0] |

Linux Bangalore and Mono’s Miguel

I thought of putting together some of my old mails and user group posting as blog entries, with some patches so that they are’ blogopatible’. They would have ended up on my blog, if I had a blog when I made these posts. I feel some of these are of lasting importance, at least with respect to the impressions they had on me.

All of these are personal opinions, probably more relevant in the context that they were originally written.

-----Original Message-----

From: James, Roshan

Sent: Friday, December 05, 2003 5:37 PM

To: MVP Mailing List

Subject: An Audience with Miguel

written in a hurry:

An Audience with Miguel

Hi, yesterday Pooja and I got to catch a part of Linux Bangalore, the annual Linux convention, and I thought that it would be nice to write to mvplist and share our experiences. We had missed the dates for the event and I was rather shocked at having missed a chance to see Miguel De Icaza in person.

Miguel, for those who don't know about him, is the creator of mc (the Midnight commander), Gnome (the rather popular open source desktop) and in recent days the lead for the Mono project. The Mono project is the only other major (when I say major here, it is not that I don’t know of dotGNU and other attempts, it is because I personally feel that Mono is more complete than those) .Net implementation outside of the Microsoft world. Mono runs on Windows as well as on Linux and probably other Unix flavors. This guy has been famous/notorious in the open source community for writing papers like 'Lets make Unix not suck' and has a rather 'misfit' personality for the typical religious ramblings of the free software types. In short this man has written his own windowing systems, his own .Net, his own enterprise servers and was running his own company called Ximian.

Ximian has been recently bought by Novell making Novell a major player in the open source world. Novell has also bought Suse - major Linux flavour and Novell seems to be on the path of becoming a significantly important open source company, along the lines of maybe Red Hat and such. However unlike Redhat and many of the 'Linux companies', Novell has a focus on delivering products rather than making Linux distributions and delivering them dirt cheap or free if they have to, to get into the market. So in short Miguel is as much a demi-god as our own Anders Hejlsberg, or Don Box.

The above paragraph was to set things in perspective. This is the mono website (http://www.go-mono.com/). Having been rather disappointed to have missed Linux Bangalore dates and missed Miguel, I happened to check their talk schedules by sheer accident around 11.30 yesterday. As fate would have it, there was a talk by Miguel, his last one, scheduled at 12.00. After a ~12km drive and some conversation with the registration counter we were at the IISC Bangalore venue – the last time I was here, I was attending the Microsoft Tech Ed.

One thing you notice up front as you enter is a big banner of Abdul Kalam, our president. The poster quotes him saying that it is probably not good to have important national software depending on proprietary solutions as proprietary solutions and highly dependant on the market that the vendors cater to and that sort of unreliability is probably not a good thing. And also that free software would really help a poorer country like ours as prices of commercial software are rather high. At least that was the message in spirit - don't think I have got any of his words right.

The crowd, as far as I could say was probably the same caliber as the technical crowd I usually get to interact with at UGs and various .Net technology events. Probably not as good in some respects - but there was this thing in the air that they were all up to 'something important'. Also one other thing that was noticeable was the set of demo computers setup, where people could sit down and try out many of the software that was being talked about in the talks.

Miguel's talk was at the main hall of the IISC venue (those who know the place will know what I mean). He was accompanied by Nat Friedman of Novell, a fellow mono-ist. What happened at the talk was something I wasn't prepared for. For those of you who have a mental image of Miguel by now, this guy is young, in his early twenties. He was carrying a digital SLR camera with a hefty lens and flash addons and was dressed in baggy jeans and black t-shirt. So was the other guy Nat. Now these guys hop on stage (literally), sit down on the floor - one of them rolls out a length of cable that he had wrapped around his neck the whole while. They pull out two laptops and they get a network setup between their systems and the presentation starts up.

The presentation showed of some parts of C# , web services, GTK# for windowing, GTK + for generating XML markup for the UI (and some jokes about how XAML is a copy of their own 6 year old idea) and more. These guys did the whole presentation sitting or lying down on the floor of the stage, sometimes editing the presentation right there in front of the audience, pulling jokes on each other and writing code the whole time for a full hour - and the whole thing was on their own Mono. Awesome.

After the talk Pooja and I met up with Miguel and introduced ourselves as being from the ‘dark side’ (He asked me if I was an ASP.Net developer because of all the questions I was asking from what otherwise seemed to be a relatively .Net ignorant audience. He was interested in the fact that I was an MVP.) and asked him if we could meet him sometime later, maybe over dinner or so. He was fine with that – that however was not to happen as his schedule did not allow him to. Various activities were planned for him for the whole of the next day (today, Friday) and he was going to flying on Saturday. He did promise to talk to our .Net user group the next time he is in India (expected to be in the second quarter of next year). I had to get back to office so I had to leave then; with the intention of returning for Jani's talk a little later in the evening.

I managed to sneak out of office again to attend the talk by Mr. Janakiram (I hope you know him - he heads Microsoft India's academic/university relationship program). Along with Jani was Mr. Gaurav Daga (he is a Program Manager of the famous Services for Unix team at Hyderabad; SFU won last years best open source software of the year award). Their venue didn't do any justice to their talk. Their talk was scheduled at one of the smaller halls and the crowd was so packed for the talk that I couldn't get close to the door of the hall. Jani and Gaurav as usual pulled a great show. They were talking about the new 'Unix' being built in windows ;) (Their demo was rather awesome: they took a Unix app ran it on windows literally, they wrote .Net code and exposed it as a web service, they built a proxy around the service and consumed the proxy as a COM component which was used by excel which was used to dump – did I miss anything?)

All the while, from when he finished his talk, Miguel and his gang were there in the main lobby, showing off code or sitting or taking photographs or ready to talk to anyone at all about anything. When he was not doing that he would be sitting around in some corner with Nat typically typing away at a piece of code he was working on.

Since I couldn't get to listen to Jani at all due to the crowd and having driven the ~12km stretch in Bangalore traffic for the third time that day, I was walking around looking rather moody. Miguel then walks up to me, 'Microsoft dude', and gets our picture taken together. He kept calling me 'Microsoft dude' and he himself wanted to be called 'The Dude'; it was interesting talking to him.

I wanted to introduce Jani and Gaurav to Miguel and Nat after their talk. It was funny because on meeting Jani and Gaurav, Miguel seemed to freeze up a little bit - I think it's that Microsoft-effect. But on the whole they were nice folk. And Jani and Gaurav were good too. Jani mentioned how he wrote a wrapper around the 'Tk' widget library and called it a windows forms assembly and got to run some regular winforms code on Linux’

In retrospect, if I wanted to pick faults with the event I could and I could say that the organizing was bad, because the projector was shaky and the stage wasn't setup properly and the food was bad and all that and be complacent about the whole thing. But looking at the good side, you see guys who are probably the gods of their community actually sitting around with the developers showing off their code and laughing and talking rather that running off after a talk or sitting in a separate area. These are probably something's that we could learn from these folk. The number of people who Miguel touched that day and the number of people who will remember him are much more that those who will remember any of the speakers at technology forums where I have had the opportunity to speak or attend.

Miguel was different from my vision of the open source advocate. Probably because he was less of an advocate and more of a real programmer.

Miguel said that no they don't say that they are going to beat Microsoft or that Microsoft is going away or anything - which is normally common talk for OSI folk I have previously met. There is place for both he said. He said he has friends at Microsoft. He said that Dave Stutz is a good friend. We talked a little about Stallman and Free software. I asked him if he would be joining MS like Don Box offered and he said probably not. He would rather be doing his thing like this and be 'helping out the poorer countries'. He says they will keep on writing software, take good ideas wherever they find them and give it out as dirt cheap or free. There was this time when he said that 'they took out the GC and put in a toy GC in rotor and took out the JIT and put in a toy JIT in rotor - but the GC in Mono is your GC as much as its my GC' and he said that to Jani - 'its your as much as its mine and I would like to keep doing that'. There is this certain element of real sincerity which I find so missing in my work place and often at our technical seminars.

Free wheeling aside and sorry for all the typos and bad language in my writing, one thing that I probably miss from speakers and from many of our communities is that people hardly seem to be doing all that for themselves as much as for the community - I haven't seen anyone sit down and give all their time and energy to the community they are trying to foster. I don't see anyone on our side actually be there with the people and spread that sense of what they are doing - most of the audiences at our talks see us as speakers on podiums, rarely as people, we don't usually give them the room for that. I wish we could do that. And I wish that when we work on our user groups we can do it for the sense of community, rather than for the sakes of meeting numbers and budgets and revenue targets and stuff. I wish we all do this because we like our technology first - the Microsoft communities have never been able to do that the way these guys have. (This was probably rant, but reading this months later I feel that a lot of human touch is still missing in the communities. Somewhere the communities are built around a carrot culture and people who do things for carrots. Miguel has his carrots, but the way I saw his carrots were from a kind of passion that I personally feel in speakers I have known, myself being equally at fault. Something about what happened there that day felt like the spirit of those things I have thought about so much – hackerdom, the free hackers, the hacker ethic, mentor’s manifesto. Something is missing here and I am sure we can fix it because we have some extremely smart people who are so passionate about their technology)

I hope I have not tipped off any one by writing all this; this is probably something that could use some thought.

Cheers

Roshan

(Left to Right: Natt Friedman (Ximian/Novell of the Mono Project, cofounder of Ximian), Me, Miguel De Icaza (Ximian/Novell - author of Mono and other great feats of hackerdom, cofounder of Ximian), Gaurav Daga (Program Manager, Microsoft - Services for Unix Team), Pooja Malpani (CTS - programmer, Microsoft MVP .Net). This was taken at Linux Bangalore 2004, the annual Linux convention.)

(President’s quote put upon a hoarding at Linux Bangalore 2004)

This was recently posted on Miguel’s blog (there is more - go read the entry):
http://primates.ximian.com/~miguel/archive/2004/Apr-24.html

Jeff seems to like Cringley's statement of "The central point was that paying too much attention to Microsoft simply allows Microsoft to define the game. And when Microsoft gets to define the game, they ALWAYS win."

A nice statement, but nothing more than a nice statement, other than that, its all incorrect.

Microsoft has won in the past due to many factors, and none of them related to `Let them define the game', a couple from a list of many:

· They leveraged their monopoly to break into new markets. The most discussed one is when they used brute force and anti-competitive strategies to get their products into new markets, but in some other cases they got fairly good adoption of their products with little or no effort: just bundle it with Windows: MSN messenger, Media Player.

· Competitors were outmaneuvered or were incompetent (See HIgh Stakes No Prisoners).

· People were sleeping at the wheel.

In 1993-1994, Linux had the promise of becoming the best desktop system. We had real multi-tasking, real 32-bit OS. Client and Server in the same system: Linux could be used as a server (file sharing, web serving), we could run DOS applications with dosemu. We had X11: could run applications remotely on a large server, and display on small machine. Linux quickly became a vibrant innovative community, and with virtual-desktops in our window managers, we could do things ten times as fast as Windows users! TeX was of course `much better than Windows, since it focuses on the content and the logical layout' and for those who did not like that, there was always the "Andrew" word processor. Tcl/Tk was as good as building apps with QuickBasic.

And then Microsoft released Windows 95.

· A few years later, everyone is talking components: Netscape is putting IIOP on their client and server (ahead of their time, this later became popular as web-services on the browser); Xerox ILU; Bonobo; KParts; the Borland sponsored event to build a small component system that everyone agrees with; language bindings are at their top.

The concensus at that time? Whatever Microsoft is doing is just a thin layer on top of COM/DCOM/Windows DNA which to most of us means `same old, same old, we are innovating!'.

And then Microsoft comes up with .NET.

Maybe, sometimes, rarely, one man can change the world – or at least make a significant dent.

Ethos | Mish Mash | Programming | User groups | Zeitgeist

Tuesday, April 27, 2004 1:04:00 PM (Eastern Standard Time, UTC-05:00)

Comments [4] |

Bangalore User Group – WSH, WMI and System.Management

Yesterday I delivered a talk at the Bangalore .Net User Group.

I gave a small introduction to the concept of the Windows Scripting Host and how the scripting host allows for multiple language interpreters to be plugged in and run via the same scripting interface. I also talked a little about the object model that is exposed to scripts under windows. Most of the time I was comparing this to the general notion of shell script in Unix, and how in the windows culture the need for small discrete utilities strung together by shell-script is replaced by having a language agnostic object model and API in windows scripting world.

The talk then went on to introduce WMI or Windows Management Instrumentation API. I went on to talk about how the API works, the class and namespace architecture and how to do reflecting in the WMI API. The sheer power of this API and how little understood and appreciated it is in Windows has always bothered me. Probably I will blog about WMI itself another day.

The talk finally wrapped by showing of some basic C# code that can use WMI. Since there were several queries about the samples and presentation I thought I will put them up here.

Here is the presentation for the talk.

Windows Scripting Host.ppt

Code snippets:

To run these on your machine, copy and them and save them as the corresponding file names. The go to the command line and type:
cscript

wshbasic.vbs

This displays all the running processes on your machine

set serv = GetObject("winmgmts://./root/cimv2")

set objs = serv.InstancesOf("Win32_Process")

for each obj in objs

wscript.echo obj.name

This shows more information about each process by accessing some more properties of the process object

set serv = GetObject("winmgmts://./root/cimv2")

set objs = serv.InstancesOf("Win32_Process")

for each obj in objs

wscript.echo obj.name &" "& obj.ProcessId &" "& _

obj.ThreadCount &" "& obj.KernelModeTime

This shows how you can find out what all properties the process has. This is generally applicable to any class not just to Win32_Process

set obj = GetObject("winmgmts://./root/cimv2:Win32_Process")

wscript.echo "------------- Properties ----------------"

for each prop in obj.Properties_

wscript.echo prop.name

And this shows you how you can access the methods of a object. You could easily put these two scripts together that will show you more or less complete information about an object.

set obj = GetObject("winmgmts://./root/cimv2:Win32_Process")

wscript.echo "------------- Methods ----------------"

for each prop in obj.Methods_

wscript.echo prop.name

This shows you how to call a method on a object. This example will close all the IE browser instances on your computer. The earlier script, wshmethods.vbs, would show you that a terminate() method existed for the Win32_Process class.

set serv = GetObject("winmgmts://./root/cimv2")

set objs = serv.InstancesOf("Win32_Process")

for each obj in objs

if obj.name = "IEXPLORE.EXE" then

obj.Terminate()

end if

This shows off the usage of WQL or the Windows Query Language in interacting with WMI. It does the same task as the above script.

set serv = GetObject("winmgmts://./root/cimv2")

set objs = serv.ExecQuery("select * from Win32_Process where name = 'IEXPLORE.EXE'")

for each obj in objs

obj.terminate()

This shows you how to know what other classes exist. Notice all the above examples used only the Win32_Process class. WMI provides hundreds of classes that you can use for various tasks. It is a fairly broad API. This shows you the classes that exist. You can then find out more about each class by examining its methods and properties using the scripts I have given earlier.

set serv = GetObject("winmgmts://./root/cimv2")

set results = serv.SubClassesOf()

for each result in results

wscript.echo result.Path_

In the WMI API, classes are organized into namespaces and the above script shows only the scripts available in the root/cimv2 namespace. This script will recursively traverse the namespace hierarchy and list all the classes available under each namespace. Neat ?

dispNS ""

sub dispCLS(ns)

Set serv = GetObject("winmgmts:\\.\root\" & ns)

Set classes = serv.SubclassesOf()

For Each classobj In classes

wscript.echo vbTab & classobj.Path_.Path

sub dispNS(ns)

Set serv = GetObject("winmgmts:\\.\root" & ns)

Set namespaces= serv.InstancesOf("__NAMESPACE")

For Each namespace In namespaces

wscript.echo "Classes in Namespace = " & ns & namespace.name

dispCLS ns & namespace.name

That’s it with the script samples. Try running them, you might be surprised. You can look at MSDN for further documentation of the WMI classes, not all are documented. A couple more of tips:

If you want to connect to a another computer, not the local one, then change the GetObject() call in the above scripts to GetObject(“winmgmts:\\”). By specifying “.” the script will establish a connection to local computer.

To run a WMI script, the script needs to be run under admin privileges. Understandable considering the powerful things that these scripts can do. If you are not running as admin on your machine (which you shouldn’t be), you can use impersonation as shown in the code below.

I got this from:

http://www.activexperts.com/activmonitor/windowsmanagement/wmi/samples/wmiremote/

Sub ListShares( strComputer, strUser, strPassword )

Dim strObject

Dim objLocator, objWMIService, objShare

Dim colShares

Set objLocator = CreateObject( "WbemScripting.SWbemLocator" )

Set objWMIService = objLocator.ConnectServer ( strComputer, "root/cimv2", strUser, strPassword )

objWMIService.Security_.impersonationlevel = 3

Set colShares = objWMIService.ExecQuery( "Select * from Win32_Share" )

For Each objShare In colShares

Wscript.Echo objShare.Name & " [" & objShare.Path & "]"

Now finally the C# source, its easy to guess what this does:

using System.Management;

using System;

class CMain

{

static void Main()

{

string qs = "select * from Win32_process";

ObjectQuery q = new ObjectQuery(qs);

ManagementObjectSearcher sr = new ManagementObjectSearcher(q);

foreach(ManagementObject obj in sr.Get())

{

Console.WriteLine(obj["Name"]);

}

I also have a do-all, end-all killer Perl script that does almost everything that you can think of with WMI. I will however make a post about that another day. I have an entry about this here

Programming | Talks | User groups

Saturday, April 17, 2004 4:38:49 AM (Eastern Standard Time, UTC-05:00)

Comments [0] |