DDL Enhancements / Wish List

This is the wish-list for the future, stuff we wish to do to enhance the DDL. Some of these are by us and some are by folk who have tried the DDL. This page is just being setup so please pardon inconsistencies.

Compiler:
In most cases, an end application or suite of applications would need to interpret a format once, and use it several times. As such, it might make sense to write a DDL *compiler*; one which produces a strongly-typed class for parsing a given data format. So, end applications could use the produced class rather than invoke the engine, in situations where this is more appropriate.
[Raj Chaudhuri (email: rajch at hotmail dot com)]

Designing a compiled equivalent of the DDL strcuture... show

I am little stuck about how to go about doing this. I am stuck about how to map DDL structures to corresponding .Net structures.
The fundamental issue is that a DDL structure can look like this:

struct SomeStruct
{
	i8 a
	when(a==1)
	{
		i8 b
	}
	when(a==2)
	{
		i16 b
	}
	when(a==3)
	{
		i8 b 
		i8 c
	}
	else
	{
		i32 c
	};
}

Now this means that the end C# structure must have all the different signatures that this DDL structure can have. So this gives me the following options in designing the interface to this compiled DDL code:

Generating C# structures of all possible signatures is a rather difficult case - and also it will not be very usable, as the the DDL will use runtime conditions (for a given data file) to see if it is one structure or the other that is to be instantiated.
The DDL exposes an API that creates types at runtime. I can use any of the .net code gereation/type generation principle to create types at runtime that will contain only the required members. So all structure member accesses will return an object-type. It will be the responsibilty of the callee to use reflection APi to determine what members are contained in a object retruned like this.
The simplest and most direct approach I can think of is to generate a single structure for every DDL structure of the form
- Every uniquely named DDL structure member will appear exactly once in the corresponding .Net structure
- Every member will be declared to be of the largest type that it can be in the DDL, ie if 'a' can be a i8 and i16, then 'a' will be a unsigned int16 type in .net
- Every member will have a corresponding _valid member of bool type which will indicate if the member actually exists in this instance. Thus 'a' will have a corresponding 'bool a_valid' member to indicate is 'a' actually exists.

Of the 3 choices, the third one seems to be the most feasible for now. The consuming client application will need to make a call that is the equivalent of a 'get initial structure'. After that he can access all it members as well as all its child members as though they were a locally defined data type. Every time he accesses a member however, indirectly a DDL call would be made to resolve the actual data access. The C# structure corresponding to the above one would be:

struct SomeStruct
{
	byte a;
	bool a_valid;
	ushort b;
	bool b_valid;
	uint c;
	bool c_valid;
}

At each point the hosting program would be expected to test the bool types and then read values from the corresponding data types. I think this can be done. It would be good however if the code was modified for hash code based access internally...
[Roshan James (email: spark at sscli dot net)]

Some clarification about the "compiler". show

A Compiler will be neat show

Mapper:
Another utility I can immediately think of is a "mapper"; something which allows for a DDL-defined data structure to be mapped or transformed into another. The basic nature of DDL seems to allow for doing this easily. Ship a generic XML mapper, and we are both buzzword-compliant and interoperable.
[Raj Chaudhuri (email: rajch at hotmail dot com)]

(design approach to be filled in...)

Optimizations for speed:
Specifically:
- Try and generate a hash code for member names and have the internals working on hash codes rather than string names, this will be good speed up (I don't know why I did not do this earlier). Also expose to user API a hash code version of the functions also.
[Authors (email: spark at sscli dot net / dolly at sscli dot net)]

Language Enhancements:
There need to be some additions to the DDL language. The current syntax cannot accommodate some data specifications, these may be hard to do or not possible at all. The language enhancements doc is an annotated version of the language specifications document with the proposed additions appropriately marked.

These language enhancements would possibly be released only with the compiler version of the DDL. This is the ddlc design page where you will find the current version of the Language Enhancements document.

[Authors (email: spark at sscli dot net / dolly at sscli dot net)]

Managed Code (conversion):
Convert to fully managed source code - for interop with Rotor/Mono. Right now, its is a mixed mode assembly which means that it relies on a C style unmanaged heap for some objects - this is not support by all CLI implementations as CLI specs don't mandate this.
[Authors (email: spark at sscli dot net / dolly at sscli dot net)]

Revamp the entire internal architecture and design more primitives to the language to support more data formats
refer to entry on language enhancements
[Authors (email: spark at sscli dot net / dolly at sscli dot net)]

DDL Future Enhancements/Wish-List

I think these are out of the way for now: