Model to Model transformations in Oslo, any plans?

Question

(I don't talk about transforming text into models. I know, MGrammar is great for doing that.)

We do M2M-transformations all day. We convert DAOs to DTOs, messages to messages, xml to xml, xml to xhtml, ...

Do you have any plany on creating a M* language that helps writing M2M transformations?

1. I can for example see, that the output of a DSL even using projections, might result in ugly models - just because it is to tightly bound to the textual representation. A DSL for M2M transformations could then help transforming that model into something more representative.
2. M2M would also be a great support for model evolution. Along with a new schema a M2M could then define how to convert from my type V1 to V2.
3. ...

What do you think?

Answer 1

Transformation is a whole area where I want to see us go. We have a powerful transformation language in the query syntax of M, but I don't believe we've done any serious thinking about model-to-model transformation. I'd be curious how you'd like to see that surfaced, Lars.

Answer 2

Hm, I'd like a mix of MGraph, LINQ and some dynamic-style language. Just an idea - I don't have any clear picture yet.

Books{

Books{

Book{ Author{"Juwal Lövy"}, Title{"Programming WCF Services"}, Rating {3} },

Book{ Author{{"Elisabeth Freeman","Eric Freeman","Bert Bates", "Kathy Sierra"}},

Title{"Head First Design Patterns (Head First)"}, Rating {2} }

}

}

 
Transformation:

module XY {

transform AuthorsOnly {

def Main(Books): Authors{ from b in Books.Book select b.Author.ForEach( x => Author { x }) };

}

}

Answer 3

Ah. You're in luck, Lars, We already have LINQ like query syntax in M. Check out section 1.3 of the M Language Specification: http://msdn.microsoft.com/en-us/library/dd285271.aspx

Answer 4

Sure, I've seen this. But isn't this only mapped to views when using the Oslo Repository? How is it usable via API? Is it the "equivalent" to XPath in the xml stack?

Anyway, it still doesn't enable m2m transformations. Do you think you will provide a "language" for M2M/M2T within the M-family?

Answer 5

What would you like that to look like, Lars?

Answer 6

As I described in my post below. Here with some comments -but still, it's just an idea though :-) ...

It could back something similar to XSL but with embedded python or ruby scripts (or any dynamic/static .NET Language?). It could support aspects, f.ex. by a arround keyword, and it should have linq, MGraph(Expressions) and maybe MSchema as firstclass citizens.

module XY {

// a transformation

transform AuthorsOnly {

// function that matches "Books" - as xsl:template match="Books" name="Main"

def Main(Books books):

// resulting MGraph

Authors{

// query books for each author

from b in books.Book select b.Author.ForEach(

// put the author into an successor named "Author" with a singleton

// successor "Name" with value x (labmda parameter)

x => Author { Name = x }

)

}

}

}

Answer 7

Lars, this is valid M code:

module XY {

type Book {

Title : Text;

Author : Text;

ISBN : Text;

Published : Date;

Publisher : Text;

};

type Books : Book*;

type AuthorsOnly {

from b in Books select b.Author

};

type AuthorsName {

Name = from b in Books select b.Author;

};

}

I don't think that it transforms in the sense that you would like, though, but I think it is as close as you can get with the current philosophy in M.

Remember, that M types cannot be updated, only initialized.   So the type AuthorsOnly cannot be changed to become something new later--that's not a supported scenario in the current M language.  Because of this, we are only saying that AuthorsOnly and AuthorsName are initialized with the values of Books.

It is also important to note that M philosophically isn't going to specify HOW it is going to do this.  It might do this by creating new AuthorsOnly and AuthorsName tables and copying the data from the Books table to the new table.   It might do this by create a view on top of the Books table.   I don't know which it does today--I couldn't get it to generate SQL code for me in Rich or Reach SQL mode.  Whichever it is today, there is no promise nor expectation that it will remain the same.  It might be nice if M supported an annotation that allows you to specify either one explicitly, but I've not seen anything like that yet.

Finally, a type in M is just a way to define a shape of data.  It is not a strict OO type, like C++ and C#, but a "duck" type like JavaScript.  If the shape of the data matches, then it is that type.   Whether AuthorsOnly is a table or a view is uninteresting in M.   Because either match the shape expected, then both work equally well inside M.

The only possible interesting scenario is what happens post-initialization.   What should happen to AuthorsOnly if Books is changed exogenously post-initialization?  Do you expect this to be updated automatically or only when you explicitly request it?  I think your example expects it to be updated automatically.  If that is the case, I think that this works for you.   If you expect the transformation to be applied later, then again you are outside the current scope of M.   They explicitly do not support updating data after initialization.

Given that, I think that the code above is the best that you can do with the current philosophy of M.

---

Erik

Answer 8

Hi Erik,

thanks for your answer. I know my "request" is not covered by the M philosophy. That is, what it is about. I think there is a need for beeing able to process models into new models somehow - post production. I don't want to change data - it's about generating new data. Like you do with xml-to-xml transformations using xslt.

It shouldn't matter where the data comes from. I neither see a problem with the ducktyping/structural subtyping here. It's just about processing MGraphs (produced by M or MGrammar) to other (can be similar) MGraphs.

cheers,
Lars

Answer 9

Hi Lars,

Model-to-model transformation is certainly something we're thinking about. Unfortunately, we don't know how that will look or when it will be available but your feedback helps, particularly with respect to detailing how you'd like to see it work.

I'd recommend using the "Oslo" Connect site to make this, or any, suggestions.

Cheers,
Michael

Answer 10

Thanks Mike. I submitted a Suggestion: https://connect.microsoft.com/oslo/feedback/ViewFeedback.aspx?FeedbackID=388047

Answer 11

Lars,
Here's my 2 cents for what it's worth...

The query syntax in M is quite robust. We did lots of investigation comparing it to LINQ and T-SQL. I believe it will serve well as a syntax for writing just the transformations you want. I'm sure we have holes. In fact I know we do because there still more language work going on to look at things like XPath and see what we can do. So stay tuned.

The bigger problem, as you stated, is that the only way to do those evaluations today is to run them in SQL. What you really want is an in-memory evaluation to go from 1 MGraph to another. Well, you may want to check out Mr.Epl. It's current a sample (with source so you can see how SpankyJ built it), but it illustrates an important feature that we hope to get soon - an in-memory evaluator of M expressions.

Also, we hope to use MGrammar + M Expressions on the RHS to generate interesting transformations of character streams.

If this was too cryptic of an answer, let me know cause it's past my bed time :)

---

-- pinky --/ http://www.tinyfinger.com/