none
My view on views, Repositories, Multi-user acces etc. RRS feed

  • General discussion

  • Hi all,

    Sorry for the long message, this probably is better suited for a blog, but I don't have one (yet).

    In various threads issues have been mentioned that deal with handling large models. For example in In Multi-User Capability there is a question on multi-user capabilities, and in Collaborating on models? a solution is proposed.

    In How to make Multiple Views a request for supporting multiple views has been done.

    In Question on roadmap to DSL features a repository feature is requested.

    We have been working with the DSL toolkit since November 2005. The first DSL's are now ready for production, several others are under development. Based on our experience with developing these DSL's, I would like to give some feedback on these requests. The main message I want to give is that it is very questionable whether features like views, repositories and model element based version control are needed at all. They are rather complex features and might introduce more problems than they solve. Let's take a look.

    Views

    The need for views arises from the fact that models are too big. Coming from the world of UML (which I certainly am :-) the first idea is that you need one big interconnected model that describes everything. Following this approach there is a clear need for views, because the full model is not displayable and understandable anymore. The question is whether there is a need for models to be big, or whether we are just used to that by our UML heritage. Within our software factory we have designed our DSL's in such a way that models remain small. For one DSL, we always create multiple models in a project. As a consequence we get more but smaller models. It does take a different point of view to modeling and to what models are.

    We have found the approach of more and smaller models much more flexible than having less and bigger models. As each model remains small, the need for views dissappears automatically.

    Reposititory

    The need for a repository is clear in a UML environment, where we work with one big interconnected model. In practice, UML repositories are overly complex and hard to build and use. I have use many UMLm tools over time, but none of them solved this issue in a good way. Also, keeping all the invividual models synchronised with a repository has proved to be hard and errorprone.

    In contract, we use many small standalone models, and have added to the DSL Tools a component that allows us to share some (not all!) specific information between models. This allows us to do full inter-model validation, such that we can guarantee that a collection of models is complete and consistent. The same component also allows us to use intellisense between models and can be used to perform refactoring over models. Models themselves use explicit reference elements to refer (by name) to elements in other models.

    Modelelement based version control

    I do not see version control of indivisual model elements as nesssecary. Having used many UML tools that try to achieve this I can only say that in my experience this never works. It only leads to an unnessecary complexity. Apart from that you always end up with non-standard version control systemns. When using smaller models, we can use standard file based version control. We all have much experience with these systems and they are known to be reliable.

    More and Smaller Models

    Using the approach of more and smaller models, we also find additional advantages. For example, resue of models becomes easier. Existing version management for sorce code be used for models as well, as each model is small enough for one modeler to work on. Just try to compare one model with one source code file. They are standalone entities, and the development environent can check and resolve any dependencies.

    Conclusion

    By moving into the direction of creating large repositories, views and finegrained version control for model elements, we will be tempted to design DSL's that will become larger and larger. In the end I fear that we will end up with UML implemented as DSL's. This will introduce all of UML's problems, which, as far as I am concerned, is exactly what we are trying to avoid. At least, I don't want to do this again.

    Views and repositories are not requirements for a modeling environment. They are solutions to an underlying problem. I think that the underlying problem is that models are too big. We have found that the underlying problem can be solved in a very different way, using more and smaller models. TO se this we need to step away from the idea that a model must be one complete integrated whole. Instead we can view a model the same way as a source file and handle it as such. This has many advantages. I therefore think that the DSL Tools team should consider features in this area very carefully, since building them will take up considereble effort which I thin can be spent om more interesting features. I am interested whether other people have similar experiences.

    Thursday, June 15, 2006 6:32 PM

All replies

  • Jos,

    Interesting post, but I'd disagree with you on the subject of Views.

    I agree that having a monolithic model describing even one aspect of a large project is a problem. Such models are difficult to maintain and lead to 'locking' or contention issues.
    Breaking up such large models into small, composable pieces is a good strategy for avoiding those problems. I've taken that approach in the DSL I'm building, and despite the added complexity I think it is well worth doing. I agree with the other benefits you describe, such as version control, model re-use etc.

    But, IMO, that says nothing about Views.
    The model I am preparing describes application behaviour in certain environments. That's a useful thing to model and transform into code, but it is not a complete application. The complete application could include some combination of a behavioural model, a logical structure (like a class diagram or services description), a data structure, a deployment structure (possibly with seperate logical and physical views) etc.

    I think it makes little or no sense to try to capture all this information into any single model structure, so to solve the problem of Views we'll have to find a generic way of creating references between models.
    For example a deployment model might reference services or interfaces in a logical model and schemas in a data model in order to link those to deployment concepts like a server or network.

    So there are lots of interesting problems to solve in order to get useful views of an application working together. I know that is something the Software Factories team want to solve, so watch this space....

    Brian.

    Tuesday, July 4, 2006 4:05 PM
  • Brian,

    I tend to disagree with your opinion about views, simply because allowing multiple views (diagrams) on a model looks to me as a (single file) repository. Such a feature would encourage the development of huge DSL's that have much language features, and will most likely address multiple patterns.

    A DSL that rich will be more difficult to change or exchange. It might also loose scope, i.e. the area in which you can use the DSL, simply because a lot of the DSL might be relevant for the DSL user except of ....

    Still there is a need of sharing common data, so that:

    - data is not entered more than once

    - cross validation over models is possible

    - kind of intellisense can be offered

    The challenge is to implement the common data independend of one or more particular DSL's. The coupling between the DSL's should not be tight.  

    John

     

     

    Wednesday, July 5, 2006 6:44 AM
  • John,

    I think we agree on the important stuff here, just that the names are giving us trouble.

    When I talk of views here I'm not thinking of different aspects of a model, but different models describing the same software. I'm getting different views of the end result by looking at it though different models.

    Brian.

    Wednesday, July 5, 2006 10:21 AM
  • Hello Jos,

    Sorry for the long answer…

     

    About Semantics

    Maybe the semantics (what it is to be modeled) could be also considered.

    In addition, despite some UML disillusionment, some of these semantics might be seen from a global perspective. OK, we cannot represent precise semantics with UML because the UML meta-model is based on a fixed set of technical abstractions. But that is not a reason, I think, to give up with any kind of general consistency.

    To the contrary, while reconstructing from a new base, it would be a shame to not even consider the resolution of this critical issue: the plethora of inconsistent concepts and terms that prevent people with different background or cultures to communicate.

    There are very few and natural concepts from which all things are derived. These are very few but these are critical, I believe, to provide a basic ontology. It is critical not only for communication but also for relating things with each others consistently.

    The alternative could be chaos i.e. a plethora of disconnected heterogeneous DSLs across various organizations i.e. a maze of redundant concepts and terms.

     

    Relationships between model elements

    A repository is, I think, one technical solution to resolve the problem of model element relationships.

    Some types of relationships are very business specific. Some others are very recurrent to a type of viewpoint or to a type of refinement path.

    For instance, a bank account (or more exactly, the information about a type “bank account”) in a domain model might be owned by a customer. That relationship is very specific to some business.

    But some part of this same bank account can also be used in a scenario, in a form, in a message, in an operation parameter, in a database etc… This type of re-use, in the refinement paths, is very generic and recurrent.

    The semantics of these generic relationships need to be clearly defined in order to provide appropriate tool support including meaningful refinement, traceability, navigation, black boxing and white boxing , (automated) guidance etc…

     

    Model Size

    I agree we need many small domain specific models versus a big one. We need small, specific little things that are precise, unambiguous… something that we can cope with.

    UML and UML tools do not prevent us from building small models, but the small UML models are not domain specific. Hence they are not precise and they are ambiguous.

     

    Multiple Views on the Models

    I don’t think that “the need for views disappears automatically”. I think that views are critical so that we can “view” and navigate multiple types of relationships that one item has with other ones. See the example of the bank account above.

    Behavior also needs to be seen from different levels of granularity and refinement. Then we also need synoptic views on otherwise very detailed items a bit like zooming in and zooming out, black boxing and white boxing….

    If we have lasting memories about some item, it is because, in our brain, we associated this item with other ones. Loosing Views is a bit like loosing memory and visibility e.g. about the relationships that we did identify, about the designed decisions that we took in the refinement path. It is forgetting about the black boxing and white boxing that we have used to create a modular solution. It is forgetting about the business rationale for automating this process versus this other one….

    In my opinion, without multiple views it is quite hard to share, to communicate and to take appropriate decisions (except in the case of very small projects).

    Unless someone comes with other arguments, I am convinced that we need multiple views in order to manage complexity.

    I don’t believe that we will manage complexity by ignoring it. We just need simple ways to organize it.

    Alain

     

    Thursday, July 20, 2006 3:51 PM
  • Yes there is a need to share information between models, but that does not necessarily mean a repository.

    Let me illustrate this with an example . If you take visual studio, you can edit source code in a c# file. You will most likely use code written in other files (as all OO and other programmers do). The visual studio intellisence will help you by offering suggestions about the code in the other file. The intellisence does not have a central repository. It just reads the relevant code files in the solution real-time. So the code files themselves form the repository. The references to that source in your code file are soft, i.e. refered just by names (ascii names).

    I think there are similaraties between source code files and dsl models and the need to share data and intellisence:

    - A dsl model is comparable with a source code file. The graphical editor for the model is like the c# editor for the code file.

    - You might refer to information in other dsl models. The reference should be soft, i.e. references by names. There are big advantages of loosly coupling the data in models and thus the coupling of dsl's.

    - The shareable information must be determined real-time, just as the intellisence tool does. Somehow dsl's must be able to request this information to enable things like intellisence, cross model validation and other stuff.

    The big difference is that there can be multiple dsl's. They all target their own problem domain. This makes it more difficult to build a specific, easy to use API like the intellisence tool has.

    Anyway, this is the idea that we use to share data to support the above functionality and to generate code from multiple models that fits like a puzzle.  

    John

    Thursday, July 20, 2006 6:27 PM
  • Need Multiple types of relationships and multiple views on same model elements

    Here is a more detailed and significant example

    Let's say I have a very automated organization exposing a lot of Web Services to other companies and their system(s).

    Let's say that I want high quality data types to ensure the quality of my services.

    To do this I could create a type dictionary.

    In this type dictionary, I could define a "Simple Type" (using SWSD terminology) called BankAccountID with some specific constraints (that could be expressed using XML facets e.g.)

    This BankAccountID is used

    - as a parameter of some of my Web Service operations

    - in some of my database tables

    - as the type of a property in some C# classes

    - as a field of a paper-based form (for manual task)

    - as the type of field in some InfoPath or ASP form

    - as the the type of a field in a BizTalk message

    ....

    Obviously, this BankAccountID is involved in some behaviors e.g. to access a Bank Account and in some structure electrnically, but als to fill the form manualy....

    So there can be many views on the same item. And there can be many kinds of relationships that this item can have (taht needs to be viewed as well).

    Hence we need multiple views, but these shouldn't be create dby hand (like in UML).

    I hope that explains better

    Alain 

    Friday, July 21, 2006 9:09 AM
  •  Brian OByrne wrote:
    John,

    I think we agree on the important stuff here, just that the names are giving us trouble.

    When I talk of views here I'm not thinking of different aspects of a model, but different models describing the same software. I'm getting different views of the end result by looking at it though different models.

    Brian.

    Brian,

    I see one model, with different ways of looking at it. If you mean that views are different (related) models of the same system I think your definition of views is different from mine. I would call a view in your terms a model. I think we agree

    Jos

    PS: If natural language was unambiguous life would be easier, but probably less fun :-)

    Monday, August 7, 2006 7:56 AM
  • Sorry for the long answer…

    [No Problem, as long as you don't care about a long reply either }

    About Semantics

    Maybe the semantics (what it is to be modeled) could be also considered.

    In addition, despite some UML disillusionment, some of these semantics might be seen from a global perspective. OK, we cannot represent precise semantics with UML because the UML meta-model is based on a fixed set of technical abstractions. But that is not a reason, I think, to give up with any kind of general consistency.

    To the contrary, while reconstructing from a new base, it would be a shame to not even consider the resolution of this critical issue: the plethora of inconsistent concepts and terms that prevent people with different background or cultures to communicate.

    There are very few and natural concepts from which all things are derived. These are very few but these are critical, I believe, to provide a basic ontology. It is critical not only for communication but also for relating things with each others consistently.

    The alternative could be chaos i.e. a plethora of disconnected heterogeneous DSLs across various organizations i.e. a maze of redundant concepts and terms.

     <jos> With DSLs the very idea is that anyone can create his own language. This means that some degree of chaos is unavoidable. I remember when the first Mac computers became popular. It suddenly was so easy to use different fonts in a document that people started using dozens of foints on one page. This "fontitis" was chaos, but it disappeared after a while.

    I think we will have to follow a simílar learning path with DSLs. People will try out many different ways of defining DSLs. They will be good, bad, useful, useless, small, big, ugly, beautiful, etc. etc. After a while we will learn, as a community, lessons about what works and what doesn't. We cannot stop this (because everyone will have the tools to do this) and I think we shouldn't try. What we should do is collect idea's on how to define DSls and setup a list of best practices from the start (i.e. now!). This will allow us to speedup the learning process.  </jos>

    Relationships between model elements

    A repository is, I think, one technical solution to resolve the problem of model element relationships.

    Some types of relationships are very business specific. Some others are very recurrent to a type of viewpoint or to a type of refinement path.

    For instance, a bank account (or more exactly, the information about a type “bank account”) in a domain model might be owned by a customer. That relationship is very specific to some business.

    But some part of this same bank account can also be used in a scenario, in a form, in a message, in an operation parameter, in a database etc… This type of re-use, in the refinement paths, is very generic and recurrent.

    The semantics of these generic relationships need to be clearly defined in order to provide appropriate tool support including meaningful refinement, traceability, navigation, black boxing and white boxing , (automated) guidance etc…

     <jos> I agree that a repository is one technical solution. The problem with a repository is that the whole model becomes one huge graph. Everyting is connected to everything. This leads to problems of model management like model versioning, multiuser access etc. etc. In my experience with using UML tools I found that none of them could resolve these problems in a workeable way. We also have experience with some of our older model driven tooling with one huge repository. The problem there is that any single change anywhere in the model may have an unknown number of side-effects. The result is that full code generation from the repository and full testing of the whole system beccomes neccesary after each (even small) change. Our conclusiomn is that the repository solution does not scale up.

    In some of the discussions I see that people too easily take the repository solution and put it down as a requirement. The requirement however is that we must be able to have relationships and manage them. This can be done without a repository. In dealing with source code at the programming level we have been able to define and manage dependecies without repositories for many years. This inspired the idea of using a similar approach at the modeling level. This solution at least makes model management much easier to handle.

    Your remark about the fact that there might be different types of re-occurring relationships is interseting. If we can nail them down it will allow us to define the type of tool support we need. </jos>

    Model Size

    I agree we need many small domain specific models versus a big one. We need small, specific little things that are precise, unambiguous… something that we can cope with.

    UML and UML tools do not prevent us from building small models, but the small UML models are not domain specific. Hence they are not precise and they are ambiguous.

    <jos>

    The problems with UML is that there is no way to have different UML models and create relationships between them, something we agree that we need. We can create small diagrams in UML but behind the small diagrams we always have one big model and one big repository.

    </jos>

    Multiple Views on the Models

    I don’t think that “the need for views disappears automatically”. I think that views are critical so that we can “view” and navigate multiple types of relationships that one item has with other ones. See the example of the bank account above.

    <jos> I agree that navigating through relations between different models is useful. This does noit mean we neccesarily need views, we only need to be able to open the model in which the referred element is defined. </jos>

    Behavior also needs to be seen from different levels of granularity and refinement. Then we also need synoptic views on otherwise very detailed items a bit like zooming in and zooming out, black boxing and white boxing….

    <jos>The questioin I would ask is whether all these different granularity and refinement levels should even be in the same model. In the idea of small models, a refinement will probably be just another model, than can be managed seperately. All we need then is a relationship between the refinement model and the refined element. Also, a refinement model might be written in another DSL than the refined element, because it probably uses different (lower level or more refined) concepts. </jos>

    If we have lasting memories about some item, it is because, in our brain, we associated this item with other ones. Loosing Views is a bit like loosing memory and visibility e.g. about the relationships that we did identify, about the designed decisions that we took in the refinement path. It is forgetting about the black boxing and white boxing that we have used to create a modular solution. It is forgetting about the business rationale for automating this process versus this other one….

    <jos>I do not propose to forget about relationships at all. I agree this would be wrong. I do propose to look at relationships in a different way and to handle them in a different way without losing all the possibilities of navigating them.</jos>

    In my opinion, without multiple views it is quite hard to share, to communicate and to take appropriate decisions (except in the case of very small projects).

    Unless someone comes with other arguments, I am convinced that we need multiple views in order to manage complexity.

    I don’t believe that we will manage complexity by ignoring it. We just need simple ways to organize it.

    <jos> You are right, inherent complexity needs to be managed. However, we should avoid solutions that introduce additional complexity of their own. </jos>

    Alain

    <jos>Regards, Jos

    PS: having read the post by Brian I am wodering whether your definition of view is the same as mine </jos>

    Monday, August 7, 2006 9:06 AM
  •  Alain De Preter wrote:

    Need Multiple types of relationships and multiple views on same model elements

    Here is a more detailed and significant example

    Let's say I have a very automated organization exposing a lot of Web Services to other companies and their system(s).

    Let's say that I want high quality data types to ensure the quality of my services.

    To do this I could create a type dictionary.

    In this type dictionary, I could define a "Simple Type" (using SWSD terminology) called BankAccountID with some specific constraints (that could be expressed using XML facets e.g.)

    This BankAccountID is used

    - as a parameter of some of my Web Service operations

    - in some of my database tables

    - as the type of a property in some C# classes

    - as a field of a paper-based form (for manual task)

    - as the type of field in some InfoPath or ASP form

    - as the the type of a field in a BizTalk message

    ....

    Obviously, this BankAccountID is involved in some behaviors e.g. to access a Bank Account and in some structure electrnically, but als to fill the form manualy....

    So there can be many views on the same item. And there can be many kinds of relationships that this item can have (taht needs to be viewed as well).

    Hence we need multiple views, but these shouldn't be create dby hand (like in UML).

    I hope that explains better

    Alain 

    Alain,

    A simple answer: BankAccountID is defined with all neccesary details in one model. In all other models you can use a reference (by name) to BankAccountID. Inter-model validation enables you to validate that the referred BankAccount is defined in some model. This is not what I call a view, is this waht you would call a view?

    Jos

    Monday, August 7, 2006 9:11 AM
  • Hi guys,

    Very interesting discussion! I as your DSL client absolutly agree that it's necessary to have relations between metamodels, but there are several type of them:

    1. Reference, for example, domain property of one the DSL's element, type of this property is an element of the second DSL's metamodel. This type of relation is possible right now, you can create ExternalType, with name of the domain class's name and namespace of the referenced DSL's metamodel. Add Reference to assembly with external DSL model. Make that property as CustomStorage, and create separate file with referenced DSL model. It works fine. Also it is possible to create handler and invoke file openning when navigating to referenced DSL's model.
    2. Inheritance relation, it is very usefull when you want to support extentions for your model. For example, I have FileSystemDefinition DSL's metamodel. I have abstract element FileModel. And I want to have a possibility to create additional DSL's to extend this base metamodel by providing implementation of abstract FileModel element. And when the instance of my model is loaded into the designer, I want to see those new element inside ToolBox. There is almost the same example, but vice versa - I want to reuse SDM or CD elements, for an instance (when you implement them base on last version of the DSL )

    Evgeny

    Tuesday, August 8, 2006 8:29 PM