Thursday, April 30, 2009

If Only it Were That Simple

Simplicity is a holy grail touted as an ideal for which to strive. Keep it simple stupid, KISS, is sacred doctrine. Yet is the world a simple place? Are the problems you are solving simple? Are Java or C++ simple languages? In fact, do simple things ever stay simple? After all, C evolved into C++, and Java too evolved to support generics and enumerations, and those clearly didn't make the languages simpler. Consider too why it is that a simple mind is not a good thing despite the fact that simplicity is such a good thing. The reason is simple: extremes are typically not optimal. In the end, there needs to be a delicate balance between simplicity and complexity. I certainly don't want my garden to be simple!

Consider for example a Turing machine. With an optimally simple set of symbols, i.e., only zeros and ones, it's effectively impossible to come up with a simpler computational model. So why don't we all flock to this simplicity? Because it's a mirage. The expression of our solution isn't helped by the simplicity of the Turing machine, rather it's hurt by the lack of expressiveness. It's for this very reason that I bristle when I hear people say that EMF is too complex. I'll argue that it's neither too simple nor too complex, it's a higher level abstraction and it's just right. If you can do better, make my day sunny.

So many people are so used to solving their problems with rocks and pointed sticks that the mere thought of more sophisticated technology scares them away. This situation is particularly ironic because these same people are typically working on foisting some synthesized technology of their own on their unsuspecting end users. Of course everything they produce is simple, right? It's always the other guy who produces complexity. After all, I understand my stuff, so therefore it's simple, right?

Here is what I consider to be the gist of rocks and pointy sticks argument. Why would I need to build a big fire with my pointy sticks, and smelt my rocks to render metal, just so I can make a better rock or pointy stick? It's such a complicated process and it's so much extra work given that I can just smack my problems with rocks and poke them with pointy sticks already. Well, guess what folks, some of us have seen a way to move beyond the stone age caves, but feel free to hang out in yours, if that's where you feel most comfortable.

I totally agree with Michael Scharf's blog about the problem of How to Explain EMF? How do you describe something that's both a floor wax and a dessert topping? It's just not that simple. The responses to his blog were very interesting too, and while there is underlying truth in them, they typically also miss the point a bit. Consider for example that given an XML Schema, or simply annotated Java interfaces, EMF will generate a fully functional Eclipse IDE or RCP application. You don't have to know much of anything about Eclipse, and poof, there you go, a running application. That's pretty simple. It takes about a minute. But then you start to dissect it and realize oh my goodness, there's a lot going on here and gosh but it all seems so complicated. That's right because it's a complicated problem, yet would it have been simpler to have spent the months it would have taken to learn how to get all that working from scratch? No it would not! But feel free to hang out in your cave chipping stones and sharpening sticks if that gives you a sense of purpose.

Even with modeling's layered architecture, people complain that all the layers are confusing. Granted very little of modeling is well documented, the marketing message is poor, and the web organization leaves much to be desired, but please people, is there an alternative to layered architecture? Don't forget too that EMF has a book. I think this really is Michael's fundamental point, i.e., poor marketing. Rather than polishing the technology, which as developers is what we (and most of all I) like best, we need to address this marketing problem with a higher level of priority.

There's even the premise that modeling is good but EMF, not so much. The argument is actually stronger than that: EMF stifles modeling and harms the ecosystem. I think the important point being missed here is the extreme value of "the one model to bind them all," i.e., Ecore/EMOF. The modeling ecosystem is rife with other meta models---gosh but I dislike using the word meta. For example, there's UML, BPMN, XML Schema, and so on. This reality is not the hallmark of a constrained ecosystem. In each case, the value of the model is enhanced by the fact that Ecore/EMOF is the underlying meta meta model---gosh, two metas in a row, I should be shot.

Here's a fundamental assertion I'm sure will never be proven wrong: if you tried to replace Ecore/EMOF and the rest of EMF's core with something better, you'd merely end up with another EMF; only if you're lucky (people care about what you've done) and highly skilled (you've learned from EMF's successes and mistakes) might it actually be relevant and a bit better. In any case, feel free to build another ecosystem if you're feeling lucky and skillful. Don't forget to take sociology into account; you're likely better off in an established ecosystem.

No doubt it's true that I'm way too steeped in this technology to understand the perspective of someone looking at it for the first time, but surely there have been plenty of people looking at it for the first time that just a few of them might have tried to help out the next guy with their fresh insights; kudos to Michael on that front! E.g., these are the five things that took me weeks to grasp and would have been nice to have realized up front. Ultimately, simplicity like beauty is in the eye of the beholder. May you always see simplicity where others see complexity so you can lead the way forward.


Mowe said...

I absolutely agree with you. Many people tend to ignore the intrinsic complexity of problems. They ask for simple solutions instead of asking for a good management of complexity.

In a discussion about usability issues (regarding a very specialized RCP-based tool) someone pointed out that "simple interfaces" such as the typical search-engine UI would be a the path to follow. Just a textbox, a button and that's it. However, the "just-think-for-keywords"-paradigm was absolutely inapplicable for the tool we discussed.

This is somewhat like pointing out that a common keyboard is too complex and one should establish alternative devices with just 3 keys (sine this works so nice for game-devices...).

Of course one can introduce complexity "artificially". But that's a matter of misusing technologies and methodologys.

To make a long story short: EMF does an excellent job in managing complexity.

Peter Severin said...


I'd like to offer the view of someone who uses EMF. Before EMF I've always thought that modeling is something reserved to men with "architect" in their title working for large corporations. Something that offers little practical value. This view was completely changed by EMF.

It's true that it took me some time to get to this point. But I saw what kind of hell can be an ad-hoc Java model maintained manually. The model is the heart of a project. If the model is ill, the project isn't well either.

In my head it was clear that the solution is a declarative approach with a generative component. I've almost started implementing a more simplistic version of EMF myself. When I found EMF I've had an instant aha moment. I look now in disbelief at all those years I've lived without it.

It's true that EMF is not simple. There are many different layers in there and one needs a good understanding to choose what he needs from them. But the practical value is real. It's not just a buzzword. And no, I don't think it could be simpler.

Ed, while I am at it I'd like to thank you for all your work. EMF is truly the light in the ocean that is modeling :)

All the best!


David Carver said...

Simplicity does not necessarily mean that something can't evolve to the complex. The phrase tossed around in Agile circles is "design what you only need now, not what you might need in the future." This doesn't mean that the design can't evolve or be refactored to add further features when you need it. It means we design for what the requirements are today right this instance not the nice to haves or it would be cools. Too much up front design can be bad, too little can back you into corners as well.

Models like anything else need to evolve and grow. They change as new features are required. It doesn't mean you have to design everything into the model that you might ever need. Thus the KISS principle does apply. But it is KISS done incrementally. Each time only adding what is needed.

EMF has evolved from the simplistic to the complex I believe following this same approach. So designing simply doesn't mean that a framework over the years may not contain the complex. It will get there when the requirements ask for it and no sooner.

How much stuff is put into models that is never used because of the thought "we might need this, or that"?

Ioan Salau said...

I also think that KISS is overrated, sometimes used by obscure vendors to sell their narrow solutions as a "good to everything" solutions. Sometimes you need to hit the wall in order to realize that the doors are the best way to get in. Not too far ago, I used a custom build modeling framework based on Java. The results were quite satisfactory but for a long term solution, I realized that soon enough I will have to spend more time on maintaining the modeling framework than focusing on my real business problems. Therefore, 8 months ago, I start doing some research to see what is available on the market and I found EMF. Even though I was working before with Eclipse and install/use plugins, I never designed/implement an Eclipse plugin, so it took me a while to understand the underlying mechanism required to build non-trivial solutions based on EMF projects. Is not easy and it takes some effort but in the end, I am quite happy with the EMF rich family of technologies. Even though many of these technologies are still in inception phase, I am pretty excited with the directions and capabilities of EMF projects, mainly because is a live and committed community to deliver innovative ways of designing and implementing software applications. Now the “not so good” things about EMF: marketing strategy and lack of consistent, non-trivial examples on how to use multiple EMF projects together. If the marketing strategy is really Eclipse’s brilliant minds concern, what is really concerning me, is the second aspect regarding non-trivial examples. Although, “ Eclipse Modeling Project as a DSL Toolkit” book recently published, does an excellent job trying to fix this problem, but you can also observe certain aspects that makes EMF usage harder than is suppose to be: EMF, GMF and TMF defines editors with minimal effort, but when you try to combine all this editors for a particular DSL, you realize that these editors are generated based on different principles and to make them work on synch you have to hack so much code. I am pretty sure that this aspect is going to be fixed in future releases, but as is right now, the synergy between projects is not quite there.

Vincenzo Caselli said...

Hi Ed,
we bought the book when it came out some months ago with latest version, after having had a taste of EMF at the Eclipse Summit Europe 2008 in Ludwigsburg.

Well, generally I am quite slow to catch with advanced technologies: I prefer iBATIS to Hibernate, tend to avoid EJB stuff and often I wonder if GWT on a simple servlet container like Tomcat or Jetty is all what I need. Very occasionally I even think I was more productive when I was a Delphi developer twelve years ago.
But with EMF things are entirely different!
EMF is not about complexity, but quite the opposite, it's all about simplifying complexity.
Designing the (ecore) Model visually (my preferred way to start) is fast, easy and intuitive. For the first time ever, every domain expert without a development know-how can design the model, isn't it just fantastic?
And what about model maintainability? Changing or even refactoring the model is not the often frustrating experience of the past, it's even funny.

Further, using EMF with Teneo persistence when developing a RCP application makes the building of a desktop application a straightforward and rapid process. After years of waiting we've got RAD again (we need just the Visual Editor on the road again, but we know it is almost fixed, thanks to Nick Bolt and Yves Yang).
Sure, a true client/server, scalable, development deserves a more complex architecture, but we got it: it's Eike Stepper's CDO.

In the end EMF is a revolutionary way to manage the model and upon EMF many other great frameworks can grow very well.

Thank you Ed for your work!

Vincenzo Caselli
Francesco Guidieri

Thomas Fletcher said...

I'll comment as someone who really appreciates the claimed virtues of modelling, but has yet to apply them to a project.

I recently (< 5 months ago) started work on a new project that is very much driven by a well defined model and has a graphical editor component. I thought it would be a great place to finally get going with EMF.

I spent two weeks trying to mesh my mental model of the product and EMF together and found that it was very slow going. I have no doubt that something would have come together for me, but the market pressure of delivering milestone drops that provide functionality resulted in having to drop EMF in favour of a hand developed solution.

I haven't given up on EMF or modelling, but at this point, the learning curve was simply too steep (and I even have the EMF book!)


Alex Ivanov said...

Yep, I think it could be simpler. I should be shot. :)

The magic word is "virtual machine". It seems the only tool can significantly simlify (or hide) complexity & rise level of abstraction is VM. Especially for newbies.

Other (frameworks, libs) tend to be more complex.

What do you think about Java Virtual Machine with built-in EMF support?

Alex Ivanov said...

More precisely, Java Virtual Machine, Java tools and Java Language with built-in EMF stuff and EMF support.

Ed Merks said...

A modeling virtual machine would be very cool indeed! In the meantime, EMF will make do with generated Java.

Stephan Zehrer said...

I think we don't need a new Java VM a new language with a compiler which build java byte code would be sufficient ... we see it with Scala you can build a complete new language on the top of Java VM.

But nevertheless I am learning at the moment EMF and I can say, reading the book is helping much, all the other documentation (I found yet) was not so helpful as the book.
It is hard to learn something new without the knowledge of the domain, I think everybody who designs models knows this. The same thing here, without the basics all seems complex but once you understand some basic principles the story is quite simple ...

So from my side a cool framework and it is closing the gap between Java language and the Unified Modeling Language (UML).

Thanks for you work!