Thursday, June 25, 2020

Eclipse JustJ

I've recently completed the initial support for provisioning the new Eclipse JustJ project, complete with a logo for it.


I've learned several new technologies and honed existing technology skills to make this happen. For example, I've previously used Inkscape to create nicer images for Oomph; a *.png with alpha is much better than a *.gif with a transparent pixel, particularly with the vogue, dark-theme fashion trend, which for old people like me feels more like the old days of CRT monitors than something modern, but hey, to each their own. In any case, a *.svg is cool, definitely looks great at every resolution, and can easily be rendered to a *.png.

By the way, did you know that artwork derivative of  Eclipse artwork requires special approval? Previously the Eclipse Board of Directors had to review and approve such logos, but now our beloved, supreme leader, Mike Milinkovich, is empowered to do that personally.

Getting to the point where we can redistribute JREs at Eclipse has been a long and winding road.  This of course required Board approval and your elected Committer Representatives helped push that to fruition last year.  Speaking of which, now there is an exciting late-breaking development: the move of AdoptOpenJDK to Eclipse Adoptium.  This will be an important source JREs for JustJ!

One of the primary goals of JustJ is to provide JREs via p2 update sites such that a product build can easily incorporate a JRE into the product. With that in place, the product runs out-of-the-box regardless of the JRE installed on the end-user's computer, which is particularly useful for products that are not Java-centric where the end-user doesn't care about the fact that Eclipse is implemented using Java.  This will also enable the Eclipse Installer to run out-of-the-box and will enable the installer to create an installation that, at the user's discretion, uses a JRE provided by Eclipse. In all cases, this includes the ability to update the installation's embedded JRE as new ones are released.

The first stage is to build a JRE from a JDK using jlink.  This must run natively on the JDK's actual supported operating system and hardware architecture.  Of course we want to automate this step, and all the steps involved in producing a p2 repository populated with JREs.  This is where I had to learn about Jenkins pipeline scripts.  I'm particularly grateful to Mikaël Barbero for helping me get started with a simple example.  Now I am a pipeline junkie, and of course I had to learn Groovy as well.

In the initial stage, we generate the JREs themselves, and that involves using shell scripts effectively.  I'm not a big fan of shell scripts, but they're a necessary evil.  I authored a single script that produces JREs on all the supported operating systems; one that I can run locally on Windows and on my two virtual boxes as well. The pipeline itself needs to run certain stages on specific agents such that their steps are performed on the appropriate operating system and hardware.  I'm grate to Robert Hilbrich of DLR for supporting JustJ's builds with their organization's resource packs!  He's also been kind enough to be one of our first test guinea pigs building a product with a JustJ JRE.  The initial stage produces a set of JREs.


In the next stage, JREs need to be wrapped into plugins and features to produce a p2 repository via a Maven/Tycho build.  This is a huge amount of boiler plate scaffolding that is error-prone to author and challenging to maintain, especially when providing multiple JRE flavors.  So of course we want to automate the generation of this scaffolding as well.  Naturally if we're going to generate something, we need a model to capture the boiled-down essence of what needs to be generated.  So I whipped together an EMF model and used JET templates to sketch out the scaffolding. With the super cool JET Editor, these are really easy to author and maintain.  This stage is described in the documentation and produces a p2 update site.  The sites are automatically maintained and the index pages are automatically generated.

To author nice documentation I had to learn PHP much better.  It's really quite cool and very powerful, particularly for producing pages with dynamic content.  For example, I used it to implement more flexible browsing support of download.eclipse.org so that one can really see all the files present, even when there is an index.html or index.php in the folder.  In any case, there is now lots of documentation for JustJ to describe everything in detail, and it was authored with the help of PHP scaffolding.

Last but not least, there is an Oomph setup to automate the provisioning of a full development environment along with a tutorial to describe in detail everything in that workspace.  There's no excuse not to contribute.  While authoring this tutorial, I found that creating nice, appropriately-clipped screen captures is super annoying and very time consuming, so I dropped a little goodie into Oomph to make that easier.   You might want to try it. Just add "-Dorg.eclipse.oomph.ui.screenshot=<some-folder-location>" to your eclipse.ini to enable it.  Then, if you hit Ctrl twice quickly, screen captures will be produced immediately based on where your application currently has focus.  If you hit Shift twice quickly, screen captures will be produced after a short delay.  This allows you to bring up a menu from the menu bar, from a toolbar button, or a context menu, and capture that menu.  In all cases, the captures include the "simulated" mouse cursor and starts with the "focus", expanding outward to the full enclosing window.

The bottom line, JustJ generates everything given just a set of URLs to JDKs as input, and it maintains everything automatically.  It even provides an example of how to build a product with an embedded JRE to get you started quickly.  And thanks to some test guinea pigs, we know it really works as advertised.


On the personal front, during this time period, I finished my move to Switzerland.  Getting up early here is a feast for the eyes! The movers were scurrying around my apartment the same days as the 2020-06 release, which was also the same day as one of the Eclipse Board meetings.  That was a little too much to juggle at once!

At this point, I can make anything work and I can make anything that already works work even better. Need help with something?  I'm easy to find...

4 comments:

del65 said...

I struggled to embed a JRE into a p2 repository with Tycho for an Eclipse RCP product.

Thankfully I found some help with the p2 documentation to bundle the JRE into a feature ( following the advice from https://www.eclipse.org/forums/index.php/t/1074080/), and used touch points to alter the eclipse.ini file to point to the appropriate target VM when installing/uninstalling the feature (which is set to allow only one installed feature at a time).

But... on Windows dropping an unused feature or plugin may require a restart of the whole operating system to remove currently accessed files.
As a result disabling a feature is better than removing it, but p2 can only uninstall (which implies deletion of) a feature ( discussion : https://www.eclipse.org/forums/index.php/t/261827/ ).

It is also better to keep some sort of fallback in case of failure of the startup following the install of the new JVM. Restart failures can come from old JVM options in eclipse.ini which become incompatible with the new JVM, or a plugin which doesn't start anymore with the new JVM (because it relies on the presence of a jigsaw module which is not available anymore).

Does the JustJ proposal include a solution for those drawbacks ?

Ed Merks said...

In general, p2 only does the deletion after a restart of the application, i.e., it's really just an automatic garbage collection phase shortly after a restart. Also, the Eclipse Installer uses a shared bundle pool by default, and the shared pool is only garbage collected manually by the user. So nothing is ever deleted automatically in that case.

Also note these bugs have been fixed:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=559148
https://bugs.eclipse.org/bugs/show_bug.cgi?id=551378

In any case, I would expect if the restart fails, no garbage collection would be done, so the older feature/plugin containing the JRE would not be deleted yet.

In addition, the JustJ installable units specify the actual package capabilities of the JRE, so if bundles at least use package imports, missing packages would cause a resolution failure rather than a subsequent runtime (wiring) failure. But of course most plug-ins don't specify package imports for JRE-provided packages...

There are no special solutions being proposed, i.e., nothing that would require additional implementation support in p2 itself. The JustJ JREs just use what p2 already provides.

wimjongman said...

Thanks, Ed, really nice. It comes at just the right time for us. Also, JET.. It has been generating our database classes since 2008. A joy to work with.

de travail en France said...

Je n'ai pas du tout la même opinion, mais tout le monde a le droit d'avoir son propre avis. Le dialogue est important !