Sunday, June 29, 2008

Eclipse Ganymede - P2 Shared Installations (Bundle Pool)


Eclipse Ganymede and P2 will change your update workflows from ground up.

There are some Cons (I'm missing the external locations from Software Update),

but many Pros so for me its time to change.

If you don't want you can still use the old Update Manager. (Preferences - General - Capabilities)



There are many new things in P2 - today I'll concentrate on:

  • Dropins - a folder where you can place and structure plugins. These plugins were scanned at startup automagically by Eclipse P2.
  • Bundle Pooling - now its possible that some Eclipse Installations can share their bundles using the Bundle Pool.
In my project I'm using some Eclipse Instances:

  • Eclipse for Modeling (EMF, UML2, openArchitectureWare, ...)
  • Eclipse for OSGI Client/Server (Eclipse Riena, RCP, Equinox, EasyBeans, ...)
  • ....

The old classic way I had to download the Ganymede Packages for Modeling and also the Ganymede Packages for RCP/Plugin and then manage them separately.

Eclipse P2 allows bundle pooling and now we'll try to install this.

Attention: All installation examples in this blog are OSX specific - perhaps you have to make some fine tuning for your OS.

Shared Installations - first try

As good Eclipse „citizens“ at first we look at the Eclipse wiki and read from Equinox P2 Getting Started and Equinox P2 Installer and then we're ready to start:

Download the P2 Installers from here - we choose the Mac OSX version:
equinox.p2.installer-3.4-macosx.carbon.ppc.tar

Doubleclick to unzip the Installer into /your_path/e34p2installer and open the installer /your_path/e34p2installer/eclipse/p2installer.app:



We're choosing Shared install, select our Product install directory (/your_path/e34modeling) and start the p2 Installer.

It takes some time to download the SDK bundles - unfortunately I cannot select a prefered mirror and there's also no chance to select the Friends of Eclipse Mirror ;-)
see @ Bugzilla 215916

Suddenly the window of the P2 Installer disappears without any message - but lets think positive and start the new Eclipse from /your_path/e34modeling.

All runs well without any problem or error :-)

But its not the end of the story - otherwise there would be no reason to write this blog ;-)

Where are the shared plugins stored ? Where's my bundle pool ? You remember: there was no possibility to select the bundle pool location from p2 Installer.

Searching the Getting started under Bundle pooling we found an example:

Documents and Settings
Username
.p2/
org/eclipse.equinox.p2.core
org/eclipse.equinox.p2.director
org/eclipse.equinox.p2.engine
org/eclipse.equinox.p2.touchpoint.eclipse
plugins/
<-- shared bundle pool But there are no plugins - the plugins of the bundle pool were stored into the P2 Installer location: /your_path/e34p2installer/eclipse/p2/ org.eclipse.equinox.p2.touchpoint.eclipse/plugins/

Now you're in luck if you don't followed the advice to delete the P2 Installer („... When finished, you can delete the installer...“) ;.-)

We need a way to select the location of your bundle pool - lets start a next try:

Shared Installations - second try


Download and unzip the P2 Installer as described above and look into the p2installer.ini file (you'll find this file under /your_plugin/e34p2installer/eclipse/p2installer.app/Contents/MacOS/ p2installer.ini ).

Inside the p2installer.ini you'll find a link to a properties file. Please open http://download.eclipse.org/eclipse/testUpdates/sdk-installer.properties in your webbrowser and you'll see:

eclipse.p2.metadata=http://download.eclipse.org/eclipse/updates/3.4milestones/
eclipse.p2.artifacts=http://download.eclipse.org/eclipse/updates/3.4milestones/
eclipse.p2.flavor=tooling
eclipse.p2.profileName=Eclipse SDK
eclipse.p2.launcherName=eclipse
eclipse.p2.rootId=org.eclipse.sdk.ide
eclipse.p2.autoStart=true


Copy the content from the browser window into an empty textfile and add following entry pointing to the bundle pool:
eclipse.p2.bundleLocation=/your_path/e34pool

Save this textfile under /your_path/e34p2installer/sdk-installer.properties



Now we have to open the p2.installer.ini file to change the path of the sdk-installer.properties:

-Dorg.eclipse.equinox.p2.installDescription=
file:/your_path
/e34p2installer/sdk-installer.propertiesfile: /your_path/e34p2installer/


Start the P2 Installer, select the product install directory (/your_path/e34modeling - screenshot see above from first try)

The same happens again: the P2 Installer appliation disappears without any message - but Eclipse (from /your_path/e34modeling) runs without problems and now all shared plugins are stored in our bundle pool under /your_path/e34pool/plugins :-)
2008-07_02 add-on: its not ok, that the P2 installer disappears - you have to add more memory, delete all and start again. see additional info in blog P2 Installer Trouble

Now we want to install the second Eclipse installation (/your_path/e34osgi_rcp) and so we start again the P2 Installer, select our product install directory, choose Shared Installation and really - this time it takes only some seconds to "download" and install the Eclipse SDK :-)

Really great P2 !


But whats that:
This time the P2 Installer window remains open and asks if we want to start Eclipse SDK immediately.

After some tests it was reproducable: in shared installations the first time a bundle pool was used the p2 installer disappears without any message.

Both SDK installations run without any problems - About Eclipse - configuration details logs that all bundles come from our bundle pool.

Good work, P2 :-)

Configure Eclipse Installations

The first steps were easy - but now you have to do some work: - unfortunately there are no prepared shared Installs like Ganymede Packages (I have found none)



Instead of selecting a whole package or groups of plugins we have to choose and select them plugin for plugin: Please start your two SDK installations and select menu Help - Software Updates...


Installed Software shows the already installed Eclipse SDK.

Got to tab Available Software - Ganymede Update: there you'll find the needed plugins. P2 makes sure, that all dependencies will satisfied, only correct combinations will be installed - so we always have a correct installation running.

In my ERP project one Eclipse installation (/your_path/e34modeling) contains the plugins from Ganymede Modeling Package + ECF and the other Eclipse installation (/your_path/e34osgi_rcp ) contains the plugins from Ganymede RCP/Plugin Package + Ganymede Reporting + Subversion.

To get an idea whats needed you can take a look at the feature lists of the Ganymede Packages, per ex. for the modeling package:



After configuring both Eclipse Installations we can look at our bundle pool to see that all plugins are contained in the one and only bundle pool :-)

Would be really nice using Software Update to select bundles directly from your bundle pool. This would be an easy way to get bundles not contained in the actual Eclipse Installation, but already installed in another Eclipse application sharing the bundle pool.
So you have to select all bundles thru the updatesite - even if they are already contained in the bundle pool.

Dropins and Bundle Pool

After finishing the installations using Software Update... in many cases you have to install more (external) plugins (bundles).

You can use two different strategies:

  • Bundles from Update Sites (Menu Software Updates... - Available Software... Add Site...) - this works the same as in previous Eclipse versions, but without the possibility to store them at external locations
  • Bundles using the new Dropins folder (a „Watched Directory“). At startup Eclipse will scan this dropins folder automatically to see if there are new or changed plugins. All bundles from the dropins folder are added to the current Eclipse Installation.

This dropins folder will be found directly inside your Eclipse Installation directory parallel to the Eclipse Application.

You can also use an own extra dropins folder: simply add a parameter into eclipse.ini (/your_path/Eclipse.app/Contents/MacOS):

-Dorg.eclipse.equinox.p2.reconciler.dropins.directory= /your_path/e34shared_dropin

This extra dropins location can be used from more then one Eclipse installations - so you can use it as a shared dropins (watched directory).

Now we have all pieces together:



Each Eclipse Installation only uses those bundles from bundle pool, which were selected from Software Update..., also the bundles from the (default) own dropins folder and perhaps bundles scanned from an extra (shared) dropins location.

You can put bundles into a dropins folder in different ways as decribed into the Getting Started.

Using link files inside your dropins directory you can manage external locations, but you loose the possibility to install them from inside Eclipse using Software Update...


Shared Installations and Target Platforms

I found out some problems with shared installations and the use of target platforms (Preferences - PDE - Target Platform).

If you create a target platform only from external locations then there's no difference to previous versions of Eclipse because you select the bundles to use by yourself without using bundles from your current Eclipse application.

But if you dont want to configure the target platform completely and instead use your current Eclipse as target, then there are some bugs:

  • Using Eclipse as shared installation you cannot select the bundles configured for your current Eclipse application as target platform
  • Bundles from dropins locations are not recognized in shared installations

Using Shared Installations the default target platform points to the bundle pool listing ALL bundles from your bundle pool, so you see also bundles from other Eclipse installations in you default target and you have to deselect the not-used bundles:




I think there should be also an option to automatically „Disable bundles not used in current Eclipse Installation“.

Next problem with dropins in shared installations. Getting Started... notes „In other words, new plug-ins installed via the dropins folder behave exactly like plug-ins installed via the user interface.

Attention: If you're using a shared installation the bundles from your dropins are missing !
You only see all bundles from pool (and there you are seeing too much as described above) and you dont see the bundles from your dropins.

The only way to change this is using Add.. for all plugins folders inside your dropins. If you have structured your dropins, this means you have to select each plugins folder from there. It can happen easily to forget something or loosing the overview.

Shared Installations brings you much more comfort, but also more work with target platforms :-(

If you're using separate (not shared) installations then the behaviour is as aspected - the bundles from all your dropins plugins folders appear automagically in your target platform:



I hope, that these bugs will be fixed soon:

Bugzilla 236923 [installer] p2 shared installation does not works without the installer directory
Bugzilla 238947 P2 Installer Window disappears
Bugzilla 238949 P2 Shared Installations - Target Platform too much bundles
Bugzilla 238950 P2 Shared Installations - Target Platform Dropins missing

and some enhancements as well:

Bugzilla 238952 Feature Request P2 Software Update should be able to select from Bundle Pool
Bugzilla 238953 request P2 Installer select Bundle Pool Location

as always: if you're interested, then please Vote for the Bugzillas ;-)

Conclusion

Over all P2 is a great step into te right direction - and please remember: its the first release - so give it a try !

the blog in german

13 comments:

Carlus Henry said...

Thank you so much for posting this. I have been searching on the mailing lists as well as started discussions within the Eclipse IRC channels to find out how to use the bundle pooling features of Eclipse Ganymede.

I too believe that P2 is a step in the right direction, I just wish that Eclipse would have included a simple How-To or Migration steps moving from using Extension Locations to using Bundle Pooling.

I am very excited to attempt to use P2 the way that you suggested, in order to get the bundle pooling to work.

Thanks Again

ekke said...

hi carlus,

glad to hear that it worked for you.

ekke

philk said...

However, editing a properties file to get this work shows once again how overly complicated P2 is. And I thought the Software Installers of InstallShield were complicated to program.

ekke said...

philk,

you're right - at the moment some things are not easy to solve. perhaps you want to vote vor http://bugs.eclipse.org/238953 to avoid editing properties to change bundle pool location ?

I can understand that many developers still use the classic update manager, but on the other side without using P2 it will not become perfect ;-)

ekke

Tiran Kenja said...

A nice explanation. But once again it is underlined that P2 leaves behind some very practical workflows. Specifically sharing plugins between installs.

Okay okay. So you post here how to share plugins. But this way I can still only share all plugins or none. So if I have 3 installs where I want some plugins to only be shared by two of them. I am out of luck.

Sure I could go back and use extension locations. But updated plugins in an extension location will be put in the main Eclipse install and not back in the extension location. Making it pointless to use them unless you want to use the old update manager to do updates though as well. But I have no idea if it works with plugins managed by P2.

So basically P2 made things less flexible.

Sure it might be good for a first release, but it is not all good news. Some of us now have to work more to keep our installations up-to-date.

And yes, there are bugs posted on that update bug. But it has been ignored for almost 2 months now. (https://bugs.eclipse.org/bugs/show_bug.cgi?id=230373)

philk said...

Ekke,

I have not looked into P2 deeply yet. As a programmer I am usally not frightend by complex APIs or products.
But first looks at P2 did not show me how it should be easier to use from a programmers nor users point.

Of course you will always run into problems when there are 2 bundles exporting the same package and the update/install mechanism has to choose one bundle over the other or give the user to choose.

I am also aware of that if nobody is using P2 it will never come out of this un-usable state. But I stopped using it when the Eclipse updater threw tons of error messages at me that not even I as a programmer could understand let alone resolve. I will surely not offer my customers such mechanism. Luckily we can still use the old Update Manager.

ekke said...

for me the worst thing are the missing external locations supported by software update. I'm really missing this flexibility.
I also tried using more then one extra dropins folder putting a comma-separated list of directories to -Dorg.eclipse.equinox.p2.reconciler.dropins.directory, but it seems to work only with one.

if I had to make rollouts to users - I would use the old UM at the moment.

my ERP soluton is work-in-progress and I can use 3.5 Milestones and so I have decided trying to work with P2 now.

I'll report my experiences and my workflows.

yes, there are many bugs open for P2 - but I noticed also a big step even from 3.4M7 to 3.4 RC4

ekke

sud said...

Thanks for taking the time to post this. I wonder if this can be integrated into the P2 documentation on the Eclipse wiki.

I went bundle hunting as well till I figured out that the pooled bundles were installed in the p2 installer's folder. And guess what happens when you delete the installer (as directed by the P2 wiki instructions)!?

These are certainly areas that Eclipse documentation can be improved.

Benjamin Cabé said...

Ekke, this is really a great post, thanks!

It just makes me a bit sad to see your thread on eclipse.technology.equinox newsgroup and observe that odd workflow:
• 20080627: Ekke asks Q1
• 20080628: Ekke self answers to Q1 + Q2
• 20080629: Ekke self answers to Q2 part1
• 20080629: Ekke self answers to Q2 part2
• 20080630: Ekke posts a link to his blog. And still not a single answer (or at least a link to a wiki article or whatever...) from the p2 team :-/

That said, did you try to add plug-ins into dropins/ at runtime? The doc (and what I understand of the source code of the reconciler plug-in) says that it's automatically watched at runtime, but I can't make it work :-/

ekke said...

additional info: its not ok, that the P2 Installer sometimes disappears without any message. its a memory problem, I've written an additional blog just at http://ekkes-corner.blogspot.com/2008/07/eclipse-ganymede-p2-installer-trouble.html

ekke

ekke said...

benjamin,
I also tried to get changes from dropins while running the application but found no way.
the dropins are only scanned after re-starting.
perhaps you'll open a bug ?
ekke

jwb said...

I wonder if there is a feature that I need to install onto the base Ganymede platform that will enable the Preferences->General->Capabilities tab. I'm trying to build a (non-Java) development environment, so I loaded the 3.4 platform runtime release. My General preferences start with 'Appearance', 'Compare/Patch', 'Content Types'. It seems some of the packages that I need for my environment can be installed if I enable the old Update Manager, Aptana in particular.

It seems I may have some other issues with the new installation, since I've seen a few messages that might be from inadequate heap size on my WinVista installation.

Looking for advice,
John

ekke said...

jwb, sorry - I always have a java development environment and work on OSX - so I cant help you with your questions.
ekke