The Equinox p2 provisioning framework

Easier Eclipse configurations with Galileo

With the Ganymede release, the Eclipse Update Manager has been replaced by the Equinox/p2 provisioning framework. Author Nathan Good gives a high-level overview of the framework, and discover its benefits for users and update site builders.

Share:

Nathan A. Good, Senior Information Engineer, Freelance Developer

Nathan GoodNathan A. Good lives in the Twin Cities area of Minnesota. Professionally, he does software development, software architecture, and systems administration. When he's not writing software, he enjoys building PCs and servers, reading about and working with new technologies, and trying to get his friends to make the move to open source software. He's written and co-written many books and articles, including Professional Red Hat Enterprise Linux 3, Regular Expression Recipes: A Problem-Solution Approach, and Foundations of PEAR: Rapid PHP Development.



24 November 2009

Also available in Japanese Portuguese

The Equinox/p2 provisioning framework, which replaces the Update Manager in pre-Galileo releases of the Eclipse IDE, can be a frustrating change for plug-in and update site builders because the tools (GUI tools, like editors for installable unit [IU] files, as well as Ant tasks and command-line tools) are not as mature as the other Eclipse tools. However, the p2 framework has a lot to offer end users. This article focuses on using two GUI tools — p2installer and p2agent — to install Eclipse and build Eclipse profiles.

Frequently used acronyms

  • GUI: Graphical user interface
  • IDE: Integrated development environment
  • JAR: Java archive
  • SDK: Software development kit

The Equinox/p2 framework also introduces changes to the way Eclipse components are packaged. New terms used to describe some of these changes in metadata information and how it's used include:

  • Installable units— Metadata that provides information, such as name, version, and requirements. The information contained in the IU allows the Equinox/p2 framework to install the products. The IU Editor is only currently available in a developer project by checking it out using Subversion (SVN). (See Resources for details.)
  • Profiles— In the Equinox/p2 framework, features, and plug-ins are the same as they were in prior versions of Eclipse. However, you can install them into profiles, which is a new concept with Equinox/p2. By creating different profiles for Eclipse, you can have completely different instances, or configurations, of Eclipse set of in different ways but using common plug-in files.

Using Equinox/p2 provisioning inside the Eclipse workbench

The Equinox/p2 provisioning framework is a replacement for what used to be the Update Manager, so when updating Eclipse from within the IDE, you use the standard Help > Check for Updates in Galileo. To install new software, click Help > Install New Software.

New in the Equinox/p2 provisioning framework, however, is the notion of drop-ins, which are plug-ins that are "dropped into" a directory location while in either an archived file (JAR) or in the standard directory structure for plug-ins and features. When Eclipse starts, it scans the dropins folder for new plug-ins and installs them. An important characteristic to note is that the Equinox/p2 developers have adopted the assumption that if it installs correctly, it also runs. So, any plug-in located in the dropins folder and has dependencies resolved will be installed when Eclipse starts.

The basic structure of the dropins folder is designed to make removing plug-ins that were installed that way even easier. The dropins folder also supports nesting the directories into structure that makes the dropped-in plug-ins easier to organize. Consider Listing 1, which shows the dropins folder location for a fresh installation of Eclipse.

Listing 1. The contents of the dropins folder
dropins/

0 directories, 0 files

Listing 2 shows how Subclipse V1.6.x (from the archived site file) and the Regex Util plug-in (see Resources) are installed in the directory. After the files were put into the dropins folder, Eclipse installed them the next time it started. I could not find them in the Help > About installation information, suggesting that they can only be removed by deleting them from the dropins folder.

Listing 2. The contents after unpacking files into dropins
dropins/
|-- regexutil
|   `-- eclipse
|       `-- plugins
|           `-- com.ess.regexutil_1.2.4.jar
`-- subclipse
    `-- eclipse
        |-- features
        |   |-- com.collabnet.subversion.merge.feature_1.10.0.jar
        |   |-- com.sun.jna_3.0.9.jar
        |   |-- org.tigris.subversion.clientadapter.feature_1.6.4.1.jar
        |   |-- org.tigris.subversion.clientadapter.javahl.feature_1.6.4.1.jar
        |   |-- org.tigris.subversion.clientadapter.svnkit.feature_1.6.4.jar
        |   |-- org.tigris.subversion.subclipse.graph.feature_1.0.7.jar
        |   |-- org.tigris.subversion.subclipse.mylyn_3.0.0.jar
        |   |-- org.tigris.subversion.subclipse_1.6.5.jar
        |   `-- org.tmatesoft.svnkit_1.3.0.5847.jar
        `-- plugins
            |-- com.collabnet.subversion.merge_1.10.0.jar
            |-- com.sun.jna_3.0.9.jar
            |-- org.tigris.subversion.clientadapter.javahl.win32_1.6.4.jar
            |-- org.tigris.subversion.clientadapter.javahl_1.6.4.1.jar
            |-- org.tigris.subversion.clientadapter.svnkit_1.6.4.jar
            |-- org.tigris.subversion.clientadapter_1.6.4.1.jar
            |-- org.tigris.subversion.subclipse.core_1.6.5.jar
            |-- org.tigris.subversion.subclipse.doc_1.3.0.jar
            |-- org.tigris.subversion.subclipse.graph_1.0.7.jar
            |-- org.tigris.subversion.subclipse.mylyn_3.0.0.jar
            |-- org.tigris.subversion.subclipse.ui_1.6.5.jar
            `-- org.tmatesoft.svnkit_1.3.0.5847.jar

7 directories, 22 files

Shortly after learning about this feature, I had already utilized it in a team environment by building an archive file of all the plug-ins my team uses on our project. After verifying that the licensing for the plug-ins allowed me to do so, I was able to share the archive that I created with other developers, who unpacked it into their dropins folder for a very easy installation.

Note: The dropins folder is not shared between Eclipse configurations (profiles in Equinox/p2 terms).


Installing Eclipse through p2installer

The p2installer is a stand-alone Eclipse installer (see Resources to download). The installer is a lightweight Standard Widget Toolkit (SWT) graphical installer that allows you to install Eclipse without downloading the entire package first.

After downloading the installer, run the p2installer application. The installer appears (see Figure 1), and you will have the ability to choose either a stand-alone installation or a shared installation.

Figure 1. The p2installer
Screenshot showing that the installer appears

A stand-alone installation installs Eclipse like older versions of Eclipse — the entire IDE, including plug-ins, will be installed into one folder. In contrast, the shared installation puts plug-ins in a common folder you can share across instances of Eclipse. This can create a large space savings on your hard disk if you have multiple Eclipse installations, such as one for reporting tools; one for PHP development; one for Java™ Platform, Enterprise Edition (Java EE) development; etc.

Enter a folder name to use for the Eclipse installation, then click Install. The installer downloads the necessary packages before continuing with the installation. When the packages have been downloaded and installed, the installer prompts you to launch Eclipse (see Figure 2).

Figure 2. The p2installer prompts you to launch Eclipse
When the packages have been downloaded and installed, the installer prompts you to launch Eclipse

Managing installations through p2agent

Similar to p2installer, p2agent allows you to administer Eclipse installations outside of Eclipse. However, p2agent provides a more administrative and less user-focused set of tasks. With the p2agent, you can define profiles. At a high level, a profile is an instance of Eclipse.

By following this example, you'll set up two profiles for Eclipse: one for PHP development and one for Java development. Both of them will share quite a few common features and plug-ins, such as the set of features used for the base Eclipse platform:

  1. Use p2agent to install the Java version of the IDE by downloading the tool (see Resources) and running it. The Eclipse Provisioning application opens (see Figure 3).
    Figure 3. The p2agent
    The Eclipse Provisioning application opens

    The list is pretty long, so I found it easier to find things by organizing the list according to category.

  2. Change the organization setting by clicking Window > Preferences and looking in the Provisioning Admin settings (see Figure 4). Select Show UIs by category in repository views.
    Figure 4. Modifying the settings for p2agent
    Change the organization setting by clicking Window > Preferences and looking in the Provisioning Admin settings
  3. Add a new profile by clicking Add Profile (see Figure 5).
    Figure 5. Adding a profile
    Add a new profile

    The Add Profile page (Figure 6) opens to allow you to type more information about the new profile.

    Figure 6. Entering the profile details
    The Add Profile page opens to allow you to type more information about the new profile
  4. Type a name to use for identifying the profile, like JavaIDE.
  5. For the install folder, type the name of a folder that will be used for the product installation. When you're done, you will be able to run the product from this location.
  6. Type a value for the Bundle pool location. This directory will hold the shared plug-in files.
  7. Type a value for Name and one for Description, and leave the rest of the values at their defaults. When you are finished, click OK. The new profile now appears on the Profiles tab (see Figure 7).
    Figure 7. The new profile
    The new profile now appears on the Profiles tab

    The newly created profile is not ready for use yet because it doesn't have any features added to it.

  8. To create the Java IDE, click org.eclipse.sdk.ide in the Metadata Repositories view, then click Install. Select the JavaIDE profile from the list and click OK (see Figure 8).
    Figure 8. Installing to the JavaIDE profile
    Select the JavaIDE profile from the list and click OK

    After you click Install, the Install window opens. You can select or clear features before clicking Next (see Figure 9).

    Figure 9. Selecting features to install
    You can select or clear features before clicking Next

    The next window lists all the features that are going to be installed with the IU (see Figure 10).

    Figure 10. The features that will be installed
    The features that will be installed
  9. If you agree to the terms of the license, accept them, then click Finish (see Figure 11).
    Figure 11. Accepting the licensing terms for the features
    If you agree to the terms of the license, accept them, then click Finish

The p2admin installer begins downloading the packages and installing them in the locations you've defined for the profile. After installation, you can navigate to the folder you defined for the install folder and start Eclipse.

Installing the PHP IDE

To create a new PHP profile, open the p2agent. Follow the procedure above, but this time, after you add a new profile, type PhpIDE as the ID. Use a different folder for the install folder location, but use the same location for the common folder as you used before.

First, install just the Eclipse Platform IDE, which is org.eclipse.platform.ide. Notice that not nearly as many files are downloaded, and the installation is much faster. When the installation is complete, install the PHP SDK feature by selecting it. (If you have it sorted by category, the feature is in the Web, XML, and Java EE Development category.) Open the context menu on the PHP SDK feature (org.eclipse.php.sdk.feature), and click Install. This time, choose the PhpIDE profile that you set up.

You can see that using the common shared folders makes quite a bit of difference in the overall file size (see Listing 3).

Listing 3. The file size savings using common files
$ du -hs ./*
259M ./common 
4.0M ./javaide 
6.8M ./phpide

Without using the concept of a shared plug-in folder location, the installation would have been nearly twice as large. If you install another different profile — such as a modeling or Business Intelligence and Reporting Tools (BIRT) profile — you'll save much more disk space.

Installing other plug-ins should, in theory, work exactly the same as installing the plug-ins from the Eclipse repository. However, because Equinox/p2 support is relatively new, repository support has not yet been added to other plug-ins' repositories. When trying to install other plug-ins using non-Eclipse update sites, the p2agent threw errors and was not able to complete the installation.


Enabling your own update sites for Equinox/p2

The Equinox/p2 framework uses a different format for update sites because it uses different metadata from prior update site versions. One of the powerful features that Equinox/p2 includes is the ability to read existing update sites that aren't in the Equinox/p2 format and generate the necessary metadata dynamically and invisibly to the user. However, doing so does take some time, so it affects performance. If you're building update sites, you can generate metadata for your existing update sites to enable them for Equinox/p2 and give your users a better, faster experience.

The major difference between the old and the new site format is that the plug-in JAR files are replaced with an optimized version of the JAR that is re-packed with the Pack200, which was introduced in Java 5. The Pack200 compression format can reduce the size of some JAR files by 50 percent or more (see Resources for more information on the Pack200 compression format).

As explained on the Equinox/p2 wiki site (see Resources), the old Update Manager used to require a download of all of the JARs listed in the site.xml file to read the manifests in the JAR files. This cost came even before the user actually chose to install anything, so it was not ideal, to say the least. This requirement has been eliminated by new files called artifacts.jar, content.jar, and digest.zip that contain the information that the Equinox/p2 framework needs to determine where files are located and which features are available in the site.

To update an existing update site to be used for Equinox/p2, run the Ant script shown in Listing 4 from the base directory of the update site. The script requires the ant-contrib project (see Resources) for some of the tasks that loop (<for>) and operate conditionally (<if>).

Listing 4. An Ant file for p2-enabling your current update site
<?xml version="1.0" encoding="utf-8" ?>
<project name="siteUpdater" default="update-site">

    <property name="site.name" value="My Site" />
    <property name="site.dir" value="/home/nathan/public_html/subclipse" />
    <property name="eclipse.home" value="/home/nathan/eclipse" />
    <!-- You may have to update this to the newest launcher -->
    <property name="launcher.jar" 
        value="${eclipse.home}/plugins/org.eclipse.equinox.launcher_(version).jar" />

    <taskdef resource="net/sf/antcontrib/antlib.xml">
        <classpath>
            <pathelement location="/opt/ant-contrib/ant-contrib-1.0b3.jar" />
        </classpath>
    </taskdef>

    <target name="assert-env">
        <available property="eclipse.dir.valid" file="${eclipse.home}" 
            type="dir" />
        <fail unless="eclipse.dir.valid" 
            message="Could not find Eclipse home directory [${eclipse.home}]." />
        <available property="launcher.valid" file="${launcher.jar}" type="file" />
        <fail unless="launcher.valid" 
            message="Could not find launcher [${launcher.jar}]." />
    </target>

    <!-- Updates the site.xml file with new attributes, if they're missing -->
    <!-- Note:  this is a very basic update based on search/replace.  If your site.xml
         has attributes for the site element already, perhaps optional xml update tasks
         would be a better fit (or manual update).
      -->
    <target name="update-xml">
        <replace file="${site.dir}/site.xml">
            <replacetoken><![CDATA[<site>]]></replacetoken>
            <!-- Make sure to update the digestURL to your correct URL -->
            <replacevalue><![CDATA[<site pack200="true" digestURL="">]]>
						</replacevalue>
        </replace>
    </target>

    <target name="update-features">
        <echo message="Updating file ${dest}..." />
        <touch file="${touch.file}" />
        <jar destfile="${dest}" basedir="${src.dir}" update="true" />
    </target>

    <target name="verify-features">
        <for param="file">
            <path>
                <fileset dir="${site.dir}/features">
                    <include name="**/*.jar" />
                </fileset>
            </path>
            <sequential>
                <echo message="Checking file @{file}..." />
                <mkdir dir="${java.io.tmpdir}/${ant.project.name}/@{file}" />
                <unjar dest="${java.io.tmpdir}/${ant.project.name}@{file}" 
                    src="@{file}" />
                <if>
                    <not>
<available file="${java.io.tmpdir}/${ant.project.name}@{file}/feature.properties"/>
                    </not>
                    <then>
                        <antcall target="update-features">
                            <param value="${feature.present}" name="file.exists" />
  <param value="${java.io.tmpdir}/${ant.project.name}@{file}/feature.properties" 
      name="touch.file" />
  <param value="${java.io.tmpdir}/${ant.project.name}@{file}" name="src.dir" />
                            <param value="@{file}" name="dest" />
                        </antcall>
                    </then>
                </if>
                <delete dir="${java.io.tmpdir}/${ant.project.name}@{file}" />
            </sequential>
        </for>
    </target>

    <target name="gen-pack-gz">
        <echo message="Processing file [${plugin.file}]" />
        <java jar="${launcher.jar}" fork="true">
            <arg line="-application org.eclipse.update.core.siteOptimizer" />
<arg line="-jarProcessor -verbose -processAll -repack -outputDir ${site.dir}/plugins" 
    />
            <arg line="${plugin.file}" />
        </java>
    </target>

    <target name="gen-pack-gz-files">
        <for param="file">
            <path>
                <fileset dir="${site.dir}/plugins">
                    <include name="**/*.jar" />
                </fileset>
            </path>
            <sequential>
                <antcall target="gen-pack-gz">
                    <param name="plugin.file" value="@{file}" />
                </antcall>
            </sequential>
        </for>

        <echo message="Optimizing jar files" />
        <java jar="${launcher.jar}" fork="true">
            <arg line="-application org.eclipse.update.core.siteOptimizer" />
            <arg line="-digestBuilder" />
            <arg line="-digestOutputDir=${site.dir}" />
            <arg line="-siteXML=${site.dir}/site.xml" />
<arg line="-jarProcessor -verbose -pack -outputDir ${site.dir} ${site.dir}" />
        </java>

    </target>

    <target name="create-digest">
        <java jar="${launcher.jar}" fork="true">
<arg line="-application org.eclipse.equinox.p2.metadata.generator.EclipseGenerator"
/>
            <arg line="-updateSite ${site.dir}" />
            <arg line="-site file:${site.dir}/site.xml" />
            <arg line="-metadataRepository file:${site.dir}" />
<arg line="-metadataRepositoryName &quot;${site.name} Update Site&quot;" />
            <arg line="-artifactRepository file:${site.dir}" />
<arg line="-artifactRepositoryName &quot;${site.name} Artifacts&quot;" />
            <arg line="-compress" />
            <arg line="-reusePack200Files" />
            <arg line="-noDefaultUIs" />
            <jvmarg line="-Xmx256M" />
        </java>
    </target>

    <target name="update-site" 
depends="assert-env,update-xml,verify-features,gen-pack-gz-files,create-digest" />

</project>

In addition to updating the JAR files to the new Pack200 format, the script creates three new files that the Equinox/p2 framework uses:

  • artifacts.jar— Contains artifacts.xml, which includes mapping rules that tell the Equinox/p2 framework where to find repository artifacts, such as plug-ins and features.
  • content.jar— Contains content.xml, which includes information about the IUs included in the site.
  • digest.zip— Contains digest.xml, which in turn contains metadata about the site, such as the features it provides and the license; the entries in this file can be displayed in the list in the Update Manager for the site.

This script automates the steps required to bring your current repository up to date. See Resources for details about upgrading your existing repository to the new Equiniox/p2 format.


The Eclipse Maynstall project

The Eclipse Maynstall project is an incubator project that has plans to build upon the Equinox/p2 provisioning framework to allow for better control for users and administrators. From the Maynstall site (see Resources), the current vision is that "The flow for Maynstall allows the user to launch a bootstrap, which brings up a launcher. The launcher walks the user through a series of steps to allow them to choose a profile."

The project's wiki site currently has steps for setting up Maynstall and running it locally if you're a developer and would like to get early access to this incubation project.


Conclusion

The Equinox/p2 provisioning framework, although still in its infancy, is a step toward enabling better installations for Eclipse. This article demonstrated, at a high level, some of the new abilities that you have as a user to manage your Eclipse installations using tools like the p2installer and p2agent. This article also covered some of the optimizations made for update sites to enable a higher-performing user experience for making updates. Finally, this article introduced you to Maynstall, a project based on Equinox/p2 that allows administrators more control over Eclipse profiles.

Resources

Learn

Get products and technologies

Discuss

  • The Eclipse Platform newsgroups should be your first stop to discuss questions regarding Eclipse. (Selecting this will launch your default Usenet news reader application and open eclipse.platform.)
  • The Eclipse newsgroups has many resources for people interested in using and extending Eclipse.
  • Participate in developerWorks blogs and get involved in the developerWorks community.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Open source on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Open source
ArticleID=447975
ArticleTitle=The Equinox p2 provisioning framework
publish-date=11242009