Kepler retrospection - was CBI a net loss?

While I was reading Kepler retrospective notes, I found this statement:

CBI was a net loss for us so far. It was a very large volume of work just to get our build back to its old state. There is benefit for others in the community to make it easier for them to run builds independently. For committers it hasn't delivered value yet.

and started immediately wondering who was the "us", and why it was "a net loss".

Fedora had it's own Eclipse build system, which was 8 x faster than current CBI build (15 minutes vs 2 hours on non-SSD drive). I've spent a lot of time getting Eclipse built with Fedora and switched to CBI before the Foundation did.

All that was (and still is) a cost, but was it really "a loss"?

I find it rather as a first step towards a really necessary move - Eclipse Foundation taking responsibility on development. The build system is first, then maybe we will get vendor-neutral architects or UX designers.

Another point is that once it is possible to easily rebuild Eclipse, we can see even right now internal Eclipse teams appearing here and there, which tend to become really valuable contributors.

Not to mention that after Kim left, it took about a month to get builds running again (at the Eclipse Foundation, we had them running in Fedora). Is it ok to have so much power depending on one person (because that is what happens if you have a large ant-based build)?

I'm sorry but I can't agree with a "net loss". It was a necessary cost. Good for the Eclipse Foundation. Good for the Eclipse.

BTW. I really hope some day UX designers or architects will be hired by Eclipse Foundation.

Cascaded builds - solution or a problem?

Fedora 19 consists out of approx 14000 of packages. Could you imagine a situation, where a security fix for one basic package (let's say kernel) would cause a need for the 14000 packages rebuild*?

It sounds strange.

On the other hand, it is perfectly normal in the Eclipse world. Let's imagine that I want to put a new EMF into my Eclipse platform. So I need to build EMF, and put it into Eclipse (a.k.a. rebuild platform with proper update sites). The whole process takes quite some time, but that's not that important.

If EMF had any dependencies, I'd have to rebuild them first.

So I've started investigating how to automate those tasks. And this is an answer I've received:

[...] From my point of view, these Java module frameworks refuse to acknowledge that  there is extensive experience with distro-level release engineering. (Basically, exact dependencies and multiple versions of the same code might be convenient now, but will seriously hurt you down the road.) [...].
Florian Weimer.

Full text in the fedora-devel.

The part about strict dependency check struck me as being "just right". Eclipse definitely needs a very strict checking at the build time and during changing the installation.

But it does not, and should not be very strict about components being changed at runtime.

And yes, rebuilding the entire Eclipse to get a minor security update in one of it's dependencies is a waste of time.

So I've put up a small wiki page in which I described how I'd like to see installation handled.

The link to the wiki page.

Any comments welcome.

* such situations happen, although very rarely, especially when there are big changes done in compilers, but this happens only when a distro was not released yet.

EDIT: corrected typo and link to fedora-devel.

A nasty bug in dropins - read it if you use them

The nature of dropins is that they were supposed to be a transitional mechanism between old update manager and P2. Dropins have more advantages that they were supposed to have, and one of them is truly easy management - just copy a plugin, and it will work (in most cases).

This time, however, a really nasty issue managed to get into Kepler. It was discovered about two weeks ago, when some of our Fedora packages started disappearing after updates.

We often get similar issues before final build of Fedora is released, so it did not looked seriously. But after a couple of days of checking our build process and testing I've found that:

If you deploy into dropins a plugin that is already installed using P2 director, your Eclipse goes bananas, especially if this plugin is a platform dependency.
The dropins mechanism will try to uninstall it, and then, depending on your configuration, will stop working, or will break your Eclipse.

The bug was reported in the Eclipse bugzilla, but it will most likely be shipped due to the bad timing in which it was discovered.

So beware of what you put into dropins - unless you want spend the same amount of time that I did trying to understand what's going on.

EclipseCon France - Wrap up

I had a pleasure to take part in the EclipseCon held for the first time in France. Here are is my main observation:
Java has more in common with javascript than continental breakfast with breakfast.

Also, for the first time, there was a true coming out of people adjusting and packaging inhouse flavour of Eclipse. A few of my live-recorded tweets:

Cern devops - a new major version cant mean starting over.
— Christopher Daniel (@kda) June 5, 2013

.@prapicault's talk: 10000 people spending 5 minutes troubleshooting Eclipse is a cost.
— Christopher Daniel (@kda) June 6, 2013

Yet another presentation saying that standardization is required if you want to scale up your dev team. There is future! #eclipsecon #devops
— Christopher Daniel (@kda) June 6, 2013

So, it's all about having a few people in an organization that know tools, and prepare tools for others. It just looks like the idea of "bring your own device" policy was wrong. A corporation needs standards. Period. And it's much better to have a team designated only to free software management than allow your employees to invent the wheel again and again.

I wish all those eclipse builders contributed back!

What's more, I believe that the future belongs to those small, single-purposed IDEs, and kind of natural consequence will be equally precise a IDE in the browser, where you rather point&click required solution rather than spending a couple of days figuring out how to set up certain tools to build a server app and then a mobile app. Ideally I'd like to be able to say "yes" to this tweet:

@RickBullotta @kda @mmilinkov cool. Can I write and debug my native mobile apps that talk to my node server?
— Doug Schaefer (@dougschaefer) June 5, 2013

That's said, Eclipse was included into the Red Hat Developer Toolset. The Developer Toolset is a set of tools that work together, and are decoupled from the underlying base, so you will be able to use a stable work environment and up-to-date development tools, without configuring things by yourself!

The most significant feature of Kepler Eclipse on Fedora

Yeah, I know I'm kind of repeating myself, but look what happens after you update Eclipse in Fedora 19:

Eclipse after update in F19

P2 discovers that you're system installation had been changed, finds what you have installed as a regular user, and then asks you to install those things again, providing you a nice wizard:

Features reinstallation wizard page

This is a big step forward since previous Fedora releases, where Eclipse package update caused a loss of user installed features, or even totally broke the installation.

Of course, not only Linux will benefit from this approach, but all setups, where users have only read-only access to a master installation administered centrally.

Edit: Things never go as smoothly as one would wish. The problem that I was having with this dialog was that when I had multiple Eclipse installed, from time to time, the dialog was trying to reinstall things from "wrong" Eclipse installation.

It turned out to be a feature, not a bug. Eclipse, when shared install is run for the very first time, tries to find previous installation on the best-can-do basis. This scenario is a bug for Fedora, because Fedora Eclipse has only one location that does not change, and therefore if there is no local configuration for that location, it means that nothing should be reinstalled. I think a solution is as easy as setting this property to the Fedora installation:

-Declipse.p2.skipMovedInstallDetection=true

Eclipse Mylyn - quite interesting feature

Did you know that Eclipse Mylyn integrates with a History view?

Exemplary history displayed in the History view.

Unfortunately, it works this way for Red Hat bugzilla, but not for the Eclipse one :-(.

Eclipse Marketplace Client just got updated in Fedora 19+

Short introduction for my Fedora audience:

Eclipse Marketplace is, well, a market for Eclipse plugins.  Eclipse Marketplace Client is a tool that allows for browsing and installing plugins from within your IDE. The analogy to other application stores is quite obvious. You can install the client by invoking:

sudo yum install eclipse-mpc

Main content:

Fedora 19 release date is approaching us fast. Since the Fedora release and Eclipse release are very close to each other, and there is a freeze between the RC (Beta) and the final release, I'm updating now the Eclipse stack to the Milestone 7 - the last milestone between the final release of Eclipse.

Going from one project to another I stumbled upon Eclipse Marketplace Client. It has no new release scheduled on it's website, but the log has revealed quite interesting features, so I've decided to include those updates.

Here is a screenshot of a new Marketplace Client:

Note new attributes that are now displayed - the number of installs and favourites stars. This will definitely help to assess which solutions are well tested in the field :-).

It also looks like there will be a newsletter in the marketplace client :-)

The market place is interesting also from another point of view.  I have written a small utility mojo that transforms a p2 repository/update site into a runnable form that could be put in dropins. So the part of rpm responsible for installing the client looks like:

mvn-rpmbuild org.fedora:feclipse-maven-plugin:install  \
 -DsourceRepo=org.eclipse.epp.mpc.site/target/site -DtargetLocation=%{buildroot}%{install_loc}/eclipse

Which is a great progress from manually unzipping features into destination :-).

Yes, yes, I'm aware that dropins mechanism is not really the best solution, but that's the only solution that we managed to get work together with rpm.