Marius Kjeldahl's homepage

I've tried them all; Windows, Mac OSX and Linux. Windows sucks for software development, but if you have to use it and are not locked into the .Net/CLR environment using Visual Studio, make sure you install Cygwin, which makes it kind of useable for some tasks (Emacs, Perl, Clojure and more). I tried going with the Mac OSX momentum as well; it has a solid Unix foundation which makes most development comfortable if you are coming from the Linux side of things. Unfortunately, it is also quite often just different enough the traditional Linux setups, so you most often need to install tons of other stuff in addition to the standard things that comes bundled with it. So pretty soon you risk having multiple versions of perl and other commonly used tools, essentially a whole "tree" of "non-standard" utilities to keep up with the stuff needed for using the latest versions of whatever tool needed. The thing that kept me from settling in was the lack of a decent full screen mode (maximized Window mode; this is from a couple of years back) and the fact that it never felt like Emacs was a first class passenger on Mac OSX (I went to hell and back trying to get it to recognize my favourite bitmap font; without success). So in the end, I wasted a lot of time (again) to get my Linux GUI setup decent, and it was for a couple of years (Ubuntu Linux 10.10 and 11.04).

A couple of years later, I'm sad to say that the state of the Linux GUI is still sad. With the right combination of laptop and age, you may very well be able to have a great functional stable GUI. Unfortunately "right combination" usually means "not too old and definitively not too new". On the "too new" side, it's mostly possible to get things working by installing development packages etc, but that's definitively not for the newbies out there and require skill, knowledge and a lot of effort. On the "too old" side of things, you may be SOL, thanks to proprietary drivers (in my case nVidia) no longer being supported.

Even more maddening is the fact that the GUIs themselves seem to have gone from "stable, functional and nice looking", to "unstable, barely functional, but possibly looking better than ever". A couple of years ago I moved from KDE to Gnome 2, after years of frustration with getting a stable networking applet running under KDE, shortly after KDE had it's own "evolution" from KDE3 to KDE4 (not a great success story either at that time). Switching to Gnome 2 gave me great hopes for the linux GUI side of things; it just worked and looked great.

Fast forward until the last few months; Gnome 2 is considered "to be discontinued", and we've gotten even more spoiled with eye candy, transparency and more on all platforms. Ubuntu has launched their own project, Unity, and Gnome is working on upgrading to Gnome 3. Both these bring entirely new GUI paradigms to the Linux desktop, and common to both of them seem to be focusing on the "laptop/tablet" factor only. That means that if you need to run applications full screen on a small screen, the new paradigms work ok; it takes some getting used to, but nothing major. But on our "old" (by now industry standard in software development) big screen, and even multiple big screen, setups, the new paradigms offered simply aren't ready.

Unity, running on top of Gnome 3, tries to go in the direction of OSX it seems, forcing the menu bar to always appear on the top of the screen. Sadly, a lot of Linux users are used to "focus follows mouse" instead of "click to focus", which means that unless your application is nailed to the top of your screen, you need to navigate to the top screen menus for your application without the mouse passing over any other application (which would activate another set of menus). Workaround are either to activate "click to focus", or run your applications full screen. The upshot is that Unity offers great keyboard navigation shortcuts, so maybe I will finally be able to adjust myself to accept "click to focus" (and use the mouse even less overall).

The Gnome Shell offers an alternative to Unity, also running on top of Gnome 3. I haven't made my mind up yet which one I prefer; I'm trying to get used to both.

Gnome 3 (or possibly the Gnome Shell part of it in a couple of cases) add a lot more eye candy.But also seem to target exclusively (at least initially) the laptop/tablet form factor. And they are even more ambitious as far as "revolutionizing" the desktop metaphors go and remove well known entities such as minimize and maximize buttons, and state the concept of workspaces are probably better anyway. Workspaces are "secondary desktops"/screens whatever that you can easily move between by pressing ctrl-alt-arrow-up and -down. That works fine, at least until you are on your dual screen setup again. Then notice that the workspace scrolling (up and down) only happens on one monitor; the workspace does not switch on the second monitor. At least Unity managed to keep the expected "workspaces" behaviour.

Generally, Gnome 3 (meaning both Unity and the Gnome Shell) seems to have messed up Alt-Tab for switching Windows. Alt-Tab now switches applications, so if you need to go to a certain window in an application, you first have to Alt-Tab to the proper application, then wait a second or two for the Application to show it's windows, and then keep tabbing to the right window. Alt-` (the button above tab) allows you to switch windows inside the current running application, but not to windows in other applications. For at least some of these changes, workarounds are supposedly available but I haven't gotten them working yet.

I've done most of my testing for the last couple of months on alpha and beta versions of Ubuntu 11.10 Oneiric, but unfortunately the improvements seem to be slow arriving, making me doubt whether any new desktop will be decent to launch sometime in October. In addition to the issues I've mentioned already, it also seems a lot of the desktop customization stuff seem to have disappeared from Gnome 3. Simple things, such as modifying system fonts and sizes and/or various theme/icon settings does not seem to be included in System Settings anymore. There is a package with a tool that has such settings (gnome-tweak-tool), but based on the type of experience I got used to from Gnome 2, that tool looks kind of foreign and a lot of the settings related to themes etc simply do not work.

Having said all this, I still believe Linux (and the people and movements behind it) offer a much more compelling overall development experience that most other systems, and I can take a lot more pain before giving up, but here's hoping and praying that Linux based groups will manage to pull through and deliver a great experience in the end. Unfortunately, I still the "Linux Desktop" for "normal users" is still a couple of years ahead. Based on the progress the last five years, it could be even longer, but I'm keeping my fingers crossed that it will be fixed long before that.

And finally, some notes to myself below on what I need to do to get a decent Ubuntu 11.10 Oneiric system going:

Alt-Tab changes: Supposedly, there is a workaround for getting rid of the awful Alt-Tab changes, but I haven't gotten it working on my system yet (Unity does not allow Shell Extensions at all; Gnome Shell allows me to install and enable it using the gnome-tweak-tool, but then Alt-Tab simply does nothing). If you want to try, google gnome-shell-extensions-alternate-tab .

Atheros wireless drivers: This one seems to be haunting Linux. For older type cards (lspci outputs "Intel Corporation Ultimate N WiFi Link 5300") it seems the full series of Oneiric alphas, and betas so far, has a regression as far as cryptography in the "iwlagn" driver goes. To work around it and get decent wifi speed again, add "options iwlagn swcrypto=1" to /etc/modprobe.d/iwlagn.conf . For newer cards, add "options ath9k nohwcrypt=1" if you're having really slow wifi. Hopefully these will be nailed (again) before the final Oneiric release.

My latest work requires me to get more intimate with development on the Windows platform again, and I'm taking this opportunity to explore Windows CLR and .Net using Clojure, or more correctly Clojure-CLR, the version of Clojure that is hosted on the .Net platform. Unfortunately, there is a lot less documentation about Clojure-CLR, so it probably is a bit harder to get started.

Downloading and building clojure-clr

You can get clojure-clr from https://github.com/richhickey/clojure-clr , where you can either download it as a pre-compiled binary, or download/checkout the source and build it yourself. If you want to build it yourself, make sure you download the lib.zip file as well; it contains the library dependencies you need for building from source (in addition to the .Net 3.5 or 4.0 SDK).

To build clojure-clr, open the Visual Studio "solution" file Clojure/ClojureCLR.sln . After successful building, you will find the binaries inside bin/4.0/Release (or 3.5 if you build against dotNet 3.5, and/or Debug instead of Release if you build a debug release). After building, you can fire up Clojure.Main.exe to have the famous REPL (http://en.wikipedia.org/wiki/REPL) up and running.

Running clojure-clr

Windows isn't well known for it's rich selection of command line shells, probably simply because they aren't used all that much. Doing simple scripting and programming in REPL style environments requires some method of entering commands and getting output, and the command line shells are usually the most "universal" method of doing this. Windows offers the default cmd.exe for all users. The Windows Powershell is also available, but possibly not installed by default. Both of these offer limited choices (if any) as far as resizing goes, including both window size and font selections and sizes. And the default font selections in Windows are usually too large and almost always in bold.

Having been somewhat spoiled from the relatively rich selection of choices in the Linux environment, I installed Cygwin (Unix toolset for Windows - http://cygwin.com/) and had a rich shell available (I selected mintty), which allows for full and easy resizing of windows and font sizes. Unfortunately, due to the way Windows console applications work by default on Windows, there was an issue with arrow keys and command line editing inside the Clojure REPL. In a REPL you need command line editing and history to work.

It seems the author of mintty has created a fix which works great with Clojure on Windows, see http://groups.google.com/group/mintty-discuss/browse_thread/thread/1f9cf... . If you start Clojure inside conin ("C:\>conin Clojure.Main.exe"), command line works great, and you get a rich command REPL environment for developing using clojure-clr.

Clojure Hello World in Windows

Microsoft has been in catch-up mode for the last decade, and unfortunately their API/development platforms shows. If you want to implement a "native" Windows GUI application today, the choices for the .Net/CLR platform is either Windows Forms (Forms) of the Windows Presentation Framework (WPF). While a lot can be said for either framework, the usual recommendation is to use WPF. I've managed to put together examples of two simple hello world style examples below for both frameworks. The first one, using Windows Forms, is pretty straightforward.

(import '(System.Reflection Assembly))
(Assembly/LoadWithPartialName "System.Windows.Forms")
(ns helloworld
	(:import (System.Windows.Forms MessageBox)))
(MessageBox/Show "Hello world")

The second hello world example is a bit more complex. It starts a Windows application, opens an empty main windows and changes the title in the window to "Hello world". As the code demonstrates, it is not as straightforward, partly because it does more (starts the GUI app in it's own separate thread), and partly because WPF requires a larger footprint to get a minimum application up and running.

(import '(System.Reflection Assembly))
(Assembly/LoadWithPartialName "PresentationFramework")
(Assembly/LoadWithPartialName "PresentationCore")
(Assembly/LoadWithPartialName "WindowsBase")
(ns hellowpf.core
	(:import (System.Windows Application Window))
	(:import (System.Windows.Threading  Dispatcher DispatcherPriority))
	(:import (System.Threading ApartmentState ThreadStart Thread)))

(defn gui-main []
	(let [app (Application.)
				mainwin (Window.)]
		(.set_Title mainwin "Hello world")
		(.Run app mainwin)))

(defn wpf-eval
	[uithread repl-ns-sym data]
	(.Invoke (Dispatcher/FromThread uithread) DispatcherPriority/Normal
					 (gen-delegate Action []
												 (clojure.main/with-bindings
													 (in-ns repl-ns-sym)
													 (eval data)))))

(let [threaddelegate (gen-delegate ThreadStart [] (gui-main))
			uithread (doto (Thread. threaddelegate)
								 (.SetApartmentState ApartmentState/STA)
								 (.Start))]
	(clojure.main/repl :eval (partial wpf-eval uithread 'hellowpf.core)))

While the Forms example can be run repeatedly from within the Clojure REPL environment, the WPF application can not. It seems related to the fact that the System.Windows.Application instance can only be used and instantiated once within an interactive environment such as the REPL. While this may not be a big problem for running the app as a "standalone app", it certainly does not make it very practical to work with such applications from the REPL environment, unless you do not mind restarting the Clojure REPL every time. According to answers I've found through Google, the Forms API seems to offer simple support for restarting an application within a process, it seems the same functionality is lacking for WPF applications. It may very well be that it is and should be possibly, but based on my finding it's not very easy. If somebody wants to clue me in on how I can restart a WPF application without restarting the process, feel free to write a comment.

I guess from my background, I'm somewhat of a computer language polyglot, like most developers need to be these days. I believe I have what it takes to understand most programming languages and paradigms, and have never been afraid of taking on new challenges. During autumn 2010, some friends and former colleagues of mine had started a new company ("Friendco"), and were looking for people. I was already in the process with another company ("Otherco"), but since that process had not completed yet, we agreed that I would work with the Friendco my friends had started at least until the process I had initiated with Otherco completed.

As faith would have it I ended up taking a position at Otherco, but I worked with Friendco for a couple of months, and this would be my first "professional" encounter with the Java programming language and environment. I've quoted "professional", considering I only got to spend a couple of months doing it. Two of my colleagues, one of which I knew from before, had been using Java professionally in their former jobs, so I guess they would qualify as Java professionals (without quotes). Needless to say, I also depended on their support to bring me up to speed.

My friends also qualify as polyglots, and their familiarity with Java and their growth plans, meant that they had selected Java as the platform for building the software platform for the company. So if I was going to contribute, I had to develop in Java. This blog post is more about my biased opinions related to my relatively brief introduction to Java, and should not be misinterpreted as critique to my friends in Friendco or the choices they've made.

The first obstacle was the development platform. I've developed on Windows, OSX and Linux, but I prefer Linux over the others. Of the two friends I would be working with, one was developing on OSX and the other on Windows (don't bash him for it; he does server side development on Linux as well, he just prefers Windows on the client and is a strong believer of dogfooding, i.e. using the same platforms as his "customers"). Regardless of the development platform, the final software is deployed on Linux servers. Despite being a fairly small company, their setup seems pretty modern, using modern tools like the Atlassian sweet of tools, together with Hudson for continous integration and deployment, together with various automated testing tools (CheckStyle, FindBugs and more).

Eclipse

Their development platform was Eclipse, the java-based IDE for java development. As programming languages have come and gone, so have IDEs, and like many "grown-up" developers, I've found my comfort zone using Emacs for editing and development, and various command-line based build tools (most often already integrated with Emacs). One of the devs said he had tried using Emacs as well when he started out with Java, but had ended up with Eclipse anyway since he was more effective with it. Specifically mentioned were automatic code reading and code-completion inside the editor, and the fact that Eclipse more or less was able to build the project while he was working on it, picking up individual file changes and recompiling automatically.

Apache Maven

Considering there was already two devs using Eclipse with success in the project, I started out working with Eclipse as well. After a few days, a lot of the "automagic" things Eclipse did started to freak me out. I was never quite sure what was built or not, and sometimes I had to invoke some magic commands in Eclipse to force a full rebuild. I also learned that the other devs experiences similar issues from time to time. I do not like having a build enviroment I do not fully understand, so I slowly but surely started digging into how the build stuff worked.

The codebase we worked on was already quite big, because source code had been licensed from another company when Friendco started up, and the project was split into separate modules/sub-projects with various dependencies. So while my frustration with Eclipse grew, I started to dig into the build process. The project used Apache Maven as a "build tool", and it seems the Eclipse magic used the Maven project files together with it's own pixie dust for automating the builds. Fortunately, Maven also has a command line utility for building Java projects, which I started using together with Emacs for editing source code. I was finally able to ditch Eclipse for Emacs and Maven. Sure, some "handholding" tools like documentation was a bit harder than inside Eclipse, but at least I knew exactly when and how things were built. Emacs it seems had similar tools in various alpha-stage add-ons, but nothing I found worked very well (I did not spend a lot of time on getting it set up either). Another thing I would probably miss at one time or the other would be the integrated debugger built into Eclipse, but as you will see later in this article I never really needed the debugger for the kind of work I was doing. Getting intimate with Maven also meant that I was able to help out in cleaning up some problems in the maven project files as well, so the whole build and continous integration process was improved overall on all platforms.

Cross-platform Java mostly works

Considering we were working on three different operating system platforms, using a multitude of tools and stand-alone software that ran more or less across all three platforms, I am actually impressed that it worked as well as it did. At least initially.

Firefox/Mozilla integration

Part of the software we were working on included a web spider that would index web pages, and based on former projects and current ambitions, my dev colleagues had decided that we would continue to actually integrate with the Firefox browser from Java and use this for the project. The main reasons were to "see" the web as users would see it (same layout, similar plugins etc), our spider would more or less "see" the web as users would be seeing it inside their own browser (assuming they used Firefox I guess...). Mozilla offers their XPCOM layer as a cross-platform integration layer. They even have a Java version, JavaXPCOM. Supposedly, XPCOM is supposed to be some kind of cross-platform version of Microsoft's own Component Object Model that enable system components to communicate. Microsoft's own COM model is known to have it's own set of challenges, and ufortunately it seems similar issues exist with XPCOM.

JavaXPCOM

The idea of integrating using JavaXPCOM supposedly is that it would be very easy to do callbacks from our own Java software into Firefox, and the reverse (Java-based callbacks from events inside Firefox). Sounds great in theory. In practice however, it is very difficult. The documentation for both interface and usage is "challenging" at the very least, and unfortunately mostly inaccurate or obsolete. There's also a whole range of issues related to Firefox versions (3.5, 3.6) and the Mozilla library versions (1.9.1 and 1.9.2); I figured it out eventually, but really, it shouldn't really be that hard to figure out. And finally, the interface itself. The lack of docs and/or working programming examples leads to lots of trial&error and cut&paste programming, finding tidbits here and there and trying to pin down what works and what does not, and why if at all possible.

But there is one place where JavaXPCOM seems to work fine; inside Eclipse. Lots of the things we struggled with getting working seems to work fine with the browser inside Eclipse. And as all real programmers do eventually, you start digging into the source code to figure out how it does it's magic. Magic is certainly the right word for it. It seems XPCOM is implemented using some kind of binary communication protocol with lots of hardcoded integer constants. Imagine the fun I had when I had to replace the constants used with symbols instead to make "Checkstyle" tools happy, without any real insight to what the constant actually did ("final const int MAGIC_INT_6 = 6").

Furthermore, I had to use Java run-time reflection APIs to figure out which XPCOM interfaces was available. Sometimes signatures using "ints" were available, other times they used "longs".

Cross platform Eclipse with XPCOM; consider it evil

Ripping through the Eclipse sources, I eventually managed to get the XPCOM integration working by more or less depending on modules provided by the Eclipse APIs. Although I did get the spider working, it wasn't elegant, and I hadn't nailed the root cause as to why things differed so much on the various platforms we used. Which is about when I discovered how Eclipse uses source code manipulation to effectively implement platform dependent macros. Those int versus longs challenges I was having earlier was due to Eclipse, as part of being compiled or built, effectively replace function signatures prior to compilation, replacing ints with longs or the other way around, depending on whether the target platform is 32-bit or 64-bit. I can't remember the specific signatures used, but they looked something like this: int /*long*/ somefunc(int /*long*/ par1 ...). So obviously, if the platform changed to 64-bit, some software would replace all those signatures in the source code to long /*int*/ somefunc(long /*int*/ par1 ...).

Firefox spiders

During my efforts, I found various alternative methods of automating Firefox for spider web pages or similar, where most of them used the add-ons mechanism in Firefox together with Javascript to provide the necessary integration. Based on my efforts, I believe that is a much better solution going forwards, and it is probably also easier to extend to other browsers as well when/if that is necessary (google "firefox automation").

Other Java observations

• Signed bytes. While working with Friendco, I also implemented a SOCKS4 proxy in Java which would be used for IP traffic accounting. Worked perfectly. The proxy, like most other low-level software, works on bytes inside IP packages. It's always a challenge when you know exactly the simple operations that needs to be done at the byte level, but the tool at hand (Java) seems to be more or less deisgned to make it very hard. Signed bytes in Java, together with converting to shorts, ints and similar, really made for an interesting few days... ;-)
• Constants inlining. We also had some interesting times where the Java compiler allows one class to inline constants from another class, so if you changed the other class it would be recompiled, but the class that had inlined the constants would not pick up the changes. In big projects, this gets really fun. There are workaround by not using constants directly (think "constant functions returning the constants" instead), but they again may not play well with the style verification tools (Checkstyle, Findbugs etc). Personally, I believe this kind of behaviour from Java compilers should be discontinued asap; if something changes in a dependency, all classes using the dependency should be forced to recompile.

In summary

This was my first real Java development in an enterprise setting (I've done some tinkering on mobiles earlier). Up until this project, I've more or less consciously decided against diving into Java, but I've also had the luxury to make the calls on most projects I've participated in so far. This experience with Java confirms what has been my beliefs so far; use Java if you must, or if you plan on hiring lots of developers fast (you need some middle ground that everybody can understand). But there are certainly better tools for getting most jobs done more efficiently (both in development time and run-time), and it'll most often be less painful than Java itself.

It's also worth pointing out that the stuff that I actually did in Java (run-time reflection, Firefox XPCOM integration and writing a SOCKS4 proxy server) probably are not good examples on how Java makes cross-platform software development easy, and I doubt Java can really be blamed for the challenges I had with XPCOM. I guess Mozilla deserves the blame for XPCOM, or Eclipse with how they implemented it. On the other hand if professional Java software like Eclipse uses "source code modifications" to implement "platform dependent macros", I would say that definitively is a step back compared to languages and systems before Java. If that's the way "the professionals" do it, it certainly does not smell good.

There are interesting developments happening on the JVM itself (the target that Java compiles to run on), including Clojure, Scala and more, but I'll save that for another blog post.

I recently decided to take a look at some open source Q&A systems, probably mostly known as Stackoverflow clones. The first one I tested was OSQA (http://www.osqa.net/).  It runs on Python and Mysql (and possibly PostgreSQL, although I didn't test). Setup was pretty easy, and everything was up and running fairly easy. I had a look at localisation as well, and it seems it uses a mix of the traditional localization tools ("gettext", .po files etc) and some custom javascript files which require a somewhat different method, at least for now. All in all, pretty easy setup and easy to get going.Another alternative is Shapado (http://shapado.com/). I decided to investigate it because it's graphical design looks a bit more polished than OSQA. It runs on Ruby and uses MongoDB for storage. My Python and Ruby knowledge is roughly equal (meaning I know Perl a lot better.. ;-), but installing Shapado and all required requisites turned out to be an absolute pain in the ass. Either Ruby with modules is moving to fast, or Shapado is too picky with what versions of required modules it needs, but either way, installing Shapado and Ruby was not straightforward. I learned a bit about the Ruby "gems" system and finally managed to get everything running, including mongodb and the required Ruby interface. Unfortunately, I never managed to get Shapado to accept any questions. It did access and store stuff in the mongodb database, but both with and without the captcha module active, Shapado never accepted my questions. Worse, it did so without any error messages, not in the UI or it's own log. I posted about it on the Shapado Q&A site, but even after waiting a few days nobody offered any suggestions on what could be wrong.In the end I learned a bit about getting MongoDB and Ruby based systems up and running, even if Shapado never decided to cooperate. I do not need a Q&A system right now, but if I did and had to select one of the two, I would have to choose OSQA for obvious reasons.