Nilesh D Kapadia

Jekyll and WEBrick with HTML5 audio

2010-08-29T00:00:00-04:00

I am working on a site that uses Jekyll and also HTML5 audio. When running the server locally (which uses WEBrick), I had issues with playing OGG files using HTML5 audio (MP3 files would play at least in Safari, OGG files would not play in Firefox). Though the GET request to the server was being received, the browser simply wouldn’t play the music files.

It turns out that certain MIME types are required, at least for OGG files. OGG files should be audio/ogg and MP3 files should be audio/mpeg. WEBrick’s default configuration did not include these mime types, and Jekyll did not add them.

To configure this for Jekyll, I edited the jekyll’s startup script like this:

  mime_types = WEBrick::HTTPUtils::DefaultMimeTypes
  mime_types.store 'js', 'application/javascript'

# START mime type edit
  mime_types.store 'mp3', 'audio/mpeg'
  mime_types.store 'ogg', 'audio/ogg'
#  mime_types = WEBrick::HTTPUtils::load_mime_types('/etc/apache2/mime.types')
# END mime type edit

  s = HTTPServer.new(

This adds the mime types for MP3 and OGG files to the default mime types. Alternatively, you could use the commented-out line that uses load_mime_types to load Apache’s mime.types file. Note that this will completely ignore WEBrick’s defaults and use whatever is in Apache’s mime.types file (or whatever file you point it to). The version of that file in my Apache2 installation did have the necessary mime types.

Embedding Apache Tomcat 7.0

2010-08-25T00:00:00-04:00

Apache Tomcat 7.0 now has a beta release out. One of it’s features is improved support for embedding. I have been embedding Jetty successfully in various projects, so I was hoping that Tomcat’s support for embedding was on par with Jetty’s. So far, Tomcat 7.0’s embedding API is quite easy to use and has been working well. However, I did not find any documentation on how to actually use it, so it took a small bit of digging to figure out how.

Details

The one suggestion I read was to look at the test cases, because they used embedded Tomcat. So I checked out the source code, and found TomcatBaseTest which was an abstract class that tests were subclassing which provided a running embedded Tomcat. The org.apache.catalina.startup.Tomcat class is what you need to use. After setting various settings, adding the application(s), you then call the start() to start Tomcat. Note that start() does not block, so if you exit your main method, Tomcat will quit.

Like it’s non-embedded version, embedded Tomcat needs to create a work directory for temporary files. You specify a base directory which the work directory will be created in. I believe this base directory is the equivalent to Tomcat home directory. Use the setBaseDir(String baseDir) method to set this. Looking at the source for Tomcat.initBaseDir() method, it looks like if you don’t set it using this method, it will try to obtain this by trying:

From the catalina.base system property
Failing that, from the catalina.base system property
Create a temp directory using user.dir system property as a base and tomcat.PORT_NUMBER as the temp directory it creates (replacing PORT_NUMBER with the actual port number.

You can set a port number with setPort(int port). This defaults to 8080 if not set.

To add an application, use the addWebapp(String contextPath, String baseDir) method, setting the webapp’s context path and specifying the location of the webapp folder for baseDir.

Once you are configured, you can call the start() method. Then you need to make sure your app doesn’t exit, the simplest way is to do an infinite loop, e.g. while(true) { Thread.sleep(999999999); }

Necessary JAR files

From the Tomcat 7.0.2 distribution, I put the following jar files in my classpath (not sure if all of them are necessary):

annotations-api.jar
catalina-ant.jar
catalina-ha.jar
catalina-tribes.jar
catalina.jar
ecj-3.6.jar
el-api.jar
jasper-el.jar
jasper.jar
jsp-api.jar
servlet-api.jar
tomcat-api.jar
tomcat-coyote.jar
tomcat-dbcp.jar
tomcat-i18n-es.jar
tomcat-i18n-fr.jar
tomcat-i18n-ja.jar
tomcat-juli.jar
tomcat-util.jar

Notes:

tomcat-juli.jar was found in the TOMCAT_HOME/bin folder, whereas the rest were in the TOMCAT_HOME/lib folder
servlet-api.jar is for the Servlet 3.0 spec. Make sure you don’t have any Servlet 2.x versions of this jar, otherwise Tomcat will throw an exception and fail to work.

Example

Here is a little example app that assumes you have a webapp (WEB-INF with a web.xml in it) in the folder examplewebapp:

import java.io.File;
import java.io.IOException;
import java.lang.InterruptedException;

import javax.servlet.ServletException;

import org.apache.catalina.LifecycleException;
import org.apache.catalina.startup.Tomcat;

public class TomcatEmbedExample {

    public static void main(String[] args) 
        throws IOException, ServletException, 
               LifecycleException, InterruptedException {

        String currentDir = new File(".").getCanonicalPath();
        String tomcatDir = currentDir + File.separatorChar + "tomcat";
        String webRoot = currentDir + File.separatorChar + "examplewebapp";

        Tomcat tomcat = new Tomcat();
        tomcat.setBaseDir(tomcatDir);
        tomcat.setPort(4040);
        tomcat.addWebapp("/examplewebapp", webRoot);
        // or we could do this for root context:
        // tomcat.addWebapp("/", webRoot);
        tomcat.start();

        while (true) {
            Thread.sleep(999999999);
        }
    }

}

Ideas on what this can be used for

This has replace embedded Jetty as my development server. I run it inside of a debugger (Eclipse), and Java hot code replace works nicely. One thing that I am experimenting with is monitoring specific files to know when to restart the app. I am currently using JNotify to monitor file changes, and restart the app when XML files or other configuration files change that need a restart to get applied. It is working great so far, and I no longer have to manually restart the server or sacrifice Java hot code replace.

New site using Jekyll static site generator

2010-08-16T00:00:00-04:00

I have switched from using Drupal for my blog to using Jekyll, a static site generator. Drupal has served me well for the last 6 years, but for this site it has been overkill and keeping up with the frequent security updates has been a chore.

How this works

What Jekyll does is use templates to generate static HTML pages. This means for each update to my site, I simply redeploy a new set of static HTML pages that I have generated using Jekyll. I am storing my site data in a Git repository. On the server, I use an update script to fetch the latest from Git, and generate the new version of the site.

I also store and work on my site data in my Dropbox folder. I work on this site on more than one computer, and relying on a SCM alone to synchronize would be way too cumbersome. Dropbox has been great for automatically synchronizing the project between machines, and seems to handle synchronizing Git metadata fine. I’m actually doing this for all my personal projects these days.

My old workflow with Drupal

My workflow for writing entries in Drupal was to edit posts in Emacs using markdown-mode (as I was using Markdown for posts, though initially I was using filtered HTML). Once a post was in Drupal, if I needed to make changes, I would use the Firefox extension ViewSourceWith to conveniently load the entry back into Emacs, edit it, then have it auto-load it back into the browser textbox once I was done editing. This worked, but it’s not as convenient or straightforward as editing a file sitting on my hard drive.

Advantages

With Jekyll, all posts are stored in files, so naturally the normal workflow is to edit files directly with a text editor. I expect this to be a lot nicer than working with a web interface.

Security is less of an issue with static HTML files. I only need to worry about security updates to the web server I am using, which is much less frequent than that of the typical PHP app such as Drupal (or Wordpress for that matter).

Though this site has never seen any heavy traffic that would test the limits of the server it was hosted on, I expect the performance of the static site to be much better than that of what a PHP / MySQL app can achieve (without proper caching at least).

I feel like I more empowered to customize and modify my site’s layout because it’s just HTML/CSS/JavaScript with a simple template engine (Jekyll uses Liquid).

Switched from shared hosting to a VPS

In this process I have also moved the site from a shared hosting provider to a Linux VPS (Linode). I am using nginx as the web server, whereas my previous host was using Apache.

Now that this is all in place, I need to get back to writing blog entries…

Script for Drupal to Jekyll conversion

It is in no way complete, but I wrote this script for converting Drupal posts to Jekyll: jekyll_drupal_conversion.rb

It is rather specific to my site, and may not work with someone else’s Drupal site, but could be a good starting point if you are looking for a conversion script. Be prepared to modify it if you plan to use this.

The script attempts to convert Drupal’s Filtered HTML format to normal HTML, but those HTML files probably needed additional work. It also attempts to convert the syntax highlighting tags I was using to those for Jekyll’s liquid tags which uses pygments, a Python based syntax highlighter. I actually ended up manually converting most of my HTML posts to markdown in the end.

The Rise of the Web OS

2009-07-08T00:00:00-04:00

I’ve been thinking about personal computers in general and its usage by non-technical people. Things are far from ideal, with Microsoft not getting any closer to making computing accessible to regular people, yet still remaining the dominant computing platform. But in observing the recent occurrences and in thinking about the future of personal computing, I think we are poised for a change. With the recent announcement of Google Chrome OS, I have some (long, rambling) thoughts on the subject which include why a web OS is the likely next step in personal computing and some predictions on it:

The Problem

Let’s face it, personal computers are too difficult to use. The average non-technical person has difficulty using personal computers, regardless of whether they are using Windows, Mac OS X, or Linux (though the Mac OS X users are marginally better off). If a non-technical user isn’t having any problems its probably because they have a very thorough technical person managing things for them. Even the highly technical person who spends 60/hours a week using personal computers is going to run into some problem that is going to require a good deal of effort on their part to troubleshoot. Having troubleshooting skills is in fact a requirement to be able to use personal computers without help. But why should it be?

Why Are Personal Computers Such a Pain?

Why are personal computers such a pain? One word: Complexity. Ridiculous amounts of complexity. Windows users have to deal with often difficult non-standardized software installation processes, having to install hardware drivers which are often of poor quality and difficult to install and use, having to edit the registry when things go wrong with updates, having to protect themselves from malware, etc. OS X users have a significantly better experience in these areas, but things are nowhere near perfect. Many Linux users sacrifice being to able to use the latest versions of software in the interest of simplicity as provided by their package managers. In all cases software updates and installations can and will cause problems. People are afraid to update their software as they have been burned by updates screwing things up. There are so many “moving parts” in a typical desktop operating system that there is a high likelihood that something will go wrong.

Complexity in the user interfaces increases the pain. Even OS X can only get so far in making a complex interfaces accessible.

What We Need in a Solution

The opposite of complexity is of course simplicity. To make computers accessible and manageable by everyone, we need to make them significantly more simple.

The user needs to interact as little as possible with functions that are not purely just using the applications they wish to use. They need to forget about the intricacies of software installation, updates, hardware setup, and even managing individual files.

The web offers to push many of these problems to the server, where there are technical people handling the issues instead of end users. And the users only have to worry about keeping their web browser updated (assuming the web sites don’t require some obscure plugins) and figuring out whatever UI the web sites offer. Even installing a decent web browser and keeping it updated appears to be a challenge for the majority of people, though.

What we need is a focused device that has a modern web browser (not Internet Explorer) and makes keeping that web browser updated as easy as possible if not completely automatic. And the web browser needs to be a first class citizen in the device’s user interface.

The Browser Interface Needs to Change

The iPhone and Google Android start to hint at what this might be like, but they don’t go far enough. What I mean by the web browser needs to be a first class citizen in the device’s user interface is that the web applications need to be the focus of the interface. Want to get your web mail or todo list? They should each be directly accessible from the main interface and you should be able to easily return the currently open session you have for the web application.

To take the example of the iPhone, the web application should be an icon on the home screen (which you can do), but instead of opening up the browser application and either opening a new page or switching a currently open page if it exists, it should just open up to the web applications. Admittedly, the iPhone’s way of handling this is good enough, but if we are going to build a whole device around browsing the web, we need to lose the idea of the web browser being an individual application. The web browser is no longer the application, the web applications are. Those of you who are familiar with Mozilla Prism you know what I’m talking.

I really hope that Google Chrome OS and any other emerging “web OS” gets this part right.

What’s Wrong With Traditional Applications?

A common response is that what’s wrong with traditional native/desktop applications (on the desktop or for specific devices such as iPhone apps). Why would you want to only use web apps? Well, we are making a sacrifice in the name of making things easier, but I think in the long run this sacrifice will be worth it. There are a number of problems which web applications and a web OS can solve:

Cross Platform

Native/desktop apps are either not cross platform, have separate builds for various platforms, or use a cross-platform toolkit or runtime. But even using a cross-platform toolkit or runtime only puts it on a limited number of devices. Web browsers on the other hand are everywhere. This may seem like an issue that only helps developers, and it does help developers as they can just make a web version and instantly have their applications available on more devices than any other method of app delivery would. But as a user, this gives you choice in where you can use the application. A web application can be used from all of your desktop computers, your netbook, your phone, etc. And it doesn’t matter which brand of device and OS you are using, as long as a competent web browser is available for it.

Updates

Users of web applications no longer have to worry about updates. They are automatically using the current version. Granted, this means the service provider is in control of what versions the users use, but its a trade-off that significantly reduces the work the users have to do.

A “web OS” that is basically just an interface around web browsing has a lot less to worry about updating. It’s main concern will be updating the web browser (especially security updates). There may also be some less frequent updates to things like the general user interface, the hardware drivers, etc. There’s no guarantee that updates won’t break things, but the less to update the better.

Convenience

I’ve already mentioned how you can use web apps on just about any device. Did I mention you could use them on any device? What I mean is that for most web apps you don’t have to carry around all your data as its stored on a server. So you can get on any internet-connected device with a browser to use a web app and have all your data available to you.

Inter-Application Connectivity

Web applications offer connectivity between each other in ways desktop apps can only scratch the surface of. This is a minor point at present, but we will see much more of it in the future.

Web Applications Can Have All the Power of Native Apps

If done right (both the web OS and the web application), web applications have the potential to provide an identical experience to desktop apps. Most of this is thanks to the use of plugins which give browsers functionality outside of what the web standards provide. While plugins start to erode the benefit that standards-compliant web browsers provide, I think they are necessary to push browsers into desktop territory. I think they could be a good thing if they are open source and accessible via standard web technology (JavaScript or HTML). I think Flash, Java, and Silverlight go a little too far away from the standard web technologies to be used prevalently (and their main implementations are not open source and are non-trivial to duplicate), but they do currently fill in missing pieces such as watching video, playing audio, and accessing webcams/microphones. But I think plugins that extend and build upon the standard web tech is a better idea than a plugin that provides an entirely different and separate runtime. Maybe things will look different 10 years from now.

Local Storage

Google Gears or HTML5 provide local storage. This answers some of the complaints about not having your data local, though it seems to be more often used for caching data locally (for offline use or for fast access) then as the primary means of storage.

Google Native Client

I almost view Google’s Native Client as cheating, as it removes the cross platform advantage. And the security implications are not to be taken lightly. But it does open up a huge amount of possibilities.

I’m not sure what the best way to approach actual plugins in web OS. The problem is that they are basically native applications. If you open up a web OS to allowing installation of arbitrary plugins, there is the potential for losing the advantages that a web OS is pushing (especially the cross platform advantage). Maybe its better for the web OS vendor to be in control of this.

Unfortunately, the locked down nature of the iPhone may actually be a good model for a web OS. Though ideally, the locked down should only be on the surface. The web OS would ideally be open source and allow for those wishing to break out of the lock down to be able to do so; but this is not something that the average user should be doing.

Not Everything is Perfect in Web OS Land

Of course, a “Web OS” is a trade-off and there are going to be disadvantages over traditional desktop/native software. Here are a few of the disadvantages:

Despite the possibility of local storage, in general, using web applications involves trusting a third party to store your data. This raises privacy concerns in some situations. Another problem is that you are trusting the third party to maintain integrity and reliability of your data. But I think most of these services are much better about backing up than the average user (though there is sometimes the odd web service that loses all its data and had never made a backup).
Security is a more significant issue on the internet. While viruses and malware can reach desktop applications, your data that is stored by third parties is open to attack 24/7, while your the data on your personal computer is only open to attack when your computer is on and connected to the internet. The security of the web browser is especially important, but I don’t think that is anything new.
Browser-based web apps are still catching up to native desktop apps. Even with Flash and Silverlight it’s difficult to deliver the same experience that native apps can deliver. I think in time we may see all the missing pieces get filled in, but this will be a slow process.
User interfaces are more the responsibility of the web applications. There is much less standardization among web apps. Similarities among the UI for web applications is generally due to them copying each other and not as much anything inherent in the platform (though there is a bit of it that is due to the platform). You don’t get the same kind of uniformity you would get with a toolkit such as Cocoa or GTK+ once you get into the more “rich” web applications. Maybe this is all a good thing and that more variations in UI will cause more rapid improvements in UI. But there is something to be said for having a consistent UI across applications that you use, especially for things like keyboard shortcuts. I hope that some consistency arrives for keyboard shortcuts in web applications, because the current state of keyboard shortcuts are not very good.

But these are all tradeoffs and not show-stopping problems.

Not For Everyone and Every Task

Let’s be clear, the web OS isn’t necessarilly going to be for everyone and every kind of task. The traditional desktop OS is still going to be alive and well and in use for a long time. But the web OS is going to become ubiquitous like televisions and telephones are. Though some of the devices it will end up on may be typical desktop and notebook computers, it will be even more prevalent on devices like small tablets and small notebooks (“netbooks” as they are currently called). I guess I sort of see the iPhone, Android, and the Palm Pre as devices that can be or are at least partially a “web OS”. And we are seeing Android being used in places where I think a “web OS” is appropriate.

It’s interesting that Google is creating a whole separate OS from Android, but I guess if you are really going to create a “web OS” there is no need for many of the things that Android provides.

Conclusion

So in conclusion, a “web OS” needs the following:

A competent modern web browser
Web applications need to be first class citizens
Automatic or extremely easy updates

If done correctly, I think this could be a good thing in that it could make personal computers much more accessible.

VMWare Fusion: Assigning IP Addresses for NAT-Configured VMs via DHCP and Port Forwarding

2009-06-24T00:00:00-04:00

In VMWare Fusion, when using NAT, you often times want the IP addresses for the VMs to always be the same, so that you can connect to them from the host using the same IP address every time (and probably assign an entry in your /etc/hosts file). You may also want to connect to the host from the guests using a constant IP address. And finally, with VMs that use NAT, you may want to setup port forwarding so that machines outside of the host can connect to services on your VM (and this relies on the IP addresses remaining the same).

The configuration files for doing this reside in /Library/Application Support/VMware Fusion/vmnet8/ (the NAT interface is called “vmnet8” hence why configuration for it is here), specifically dhcpd.conf and nat.conf. These files may be read-only (even for root), so make sure to give root write permission before you edit them.

Any changes to these files requires that the following be executed before they take effect:

sudo "/Library/Application Support/VMware Fusion/boot.sh" --restart

Assigning consistent IP addresses

As explained by this blog entry you can edit the dhcpd.conf file to configure VMWare Fusion’s DHCP server to assign specific IP addresses to a VM based on its MAC address. You should see something like this in your dhcpd.conf file:

subnet 192.168.87.0 netmask 255.255.255.0 {
    range 192.168.87.128 192.168.87.254;
    option broadcast-address 192.168.87.255;
    option domain-name-servers 192.168.87.2;
    option netbios-name-servers 192.168.87.2;
    option domain-name "localdomain";
    option routers 192.168.87.2;
}

For our IP address assignments, we want to pick an IP address that is outside of the range that is set above. So in my case I picked 192.168.87.100 and 192.168.87.101 for two VMs that I wanted to configure.

We also need to get the MAC addresses for the VMs. We can either get it from the VM’s .vmx file under the property ethernet0.generatedAddress or if the VM is currently running we can run ipconfig if it is UNIX/Linux or ipconfig /all if it is Windows.

So if we were to configure two VM’s to have fixed IP addresses, we would add something like this to the dhcpd.conf file:

host uvm {
    hardware ethernet 00:0c:29:02:00:f1;
    fixed-address 192.168.87.100;
}

host wvm {
    hardware ethernet 00:0c:29:44:57:46;
    fixed-address 192.168.87.101;
}

I put entries in my /etc/hosts to point to these, so I don’t have to remember IP addresses:

# Ubuntu VM
192.168.87.100        uvm
# Windows XP VM
192.168.87.101        wvm

Connecting to the host from the guests

Since the host acts as a router, we can use the router IP address as configured in dhcpd.conf to connect to the host from our VMs. From the above excerpt, this is the line that describes what the router IP address is:

    option routers 192.168.87.2;

And inside of the VMs, I add an entry to my /etc/hosts file so I don’t have to remember this IP address. Note that the VMs can see each other, so it’s a good idea to put IP addresses of other VMs in /etc/hosts as well if you are going to make connections between VMs.

# As defined in the VMWare host's dhcp.conf
192.168.87.2    vmhost
# Ubuntu VM
192.168.87.100    uvm

Port forwarding

To make services in NAT-configured VMs available outside of the host OS, we need to set up port forwarding. To do this edit the nat.conf file. Add entries under [incomingtcp] if you are forwarding a TCP connection and under [incomingudp] for a UDP connection. There are examples in this file, but to go along with our above examples, we might do something like this:

[incomingtcp]
# Forward host port 2222 to SSH on Ubuntu VM
2222 = 192.168.87.100:22
# Forward Remote Desktop connections to Windows VM
3389 = 192.168.87.101:3389

[incomingudp]
# Forward Remote Desktop connections to Windows VM
3389 = 192.168.87.101:3389

Restarting VMWare services on host

Don’t forget that after we make any of the above changes, we need to restart the VMWare services using this command:

sudo "/Library/Application Support/VMware Fusion/boot.sh" --restart

Prompt Before Closing Emacs

2009-06-13T00:00:00-04:00

I developed a bad habit of quitting Emacs from my earlier days of using Emacs mostly in a terminal session (where even there quitting it is not the best idea). Using GUI versions of Emacs, I sometimes find myself hitting C-x C-c and wishing I didn’t because I really intended to kill the current buffer, not completely close Emacs. So what I have done to deal with this is, when running a GUI version (which can be detected by checking the window-system variable), I always prompt to ask if I really want to quit. Here is the code I use:

(defun ask-before-closing ()
  "Ask whether or not to close, and then close if y was pressed"
  (interactive)
  (if (y-or-n-p (format "Are you sure you want to exit Emacs? "))
      (if (< emacs-major-version 22)
          (save-buffers-kill-terminal)
        (save-buffers-kill-emacs))
    (message "Canceled exit")))

(when window-system
  (global-set-key (kbd "C-x C-c") 'ask-before-closing))

I do this only for the GUI version, but you could remove the check for window-system if you want it to work in the terminal version as well. I use when instead of if in this case, because I do other things if I am running the GUI version, like (server-start), which I have excluded from this code sample.

SVN in Eclipse: Subversive and SVNKit for Subversion 1.6

2009-06-05T00:00:00-04:00

Previously I had blogged about using JavaHL in Linux. JavaHL is the bindings to the native Subversion libraries which uses JNI, and it sometimes requires a bit of effort to get working. I no longer find using JavaHL necessary as I no longer have problems with the pure Java library SVNKit (formerly known as JavaSVN). I also have switched to Subversive instead of Subclipse because I have found it to be less problematic.

Despite being an official Eclipse project now, Subversive won't install from Eclipse 3.4's default repository, and you still need to get some plugins directly from Polarion. If you want to use the latest Subversive and SVNKit that supports Subversion 1.6, add the following update sites:

http://download.eclipse.org/technology/subversive/0.7/update-site/

http://www.polarion.org/projects/subversive/download/eclipse/2.0/update-site/

And then install the following from those update sites:

Subversive SVN Connectors (2.2.0)

Subversive SVN Team Provider (0.7.8)

SVNKit 1.3.0 Implementation (2.2.0)

Note that the version numbers will change in the future, but these are the current versions that work with 1.6.

URL Shortener Web Application Using Django

2009-06-02T00:00:00-04:00

I created a URL shortener web application using Django (a Python web framework). I have made the source code available on GitHub and it is under an open source license (MIT license).

The shortened URLs use the base 62 value of ids of the model they are stored in, using the code from here (it uses A-Z, a-z, and 0-9), which should be fairly compact for a long time. A count of how many times the URLs are used is kept. The main page shows the 10 most recent and 10 most popular URLs.

You can see a running instance at n1l.us (which I am using to link to this blog and other links to my own content) and uu4.us (which I am using for all other URLs). I currently don't allow public submissions of URLs on either of those sites, but I will be opening it up for uu4.us at some point (perhaps once I add some defenses against spammers). The application itself has a configuration option to be able to require or not require login to be able to submit URLs (the REQUIRE_LOGIN setting in settings.py). Note that if you require logins, you have to use the admin app (at /admin/) as I have not created a separate login page.

I've so far done the bare minimum as far as HTML/JavaScript is concerned (and there is no CSS to speak of yet). Just enough to get things working. So don't expect a good looking site with this yet. It does produce a functional bookmarklet, assuming that you set SITE_NAME correctly for your site. Though I should probably make the bookmarklet check for rev=canonical to be a good web citizen.

This was developed using the current Django from SVN. I don't believe I used any features specific to 1.1, so it may work with 1.0.x, but I'm not absolutely sure.

Note that django-shorturls solves a different problem and is not a general URL shortener. It is to be used in an existing Django application to automatically map short urls to specific model instances and to produce the containing the short url in the templates.

The biggest issue with Adobe Flash Player

2009-06-02T00:00:00-04:00

Update: Looks like they have fixed this in Flash Player 10.1

The biggest issue with Flash Player at the moment is that there is no way to create a top-level exception handler. What this means is that there will be exceptions (errors) that you simply cannot trap in your application code, even if you put try/catch all over your code (and especially if you are using Flex's MXML). This means that creating things like an error reporting tool are impossible for applications that run in the Flash Player (that includes AIR apps, too, not just browser apps). If you are running the debug version of the Flash Player, you will see uncaught exceptions, but the typical user is not going to be running the debug version. In the regular version of the Flash Player, uncaught exceptions just fail silently (often causing strange behavior), giving no indication to the user that an error occurred.

There is an issue in Adobe's issue tracker for this:

http://bugs.adobe.com/jira/browse/FP-444

It is the highest-voted issue specifically for the Flash Player. It is only surpassed by the request to keep developing Flash Builder for Linux, which I think is an important issue, but this issue is much more important. If you haven't already, create an account on Adobe's issue tracker and vote for this issue.

Bash Completion

2009-05-23T00:00:00-04:00

The Bash shell does tab completion executables, bash commands, and files/folders out of box. But it’s completion can be extended to support even more types of completion including arguments for commands and things like hostnames through the use of programmable completion.

There is a package called bash-completion which includes a set of completions for various utilities. For example, for Subversion, if you type svn <tab><tab> (replacing <tab> with actually pressing tab) it will give you a list of all the svn subcommands that are available. If you type svn rev<tab> it will complete it to svn revert. For ssh, if you type ssh <tab><tab> it will list all the hostnames and IP addresses it knows about (it looks like it uses a combination of what’s in /etc/hosts and ~/.ssh/known_hosts). You can of course do ssh username@<tab><tab> if you are using a specific username.

Once the bash-completion package is installed, you can copy additional completion scripts to /etc/bash_completion.d and they will be automatically included. I did this with the Maven 2.x completion script.

Git has a completion script that is included in its standard distribution. Find out where you have Git installed, and source the script GIT_INSTALL_DIR/contrib/completion/git-completion.bash in your .bash_profile. For example, I have Git installed via the binary installer for OS X Leopard, and thus I added this line to my .bash_profile:

source /usr/local/git/contrib/completion/git-completion.bash

Note that this is independent of the bash-completion package described above and does not require it (since we are sourcing it directly).

Django also has bash completion script included in its distribution. Replacing $DJANGO_DIR where you have the release extracted (or checked out from version control):

source $DJANGO_DIR/extra/django_bash_completion

With this, you get completion for django-admin.py and manage.py. For manage.py you need to set execute permissions and run it as ./manage.py on the command line rather than python manage.py in order to get completion for it.

Suppress unchecked warnings on method calls in Java

2009-05-21T00:00:00-04:00

Let’s say you have a method call like this:

List<Object> list = (List<Object>) beanWrapper.getPropertyValue(key);

And beanWrapper.getPropertyValue)</a> returns an Object. If you have the “Unchecked generic type operation” compiler setting set to warning, you will get the following warning:

Type safety: Unchecked cast from Object to List<Object>

If you are using Eclipse, it will give you a quick fix to add a @SuppressWarnings(value = "unchecked") annotation, but only give you the option to add it to the current method.

(the current method in this case is getBean())

But we may not want to suppress warnings for the entire method. We can just put the annotation on the method call itself:

@SuppressWarnings(value = "unchecked")
List<Object> list = (List<Object>) beanWrapper.getPropertyValue(key);

CXF SOAP client built using Maven

2009-05-20T00:00:00-04:00

I recently created some SOAP web service clients using Apache CXF (which is a project that resulted from the merging of XFire and Celtix). Rather than hunt down all the jar files I needed and write an Ant script to do the build, I decided to use Maven for this. It worked nicely and saved me some time (I can’t say the same for some of the more complex projects I’ve tried to use Maven with).

I found the following blog entry very useful for setting this up:

How to create a WSDL-first SOAP client in Java with CXF and Maven

One thing I did different was use a property to specify the CXF version in one place:

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <cxf.version>2.2.1</cxf.version>
  </properties>
  <dependencies>
    <dependency>
      <groupId>org.apache.cxf</groupId>
      <artifactId>cxf-rt-frontend-jaxws</artifactId>
      <version>${cxf.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.cxf</groupId>
      <artifactId>cxf-rt-transports-http</artifactId>
      <version>${cxf.version}</version>
    </dependency>

Pay attention to what it says about the cxf.xml file. I used the Maven Eclipse Plugin to generate my Eclipse project files. If you do this, but forget the cxf.xml file, your client may still work fine in Eclipse, but when you try to use the Maven-built jar file, you will get an error like this:

javax.xml.ws.WebServiceException: org.apache.cxf.service.factory.ServiceConstructionException
	at org.apache.cxf.jaxws.ServiceImpl.getPort(ServiceImpl.java:314)
	at org.apache.cxf.jaxws.ServiceImpl.getPort(ServiceImpl.java:299)
	at javax.xml.ws.Service.getPort(Service.java:40)
...
Caused by: org.apache.cxf.service.factory.ServiceConstructionException
	at org.apache.cxf.frontend.ClientFactoryBean.create(ClientFactoryBean.java:59)
	at org.apache.cxf.frontend.ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:102)
	at org.apache.cxf.jaxws.JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:115)
	at org.apache.cxf.jaxws.ServiceImpl.createPort(ServiceImpl.java:434)
	at org.apache.cxf.jaxws.ServiceImpl.getPort(ServiceImpl.java:312)
	... 33 more
Caused by: org.apache.cxf.BusException: No binding factory for namespace http://schemas.xmlsoap.org/wsdl/soap/ registered.
...

So make sure to put the cxf.xml file in your src/main/resources folder (create that folder if it doesn’t exist).

Use web applications more like desktop apps with Prism (or Fluid)

2009-05-10T00:00:00-04:00

Many web applications these days work more like desktop apps and/or are applications that you use throughout your day. Using the applications through your regular browser presents a problem: to get to the application you have to switch to your web browser, find the window that you have webapp open in (or open a new window or tab if you're launching it for the first time), and possibly switch to the tab it is in. The steps it takes to get to an already open webapp vary depending on the current state of your browser and what windows and tabs you have open.

Instead, it would be much nicer if the webapp had its own entry in your operating system's taskbar or dock (or possibly in the tray if you are on Windows). With the help of Mozilla's Prism you can achieve just that. If you are using Mac OS X, you also have the option of using Fluid which uses the same rendering engine as Safari (Webkit).

What both of these do is build a standalone executable based on a given URL, title, and icon. This could even be useful for those deploying webapps to end users. I've seen Adobe Air used for this purpose, but Prism provides an open source alternative to this which doesn't require any additional runtime.

I've been using Prism for my GMail accounts and web-based todo list for a while now, and it makes using those very convenient (especially since these are apps that I use frequently throughout the day). The latest version (1.0 beta) has some support for badges (for example, can be used for showing number of emails in dock icon), but it looks like that is up to the web app itself to implement that (or possibly through an extension or through scripting). If you select "Show status messages and progress" when creating the app with Prism, the app will have to ability to install extensions, though I have not yet been able to successfully install the Firefox extensions I've wanted to.

Fluid, on the other hand, appears to have more features, such as a Greasemonkey-like "Userscripting", but I have had trouble getting it to actually work. For GMail, it tries to open the app up in my default browser when I try to login.

Use rlwrap to wrap command line apps with readline

2009-05-08T00:00:00-04:00

rlwrap wraps any command line utility with the GNU readline library, which gives you history and Emacs-style keyboard commands for line editing that you'd get with anything that uses the readline library (such as the Bash shell).

I started using it with Oracle's SQL*Plus command-line utility which lacks history and decent editing capabilities. rlwrap makes it much more pleasant to use.

There is a package for rlwrap in Ubuntu 9.04, so it is likely to be in other recent Linux distros. If you're on Mac OS X, Fink has a package for this. If you're on Windows, it looks like there is a package for Cygwin.

Get Emacs-style keyboard shortcuts in Cocoa text widgets

2009-05-05T00:00:00-04:00

Mac OS X already has defined some basic Emacs-style keyboard shortcuts (CTRL-N/P/B/F/K/Y/T/etc) which are available in Cocoa text widgets. But Cocoa is highly configurable, and allows for configuring a larger subset of Emacs-style keybindings (among many other possibilities).

These two references written by Jacob Rus are very useful for understanding what you can do with the Cocoa text system:

Customizing the Cocoa Text System
Customize the Cocoa text binding system

If you want to see my Emacs-style keybindings you can get it here. I did not enable incremental search yet as I couldn't get the input manager for it working yet.

Now this only gives you these keybindings when you're editing text in a Cocoa application. Non-Cocoa applications will not be affected. Firefox is not affected even though it uses Cocoa. It does appear to affect Java Swing apps, though, such as IntelliJ IDEA and Oracle SQL Developer, which both have Emacs keybindings that don't work properly until you make these changes (combinations such as Option-v output a special character otherwise, rather than performing M-v).

In a future entry I will talk about how to use Butler and Firemacs to give Firefox more Emacs-style text editing and keyboard shortcuts. Butler's ability to simulate keystrokes (and trigger on keyboard combinations) is pretty powerful.