Refresh

I’ve been playing around with a new web service called Stalkify for the past week or so. The notion is simple: Stalkify makes the social graph from Last.fm available in Spotify. So all the tracks you’re scrobbling to Last.fm (and have been for the past decade or so…) suddenly makes sense in the brave new world of Spotify.

Stalkify was inspired by @Claus’s Spotify DJ, which allows you to have peers listen into DJ sets. Classy clever used spotify: URIs and some Adobe Air magic to patch this piece of software together, but I figured that you could build the same live experience by using tracks recently scrobbled to Last.fm — and making a live-updated playlist in Spotify.

So that’s the first feature: You’ll key in a username (or your username) and you’ll get a live playlist of what’s being played by that user.

An awesome upside of using Last.fm as the data exchange is that you’ll also be able to stalk non-Spotify users. For example, my good friend PolleTheWonder is a heavy Last.fm user with a list of almost 20.000 listened tracks; but he isn’t a Spotify user. With Stalkify, I’ll still be able to listen live to what Paul is playing.

I have four years of Last.fm usage and 22.559 plays to my name, and Last.fm is pretty good at collating this information into a sense of my musical taste. For example, they compile lists of my favorite tracks (from both my Winamp, my iTunes and my Spotify days), which is something I’d love to have ready at hand in Spotify. Stalkify does that: It give me a list of daily updated list of what I’m listening to the most.

(I’ll be updating Stalkify to use more features from Last.fm in the future: Top albums, Recommended artists, Neighbouring users…)

For the geeks…

Stalkify is pieced together from The Last.fm API using pylast, from libspotify (0.0.3 because 0.0.4 is buggy in playlist handling) using greenstripes and Spotify’s Metadata API using spotimeta.

All the code is a) ugly and b) available at github.com/steffentchr/Stalkify.

The node.js project has been getting a lot of good press and positive attention over the past few months. With good reason. If this is the first you’re hearing of node.js, it’s officially an “evented I/O for V8 JavaScript” but more generally it is used as a tool for writing non-blocking network servers. This means that it can easily be made into a JavaScript-based and highly efficient web server. If you’re curious, have a look at Simon Willison’s nice write-up.

Node is still in it’s infancy. For example, there’s is still a lot of discussion going on on the mailing list about core API (i.e. how to handle communication between and chaining of events or even how the final file API is going to look). Since node isn’t designed explicitly as a web server, there’s only low-level http support — and of course, there is no standard web application framework for handlings requests, sessions, cookies, and users.

The first question to be answered is if you’d actually want big web applications to run solely on node.js? But for now I’ve skipped that question entirely and simply asked how I’d approach writing such and application (hopefully reusing some of the code being currently being pushed by the node community)?

It seems as if every person in that community has written her or his own lightweight library for handling web requests. To name a few, there’s Djangode, Express, Coltrane, Vroom, and node-router). A lot of these are obviously inspired by the approaches taken by either Django or by Rails (it’s right there in the names), but taken as a whole they’re very limited in how they approach the task. Most of the frameworks are just different syntaxes for binding a JavaScript function() {} to a URL — without the modular approach taken by their more mature counterparts.

This means that you can easily use any of these frameworks if you need to write up a simple — and fast! — web app in a single file with a few hundred lines of code (in that case, I’d opt for Djangode), but you’d be stomped if wanted different, re-usable applications meshing into a single project. Benjamin Thomas has gone in designing mode in order to come up with a mini-framework which isn’t quite as mini as the ones listed above. His ambition is to build…

- The ability to bundle up a collection of controllers/views, models and templates into “apps” for easy portability and reuse.
- Smart defaults that hide a lot of the options/power/complexity. I think this is important for getting up and running quickly.
- A code base that is as small and simple as possible.
- I want it to feel javascript-like. Whatever that means.

The end result is Bomber. I like how Bomber creates simple URL routing to file-based (or rather node.js module-based) views, and how it lightly extends the core http.ServerRequest and http.ServerResponse objects. It feels almost native, but you see that enough of a foundation exists for this framework to actually include the ability to handle POSTs, sessions, cookies, multiple applications within a project and so on. There are no great ambitions where the framework will suddenly include a native XML parser (just a random example, sorry), but there’s enough room to grow and spread the webby wings a bit.

The Bomber project is very much in flux. In fact, it might even be dead since the last commit is a month old. And the entire project is build around an Action API which about to be deprecated. Hopefully to be replaced with DOJO-style, native Deferreds.

I’ve been playing around a bit by adding some extra stuff to Bomber through my own fork at github. I’ve already tweaked a little with the app structure to allow for multiple projects to be bootstrapped with different configurations from the same Bomber library, and there is some copy-paste and bug fixes in there as well. This will likely include experiments with multi-application projects, per-application media directories, cookie and session handling and templating (although I disagree with Benjamin that Bomber should include its own templating system; this could probably be left for the programmer to choose).

And now, back to Jersey Shore, bitches!

I’m beginning work on a project which requires users to be able to navigate the content of a web page using only the keyboard, and where switching focus using the standard method of tabbing is not an option either. To solve this problem, I’ve pieced together KeyboardFocus.

KeyboardFocus does one thing: Look through all elements on a web page with a given class name and make those elements selectable and clickable through the keyboard — using the arrow keys and the enter key.

Using the code is really quite easy:

• Add the CSS class name focusable to all the items you need to be addresseable with the keyboard.
• Add some CSS style for .focusable, for .focused and possibly for .hovered.
• Include the Prototype and KeyboardFocus scripts in your header.
• Add the line var kf = new KeyboardFocus(); somewhere in your code.

It could look something like this:

<html>
  <head>
    <script src="/script/prototype.js"></script>
    <script src="keyboardfocus.js"></script>
    <style type="text/css">
    a {color:#06c; text-decoration:none;}
    .focusable {border-bottom:1px solid #06c;}
    a.focused {background-color:#06c; color:white;}
    </style>
  </head>
  <body>
    <ul>
      <li>A link <a class="focusable"
          href="http://www.google.com">Google</a></li>
      <li>A <a class="focusable"
          href="http://refresh.dk">different link</a></li>
      <li>A <a href="http://www.yahoo.com">
          non-focusable link</a></li>
      <li>And something <a class="focusable"
          href="http://xkcd.com/">completely different</a>.</li>
    </ul>
    <script>
      new KeyboardFocus();
    </script>
  </body>
</html>

There’s also an example with random boxes.

A while back a spent a little time creating a simple service that allows tune-tweeting with legible, short URL. For example, twune.net/DioDiver will play Holy Diver by Dio. And twune.net/BeatlesHand will give you The Beatles’ I Want To Hold Your Hand.

The scheme was inspired by some of the cool work that Morten Just has been doing by mashing up YouTube videos with different data sources — and I was happy Morten let me re-use some of his code.

The service works by parsing the search string from the url (using WikiWords or whatever you’d call them). It then tries to find a perfect match on Yahoo! Music which will be played in Yahoo’s video player. If that doesn’t work, the search is passed on to YouTube with the help of Morten’s player. It’s that simple.

The name of the service was suggested by @guan. I had originally wanted to call it Twack. Still unsure which is best?

About a week ago some friends launched Twittertinget.dk, a sleek app that aggregates tweets from politicians in the Danish parliament. In the name of democracy, the web application has a vital flaw though: It only lists tweets from politician, not to them. That makes for one-sided communication.

As a friendly gesture, Socialsquare asked me to create a complementary service, which does the exact opposite thing: Let’s you know what people are tweeting to their elected representatives. This ended up in Folketwinget.dk, a simple site hosted on Google AppEngine.

Now, aggregating tweets to and from the Danish Folketing seems like a faily specific, but it turns out that such an aggregator of can easily be used by any other group — so we’ve open sourced the code over on Github. The code runs directly on AppEngine so it’s easy to host and have running in a few minutes.

For example, if you have a company called 23 with a few employees on Twitter it might be cool to show all the incoming and outgoing tweets from the company.

Get started by installing the Google AppEngine SDK, start up a terminal and browse to the AppEngine folder on your local machine. Then…

1. Get the code
Run this command to download the code. (Obviously, you’ll want to change the folder name from 23tweets to something else.)
git clone git://github.com/steffentchr/twitterengine.git 23tweets

2. Update app.yaml and create config.yaml
Open up the file called app.yaml in the newly created folder and change the application name in the very first line of the file. In this example, we’ll change it from twitterengine to 23tweets

Also, rename the file called config.yaml.sample to config.yaml

3. Create an AppEngine project
Open up GoogleAppEngineLauncher (it’s called that on Mac anyway). Right-click in the window and select “Add existing..”. Now select the folder we’ve just created and click OK.

This is all you need to actually get the application up-and-running. Click the Run icon to start your development server and open up http://localhost:8000 in a browser.

4. Configure the application for your dirty purposes
Before you can import some tweets into your application you will need to update the configuration. Open up config.yaml in a text editor and change what you think is necessary. For my example, I’ll use this configuration:

site_name: 23 tweets
site_tag_line: What is up with 23?
site_domain: 23tweets.appsport.com

twitter_username: 23pull
twitter_password: secretpassword

users:
- steffentchr
- mygdal
- mmmmmariaaaaa
- abemad
- guan
- burtblancher
- 23

hashes:
- 23
- 23hq
- twentythree

Basically: Use the user account of 23pull to get all tweets to and from @steffentchr, @mygdal, etc and all tweets with the hash #23. Then call the site “23 tweets”.

(This could just as easily have been used to create a Folketwinget-sibbling for the Norwegian parliament or for aggregating conversations of everything involved in a specific open source community.)

4b. Import the tweets
Now, go to http://localhost:8080/import to trigger an import of everything specified in the config file. When you return to http://localhost:8080/ you should see all a batch of the latest twitterings.

When you’ve finished step 5., the application will automatically import tweets every five minutes. The frequency can be changed by editing cron.yaml.

5. Deploy your app
Before you can deploy your new twitter aggregator to the wide web, you will need to create the application with Google. Go to the AppEngine dashboard, click the “Create an application” button and fill in the form. Remember to use the correct application identifier, i.e. 23tweets.

When your application has been succesfully created, go back to GoogleAppEngineLauncher and right-click your project. Then select the deploy option and sign in to your Google account. That’s it. It’s online.

Again, you’ll need to visit http://projectname.appspot.com/import to import tweets into the online database and then go to http://projectname.appspot.com/ to see the result.

6. Customizing the design
You’ll find that your site is in Danish and generally doesn’t look the way you want it to. Changing the design though is fairly easy: Modify index.html and resources/style.css for your purposes. If you need anything fancy, the html file is structured using the Django templating system.

Update April 24th 2009: Originally, this post contained information for running external cron jobs to import tweets. Today, the code has been updated to utilize AppEngine’s new native cron feature which makes it all happend automagically..

Wouldn’t it be nice if Google Gears included the entire Chromium rendering engine?

A few people have been posing the idea of using Flash’s own WebKit-powered browser to render web pages in Internet Explorer, which would achieve the duplicate goal of leap-frogging the web platform forwards and not forcing users to upgrade their browsers. (Such an approach would be a hack at best, and it would introduce other problems.) In a similar vein, we’ve seen people build plug-ins for IE from pieces for Mozilla to upgrade the Explorer’s capabilities.

Up to a few days ago, Google’s approach to this problem was to distribute light platform upgrades (such as offline browsing and location awareness) to existing browsers in their Google Gears plug-in. With the release of Google Chrome this strategy has shifted: Rather than upgrading the existing platform, Google is making a play to replace the platform. Presumably Gears is going to co-exist with Chrome. We’ll see development on Gears as long as support in Google’s products for IE, Firefox and Safari is needed. Which is going to be a long, long time.

Now, here Google presented with an appealing opportunity. In addition to just embedding Gear in Chrome they could embed Chrome in Gears. This would allow developers to require Gears in Internet Explorer (IE, of course, being just a random example) and then use Chromium to render pages rather than IE’s own native engine. Intriguing, right?

I was surprised to learn that there’s no way to access a user’s location history in Fire Eagle. And, in fact, Yahoo only stores the most recent user location. This a really, really, really limits how Fire Eagle can be used from both a user perspective and from a developer point-of-view.

As a user I’d much, much rather have Fire Eagle store my history centrally (and even forbid developers for storing this history themselves). This would allow me to control which applications can (at least in theory) access my trail, and which parts of the trail is available. To me, relegating this responsibility to 3rd-party developers negates much of the purpose of FE, because then there’s no one-stop place for storing and accessing locations data.

As a developer, I’ll probably need to work around this limitation by building a FE-specific data model for storing locations and by polling for user locations every five minutes or so. This seems highly inefficient, since I won’t be the only developer facing this problem.

Read more in the Fire Eagle discussion group…

I wrote about the huge potential in government as a service the other day, and apparently i’m not the only one who has seen the opportunity. The only difference is that while I just wrote a few thought and did nothing, others have actually done exciting work on this front. Some articles in English:

• Public Danish Data REST Interface
• OIO REST: RESTful web services developed by the Danish Public Sector

And more in Danish

• Free Willy
• OIO REST
• Danmark Web Service
• Denmark in REST
• Write-up of the OIO REST workshop

uploadr is a demo of Yahoo’s upcoming BrowserPlus and it looks amazing. When you install the browser extension, developers get access to features usually not available in browsers. Such as drag-and-drop from the operating system and browsing-for-multiple-files. That should some on handy. And I want it for 23. Now.

Now the question though: What do I do with those files? How do I transfer them from the user’s computer to my server? According to the current list of services, that isn’t on the menu for version one. From the demos it seems that the only way to actually use the dropped or browsed files is through Ruby (on which a FlickrUploader corelet is based). Weird. It’s nice to see Yahoo trying to compete with Google on openness and extensibility, but am I the only one seeing a big black whole in the service offerings here?

The Notify service look interesting as well. Too bad that it’s text based only. I’d love be able to show recently uploaded photos to interested people visiting 23. Or buddy icons when people post on twitter. Or … [Update: Even better. There's a Yahoo logo on popup notifications in Growl.]

ReadWriteWeb just posted an intriguing article about the next generation of e-government. The post basically lists the arguments from an upcoming scholarly article about the subject. The main argument is simple: Government shouldn’t spend it’s time building web sites, instead it should “focus on creating a simple, reliable and publicly accessible infrastructure that exposes the underlying data”. Rather than building and maintaining a website of upcoming bills at ft.dk the Danish parliament should make the data available and have others mash it up in usable ways.

Now, I’m not a deregulating liberalist, and I’m absolutely certain that a strategy of only making data available to private entrepreneurs simply wouldn’t work in a Danish contexts. But I’m also convinced that a mandated policy of making data easily available to developers has explosive potential.

Let me list a few examples. The best Danish mapping service, Findvej.dk is a mashup of both publicly and privately owned data. It was built by one guy in with a good idea and access to the necessary data. The data was probably hard to find, and the service would be much better with even more accessible data. But you get my point.

There are hundreds, if not thousands, of possible counter-examples and I’ll just pick one. The weather. Denmark has a tax-funded, public weather service, which generates plenty of data every day, and this data is available to everyone through a web browser. Now, given all the data from the weather service, would I be able to build a better website serving all the same user groups as DMI? Probably not. But given access to that data I would be able to build a service which suits my needs much, much better that DMI.dk. And most likely, my service would serve others as well.

Why isn’t there an Danish version of EveryBlock? The reason isn’t lack of interest. The reason isn’t lack of data, because statistics and incident reports are being kept by the police. The reason is that the data isn’t accessible as an open building block (that metaphor didn’t work).

This presents a major challenge to government, and even if I’ve used Danish examples, those challenges are probably both very global and very local. The past few years have shown convincingly that web developers are hungry for interesting data to reinvent by merging across data sources, and this potential could be easily harvested by some sly government initiatives.