I've had a project on the back burner for a while that involves pushing data into LSL (the Linden Scripting Language runtime that Second Life uses) over Comet.

And it works amazingly well! Of course, I just had to toss REST in there as well.

I'll release this code as open source next week. Meanwhile, here's a movie to show you what the hell I am talking about. I also like how this movie shows off the surreal aspect of collaborative programming!

http://twitter.com/donovanpreston

Picked this up at Kid Robot in the Haight this weekend. It is so freaking awesome. I want more, but can't read japanese :-)

Recently I was tasked with discovering whether it is possible to do non-blocking file reads and writes to a filesystem that is mounted over NFS. I tried on OS X, and I was unable to get a read to return EWOULDBLOCK. So, I decided to install Ubuntu on Parallels. I downloaded the iso, burned it to a CD, created a new virtual machine, and installed it.

Everything worked flawlessly. Ubuntu has always been incredibly high quality, and it has only gotten nicer in the year since I used it last. It's polished, beautiful, and just works. It is definitely something that I could install on my machine for my Mom with a web browser and mail reader.

Here's a screenshot:

import readline
readline.write_history_file('my_history.py')

I'm sure this is going to come in handy many times.

Very busy at work lately. Getting a lot accomplished, and it is challenging, fun work. We're starting to ramp up pretty fast and I have been doing scalability work on the backend, which is surprisingly enjoyable after doing web front end work for so long.

I'm at SuperHappyDevHouse VII tonight. I went to II, and had a good time, but haven't been able to get back until now for various reasons. It was supposed to be last weekend, which falls on my weekend I am scheduled to work in the city, but for some reason it got bumped to this weekend. My goal for tonight is to release some code which was written almost a year ago: NevowPavel. If you have a short attention span, watch the screencast, wherein I explain the basic idea behind the project and give a demonstration of the currently implemented features.

This is the secret LivePage project I wrote about the last time I went to SHDH. It is a multiuser real-time updating wiki similar to and inspired by (the precursor to) Jot Live, the "live" version of Jot that broadcasts real-time updates to everyone that is participating. The difference with the NevowPavel implementation is that it is "spatial". Instead of editing a contiguous page of text, each piece of text is contained in a sticky note which can be moved and resized at will to organize the information as one wishes.

Links are also handled in an interesting way. Links are inserted by dragging out a link instead of a sticky and typing a new page name. Following the link takes you to the new page, where you can create and organize new pieces of information. Each page has a user list, so you can see who is there editing with you. To organize information cross-page, you drag stickies onto links. People who are on other pages see the information appear magically.

The ultimate goal of these template objects which you can spatially place in the page is for them to be pluggable. New types of objects should be able to be created which can provide code which renders content and handles input events for instances placed in the page. Eventually, the code should be through-the-web programmable. Obviously this involves lots of different things, like run-time code modification, restricted code execution, perhaps distributed code (to move code across multiple Pavel servers), and some sort of persistence. I'm also working on a Prototype-based object system with very simple persistence which is coming along well, but isn't integrated with the old project at all yet. This is a multi-year project, so check back again next year :-)

Get the latest version of Flash to view Slide Show, above. Visit Slide.com to create your own Slide Show

I have no idea how I came across your site. Probably after reading the wikipedia Chiptunes page and searching around a bit (since you don't seem to be linked on there... go add yourself!). I had the contents of your Chiptunes page on my ipod for a while before I really listened to it a lot. The first song I downloaded and listened to a lot was the thriller cover, which I thought was just so cheesy to be brilliant. I used it to test a Mac audio editing application I was working on this summer, SoundFarmer. Check it out:

http://donovanpreston.slide.com/c/SoundFarmer

Specifically:

http://item.slide.com/y/uid=5eilEp2bBu3zp7fdnfWThvdchdJNe1P50VhphDVUTV35s_UPnkCLsZB6WKutK_7Y/SoundFarmerAugust05.png

I didn't really listen to much of the other stuff from the chiptunes page or thriller for a while. Then, I was at the gym yesterday on the treadmill, sorta in the zone, just closing my eyes and concentrating fully on what I was hearing. I think it was in the middle of "blastoff", right about 1:08, where the progression slams right through the fucking roof and the chorus kicks in and breaks it down and then the next section starts slicing microthin shavings through your brain with the distorted guitar and then here we go up and up and up and arpeggio and staccato and tension hold and release and OH FUCK YEAH bring it back around to the chorus again!!! I was almost in tears. I actually had chills. Goosebumps. (All while sweating my ass off at the gym!) I am not shitting you.

Anyway, I went and downloaded every other single mp3 I could find on your site earlier this morning (say hi to your server logs for me ;-) and was digging the gameboy choons but it wasn't until I got to MC Nachbar that I realized that you are a fucking GENIUS. GEE-NEE-USSSSSS. A fucking polymath. So I wanted to mail you and let you know. You made my year.

Nostalgia and speculation on the nature of human creativity follows:

When I was in highschool, I found a copy of SoundTrecker (a Mac mod tracker) and a few mods somewhere. Pretty soon I was downloading mods from boards as fast as I could. At one point I had an entire syquest 44 cart filled up with em. Pretty amazing over a 9600 baud modem! I remember when I eventually had a 28.8 and I could download mods faster than I could listen to them... that was quite an epiphany (I had the same epiphany when I discovered mp3 ftp servers a few years later when I worked at the state of michigan, which had a T3. I have not yet had that epiphany with movie torrents yet, but I have a feeling we're close. I also don't really like video as a media). Then when I was in college I found my favorite song ever... FOOP.MOD, a brilliant breakbeat choon that I still love to this day. I was also buying CDs at the time from various random techno acts of the day, but with the exception of Aphex Twin, Luke Vibert, and Mike Paradinas, I was continually disappointed. There was just something about the raw emotion and uncut feel of mods that compelled me much more than some random crappy-ass major label release.

There is a psychic difference between the product that one creates for love versus money. When someone really, truly loves something and does their absolute best, it bleeds out between the lines, oozes all over your hands and sticks to you and won't wash off. I work at slide because I love programming, and the fact that I get paid is a pleasurable side effect. It is this raw hot plasma passion that fuels the internet, the blogs-versus-journalists and mp3-versus-record-mafia battles. The 9 to 5 work week and the boss-employee fiction is a recent creation in human evolution. The creative drive of which I speak is not. It may not happen tomorrow, but the politcal and social fads of the day will fade. They will be replaced with new social customs and norms. Each step of the way humanity will delude itself into thinking this custom is "natural" and has somehow always existed. But the only thing that has always existed is the creative big bang. God. The Universe. Order from chaos, strange attractors in the cellular automata. Find it, feel it, follow it, hold on.

So yeah. Thank you for who you are and for letting me see it through the magic of the internet.

Recently, I changed jobs. I joined a company called slide, which is building a hybrid photo sharing application using a combination of fat client and rich web technologies. In my drive to create responsive web applications I have been exploring techniques which make heavier and heavier use of client-side JavaScript. I now find myself doing less and less Python and more and more JavaScript rendering code.

I don't really mind this too much. I think JavaScript is a reasonable language, with reasonable anonymous function syntax, real closures with mutable parent scopes, and prototype inheritance (with the most bizarre implementation that unfortunately renders it almost unusable). However, every once in a while I hit some issue which drives me absolutely insane trying to debug.

Here is what I ran in to. The following:

{'foo': 1}

Is not valid syntax. However, the following:

foo = {'foo': 1}

Works fine. I ran in to this issue while trying to eval constructed strings of javascript. Even though eval returns a result, giving it a single JavaScript object literal results in an exception. Tacking on an assignment at the beginning fixes the issue. Ugly.

Before boarding a plane for Detroit, I spent a night at a hotel in San Francisco. They apparently had internet access from out of the 19th century.

The view from my hotel in San Francisco was stunning...

...while the view from my hotel in Detroit is less so...

...and apparently in a low-rent neighborhood.

The snacks at MacHack are sponsored by Google this year and are therefore of very high quality...

...especially the candied fried eggs, which are delicious.

]]> 2007-08-02T04:38:48Z 2005-07-12T04:08:52Z tag:ulaluma.com,2005:/pyx//4.1463 2005-07-12T04:08:52Z

Thank you.

I spent a few years coding, but it was slow going. Both network applications and GUI applications require a different mindset than the batch process style of programming which is what you learn in computer science classes. Event driven programming takes a lot of getting used to, and even when you understand what you are doing, it can be difficult to write and debug event-driven in C or C++, the language of choice in the mid 90s.

In 1999, I was doing a lot of Flash work and saw how it might be possible to tie a Flash front end to a multiuser back end server. When Flash 5 came out, I decided to revive the idea of a graphical front end to a multiuser programing environment. I wrote a simple HTML page which contained some JavaScript DHTML code I put together for mutating the DOM and embedded a small, wide Flash movie towards the bottom of the page. The movie contained an input text box and some ActionScript which sent the contents of the box over an XMLSocket when the user pressed return.

Because the XMLSocket used null bytes to delineate each message sent across the wire, I hacked the C source of the MOO server (running a MOO core my friend Pictwe and I had been running since 1995 -- MOOf) to send null bytes between each "line" of output from the server. Because of restrictions Flash placed on the hosts and ports to which you could open an XMLSocket, I had to create an application server capable of both serving the HTML, JavaScript, and Flash files, and proxying the XMLSocket connection to the actual MOO server. I had done a few simple CGIs in perl and thought about using that, but perl sucked. I did a bit of web searching and discovered Python.

After reading the socket and threading documentation (old habits die hard) I had a simple server working. You could load the web page, type commands in a text box, press return, and your command would be sent to the server where it was executed like it came from any other client being used to access the MOO. Any time the MOO generated any output on your connected socket, the intermediary server would push it into the browser over the XMLSocket, the ActionScript would unpack it and send it to JavaScript using LiveConnect, and the JavaScript would use DHTML to change the page. Sound like AJAX? :-)

Shortly thereafter I got a full-time job writing Python web applications. After developing for a few years in Webware and Zope, I decided to play around with some ideas I had for an easier to use templating system. This led to the creation of DOMTemplate, Woven, and Nevow, and it turned out to be a longer process than I had hoped. Sometime in late 2002 I decided to revisit the idea of performing out-of-band communications with a server to allow the web application running serverside to push and pull information into and out of the browser. I called it LivePage, and it was based on a highly transparent Model-View-Controller design, where controllers received events from the browser, updated models, and views automatically re-rendered themselves based on model dependencies. The results were then shipped to the browser and some DOM hackery was employed to replace the old DOM fragment with the newly rendered view.

The original Woven implementation of LivePage used Flash XMLSocket as the out-of-band event conduit. XMLHttpRequest was around at the time, and I did attempt to use it, but it reeked of MSIE nastiness and wasn't very standard cross-browser. XMLHttpRequest also has the distinct disadvantage of being based on HTTP. XMLSocket is a persistent, two-way, asynchronous socket architecture that made it delightfully easy to implement both Client-to-Server events and Server-to-Client events. XMLHttpRequest can only send data to the server once, and if you repeatedly send data from the server to the client and handle it incrementally, the document will consume memory without bound.

People thought I was crazy to even try to make web browsers do the things I wanted to do. It'll never work, it'll never be cross-browser, browsers can't handle it, people won't understand it... But I persevered. Browser stability was a real problem. After leaving a live page open for a while, with lots of changes being sent, browsers would leak all over the place and eventually crash. Both IE and Mozilla crashed like crazy. Eventually, after Gmail came out, the browser vendors seemed to get things under control and DOM mutation is relatively reliable now.

In October of 2003 when I sat down to spend a bit of hacking to try to come up with ways to simplify Woven, Nevow went from proof-of-concept to ready to use in a weekend. It wasn't until Gmail was in private beta that I decided it was time to revisit LivePage. This time, the aim was to keep the implementation as short and easy to understand as possible, with the hopes that other people would be able to figure out how and why to use it. I also decided to use XMLHttpRequest as an experiment, to reduce the number of dependencies. XMLHttpRequest makes some aspects of the architecture more difficult (Server-to-Client events) but in some ways makes other parts of the implementation much easier.

Since I only get to work on LivePage in my spare time, it has been slow going. However, the implementation as it is in Nevow is now almost ready to tackle the task of constructing very dynamic, complex applications. There are some implementation details which, after being exposed to the real world for a while, need refactoring. I have a few ideas on how to make it even easier to use from an end-user programmer's perspective. Finally, a coherent idea of how error handling occurs between client and server is starting to take shape. In the latest livepage branch which includes some unfinished improvements, which I hope to merge in the next week or so, the server can reliably handle an exception which occurs on the client. That's right, Python code can be written to handle client-side JavaScript exceptions. Also, LivePage as it exists in the 0.4.1 release has very robust "User has left the page" notification support. If the user clicks the back button, closes the browser, or even crashes their machine, the server is guaranteed to know about it no more than a minute or so later.

I have spent 10 years of my life, on and off, working towards this dream I have. Nevow and LivePage are the latest incarnations of puzzle pieces which fit into the mosaic of this dream. Hopefully someday very soon I will start fitting the pieces together and start to see what a true multiuser web application looks like. We already have a taste for what collaboration can enable; sharing information using weblogs, instant messengers, wikis, file sharing applications, and tools like SubEthaEdit allows to more rapidly fine-tune our idea of what others are thinking. Only a few pieces are missing; a more graphical, spatial ability to arrange information, where information can be picked up easily and carried from place to place in order to sort and categorize it; a more real-time sense of community where presence information gives us a sense of who we are near and allows us to easily communicate with them; and a more immediate programming model where changes can be made to live applications and applied immediately, with results available for all to see. All the pieces are there, they just need to be put together to complete the puzzle.

ssh to the remote host and:

mkdir .ssh
chmod 700 .ssh

Back on the local machine:

cat .ssh/id_dsa.pub | ssh remotehost "cat > .ssh/authorized_keys; chmod 600 .ssh/authorized_keys"