Pages

Saturday, October 11, 2014

How to Write Hello World in Go (in Go)

Let's write "hello world" in go.

Sure, you can do it this way:

package main

import "fmt"

func main() {
fmt.Printf("Hello world.\n")
}

But that's no fun, is it?
"Give a clever engineer a straightforward problem and they'll add complexity until it's interesting enough to solve."
Let's write a program that writes hello.go for us!  That's right: write go, using go.

(Sure, you could just write a program that echos the above source text as a quoted string to stdout, but that's not much fun. We must add more complexity to make it interesting.)

The "go/ast" package is used by gofmt and gofix (and various other tools) to parse and manipulate go source code in the form of abstract syntax trees.

We're going to build a new go program from scratch, in go, so we're not really interested in parsing so much as constructing an AST and printing the result as human-readable source code.

Here's a skeleton that uses go/ast to construct an AST for a very minimal go program that compiles, but doesn't actually do anything:

package main

import (
"bytes"
"fmt"
"go/ast"
"go/printer"
"go/token"
)

func main() {
// Start with a file
f := &ast.File{
Name: &ast.Ident{
// The package name is "main"
Name: "main",
},
// Top-level declarations in this file:
Decls: []ast.Decl{
// A basic func declaration with no receiver:
&ast.FuncDecl{
Name: &ast.Ident{
// This func is named "main"
Name: "main",
},
// With an empty func type (no params, no returns)
Type: &ast.FuncType{},
// And an empty body.
Body: &ast.BlockStmt{},
},
},
}

fset := token.NewFileSet()

var buf bytes.Buffer
printer.Fprint(&buf, fset, f)
fmt.Printf("%s\n", buf.String())
}

Try it out on play.golang.org here.  It produces the following:

package main

func main() {
}

Which does compile, but doesn't actually do anything.  Let's add the next pieces: The import statement for "fmt" and the fmt.Printf statement that actually prints "Hello world."

To add the import statement, add a new element to f.Decls:

// Start an "import" declaration
&ast.GenDecl{
Tok: token.IMPORT,
Specs: []ast.Spec{
&ast.ImportSpec{
// With a string literal of "fmt"
Path: &ast.BasicLit{
Kind:  token.STRING,
// Note the "" contained in ``
Value: `"fmt"`,
},
},
},
},

If you leave f.Decls as it is here and run it, it will produce the following:

package main

import "fmt"

func main() {
}

Which will fail to compile because "fmt" is unused.  So let's use it by adding the fmt.Printf statement inside the body of main(). Change the empty Body: &ast.BlockStmt{} in the above skeleton to include some ast.Stmts:

Body: &ast.BlockStmt{
List: []ast.Stmt{
// Start a stand-alone expression statement
&ast.ExprStmt{
// Representing a function call to "fmt"
X: &ast.CallExpr{
Fun: &ast.SelectorExpr{
X: &ast.Ident{
Name: "fmt",
},
// With a selector for Printf
Sel: &ast.Ident{
Name: "Printf",
},
},
// And a single-element arg list consisting of a string literal
Args: []ast.Expr{
&ast.BasicLit{
Kind:  token.STRING,
Value: `"Hello world.\n"`,
},
},
},
},
},
},

This will finally produce a runnable hello world:

package main

import "fmt"

func main() {
fmt.Printf("Hello world.\n")
}

Try the final product out on play.golang.org here.

In conclusion, if you'd like to do some code generation with go, this might not be a bad place to start. You can explore other parts of go/ast by adding some extra function declarations with actual parameter lists, return values and even receiver types, and then calling them from main.

If I was really bored, I'd write a further iteration of this program that constructs itself :)


Tuesday, September 21, 2010

CSS3 Star Wars Tweet Scroller

I've been playing around with some new CSS3 3-D transforms and whipped up this Star Wars scroller for Tweets:


Caveat: This currently only works on Safari.  It may work on a bleeding edge Chrome dev build but I haven't tried that.

There are some other examples of this type of scrolling effect out there but I wanted to make one for tweets.  I also noticed that there are some odd effects if you naively keep appending content to a div that's been rotated in 3-D.  As it reaches a certain height, webkit apparently starts downsampling (I guess to save memory in the rendering pipeline?  A limitation of hardware?  Who knows) and you get this pixelated effect:

This isn't a downsampled down screen shot.  It actually looks like this if you let the rotated div get too large.
As you can see, it becomes unreadable after a while.  It also starts eating up an increasing amount of CPU: it got to 100% (out of 400%) on my quad core iMac after a few minutes.  Not good.

So instead of putting all of the tweets into the same rotated div, I rotate each tweet element individually and that seems to fix the problem.  Also, I start deleting tweets at the top (where you can no longer see them anyways) after a while, just to keep the dom size down.  They are now readable and the CPU is happy.

Other stuff:

Instead of using CSS3 animations, I opted for javascript because I was loading the tweets with JS already and I'd have to manipulate the animations with JS anyways.

Also, the background is generated dynamically using the Canvas element.  I wanted a more realistic looking star field so I did some research on the distribution of star sizes and colors to see if I could perhaps simulate it with a Pareto distribution.  Turns out the distribution of star colors and brightness is some other weird distribution so I just made it more or less random, with smaller stars slightly more frequent than larger stars.

Saturday, August 21, 2010

λ as a variable name in JavaScript

JavaScript is a functional language, or at least has very functional roots we could all probably agree.

Functional languages are based on lambda calculus and the CS literature often uses examples where a lambda expression is called simply, "lambda."

In JavaScript, one common use of lambdas is when you register a callback function with some asynchronous operation like an ajax request. Another is for operations on collections of objects like visiting every item in an array.

I wondered if one could use the lambda character, λ, as a variable name in javascript. It would be more compact (one character instead of five six) and (more importantly) would look cool. Turns out you can do this!

See it run here.



Notice that the character encoding is set to UTF-16. If you don't do this you'll get errors. Since the encoding is set that way, I don't even use html entities like λ to write λ :)

Note: I haven't tried this in any version of IE but it appears to work in Chrome, Safari and Firefox.

Now, how do you actually type λ into a text editor? That took some digging. Here's what I did on OSX:

  1. Open System Preferences -> Language and Text -> Input Sources
  2. Scroll down to "Unicode Hex Input" and make sure that checkbox is checked.
  3. Open the US Flag (or nationality of your locale :) icon in the upper right of the OSX menu bar and switch to "Unicode Hex Input" (the "U+" icon)
  4. Now whenever I want a λ I can just hold down the Option key while I type 03bb and blammo I get a λ. Which by the way is still five key presses but looks way cooler than "lambda" and takes up less space on the screen.

I have no idea what you have to do on linux or windows.

You're right, this isn't very practical. And setting your encoding from UTF-8 to UTF-16 like, doubles the bytes you have to transmit. But you ARE gzipping all of your http responses aren't you? AREN'T YOU?

Thursday, February 25, 2010

PubSubHubhub for NodeJS: Callbacks All the Way Down

NodeJS is a callback-based Javascript server API.

PubSubHubbub is a callback-based web protocol.

I put them together and the result is a PubSubHubbub client for NodeJS:

node-pshb on github


This project only includes a PubSubHubbub client interface, but to me that's the interesting part. You can specify an atom feed url, and functions to call when events happen on that feed due to PubSubHubbub.

The client library takes care of identifying the hub for that feed, requesting a subscription, and listening for subscription confirmation requests and feed updates from the hub.

It provides callback hooks for "subscribed", "update" and "error" events.

A simple client app looks like this:
var callbackPort = 4443;
var subscriber = new pshb.Subscriber(callbackPort);

// Start listening for subscription confirmation callbacks.
subscriber.startCallbackServer(); 

var topicUri = url.parse("http://localhost/foo"); // Dummy feed, always has updates

var feedEvents = subscriber.listen(topicUri);

feedEvents.addListener('subscribed', 
  function(atomFeed) {
    sys.puts('subscribed: ' + atomFeed.id);
  });

feedEvents.addListener('error', 
  function(error) {
    sys.puts('ERROR: ' + error);
  });

feedEvents.addListener('update',
  function(atomFeed) {
    sys.puts('got a PubSubHubub update: ' + atomFeed.id);
  });

I tested this out with the Demo Hub running on a local AppEngine launcher.

The demo app creates a a second server to host a dummy feed on port 80, so http://localhost/foo always returns a feed with the current time as it's "last updated." This is so the test hub always thinks there's an update ready for you.

So start the appengine with the hub running locally (in this demo it's assumed to be on port 8086), then run the test.js app, then go to your hub with your browser and manually update http://localhost/foo.

I noticed that I had to manually run some tasks in the hub's work queue so if you don't see any updates try checking the Task Queues in the app console for the hub. Run any pending "feed-pulls" and "event-delivery" tasks. I imagine there's a way to make them do that automatically but I haven't dug around enough in there to find it.

So there you go, NodeJS and PubSubHubbub: it's callbacks all the way down.

Friday, February 19, 2010

Webfinger Client for Node.JS

In a previous post, I demonstrated how you could use webfinger with nothing more than curl.   This post is about how you can use webfinger from nodejs with a non-blocking webfinger client.

Code for node-webfinger is here on github.

The project contains a simple webfinger-buzz.js command line app that demonstrates the webfinger client. It uses webfinger to find a google buzz feed based on a gmail address, then fetches the updates as an Atom feed, and then prints out the latest entry from that feed.

This could be generalized to support any other webfinger-enabled site like yahoo (though it looks like they're using an older version of XRD which my code can't parse :/).

The webfinger-buzz.js client looks something like this:
var sys = require('sys'),
  http = require("http"),
  url = require("url"),
  atom = require("./lib/atom"),
  webfinger = require('./lib/webfinger-client');

if (process.argv.length < 3) {
  sys.puts("usage: " + process.argv[0] + " " + process.argv[1] + " <user uri>");
  process.exit();
}
 
var userUri =   process.argv[2];
 
sys.puts("fingering " + userUri);

var wf = new webfinger.WebFingerClient();
var fingerPromise = wf.finger(userUri);
fingerPromise.addCallback(function(xrdObj) {
  var statusLinks = xrdObj.getLinksByRel("http://schemas.google.com/g/2010#updates-from");
  var statusUrl = url.parse(statusLinks[0].getAttrValues('href')[0]);
  var httpClient = http.createClient(80, statusUrl.hostname);
  var path = statusUrl.pathname;
  if (statusUrl.search) {
    path += statusUrl.search;
  }
 
  var request = httpClient.request("GET", path, {"host": statusUrl.hostname});
 
  request.addListener('response', function (response) {
    response.setBodyEncoding("utf8");
    var body = "";
    response.addListener("data", function (chunk) {
      body += chunk;
    });
    response.addListener("end", function() {
      var atomParser = new atom.AtomParser(false);
      var atomPromise = atomParser.parse(body);
      atomPromise.addCallback(function(atomFeed) {
        sys.puts("Feed: " + atomFeed.title);
        sys.puts(atomFeed.entries.length + " entries");
        sys.puts("Updated: " + atomFeed.entries[0].updated);
        sys.puts(atomFeed.entries[0].title + ": " + atomFeed.entries[0].summary);
      });
    });
  });
  request.close();
});

hehe fingerPromise. Is that a generalization of pinkySwear?

In the process of writing this webfinger client I used a couple of libraries I found on teh internets: sax-js by Isaac Z. Schlueter - a SAX parser for nodejs, and this URI Template library by James Snell. Both worked well and I recommend them.

The remaining non-webfinger-specific pieces I needed were an XRD parser and an Atom parser, both for javascript and SAX (as opposed to DOM). I couldn't find much in the way of those, so I rolled my own. They are included in the node-webfinger project in the lib/ directory. They're pretty crude parsers but they worked for this example. I'll probably use them in other projects in the future and make improvements as necessary. Unless something better comes along. That seems inevitable.

Monday, February 15, 2010

Bring the (Perlin) Noise

If you just want the source code: The JavaScript Perlin noise generator code is here.

I mentioned in a previous post that I was working on a Perlin noise generator for Art Evolver.

Perlin noise is a function of (x, y) that produces a random-ish pattern. It's not completely random because it has smooth hills and valleys, but the distribution of those hills and valleys is random.

A side note about this algorithm: Usually if you're a computer scientist and you come up with a clever algorithm to solve a particular problem, you get an award from a university, or a CS-centric professional organization like the ACM or IEEE. Perlin got an Academy Award for this noise function. As in, the Oscar kind of Academy Award.  For an algorithm, something not usually consider artsy.  I found that interesting.

Anyways, rather than dive into Perlin's impenetrable description of how the algorithm works, I set out to find an existing JavaScript implementation. That let me to this message board, and specifically this example.

Unfortunately that implementation has some pretty serious directional artifacts:

Note the horizontal and vertical stripes.  There's almost an upside down cross in the lower right. SAAAATAAAAN!

Rather than try to fix that source code (which the author apparently closureized (making it very difficult to understand)) I kept searching.

From the main Wikipedia entry on Perlin noise, I ran across a variant called Simplex noise.  This is an improvement on the original algorithm, also written by Ken Perlin, in 2001.  That Wikipedia page linked to a paper by Stefan Gustavson(pdf) that explains both classical and Simplex Perlin noise in a much easier to grok way than anything I've read by Perlin himself.  I highly recommend Gustavson's paper if you found Perlin difficult.

I took the Java source code in Gustavson's paper and ported it to JavaScript, and the results are here on github.

I ran some performance comparisons between the classical and Simplex algorithms, and for 2-D I only saw a ~10% improvement with Simplex.  Granted, the latter is supposed to be faster in higher dimensions (classical is O(N^2) vs. simplex O(N) where N is the number of dimensions) so it doesn't matter much for my purposes.

Classical Perlin noise

Simplex Perlin noise

Subjectively I think I prefer the Simplex noise to classical, so I'll probably go with that for Art Evolver.

Again, the source code is here.