I’ve recently finished reading Getting Things Done. And I’m trying it out as my method for organising all of my work.
Regardless of how it goes there are several key ideas which I think transfer directly onto the world of software development. These are (in no particular order):
If something takes less than 2 minutes do it,
Stuff that needs doing is defined as something that isn’t how or where is should be.
A project (something that needs planning) is something that requires more than one action to complete
Always think in terms of “What’s the next action?”
This last one I consider crucial especially when it comes to learning new things. I’ve been guilty of having goals such as “learn functional programming”.
This type of thing is far too broad, you need to think it through until you end up on the next thing that needs doing to reach your goal.
All of us are guilty of having a number of projects “on the go” as well. I’ve found forcing myself to think about the next action has really helped me make progress with them.
In some cases I’ve made progress on projects that haven’t made progress in >2 years.
Like everything, GTD is no silver bullet, and isn’t for everyone, but I think the book is worth reading, if you don’t find it useful, at least you know.
I don’t normally find programming koans enlightening or amusing, there is one execption – which pretty much sums up my approach to vim:
12345678910111213
Master Wq and the Markdown acolyte
A Markdown acolyte came to Master Wq to demonstrate his Vim plugin.
"See, master," he said, "I have nearly finished the Vim macros that translate Markdown into HTML.
My functions interweave, my parser is a paragon of efficiency, and the results nearly flawless.
I daresay I have mastered Vimscript, and my work will validate Vim as a modern editor for the enlightened developer!
Have I done rightly?"
Master Wq read the acolyte's code for several minutes without saying anything. Then he opened a Markdown document, and typed:
:%!markdown
HTML filled the buffer instantly. The acolyte began to cry.
But I was starting to get the feeling that most vim users had loads of plugins installed – so I decided I wanted to find out if this was true.
20 minutes later I had hacked my way through a basic survey – which was full of holes and typos. Despite this I had over 1000 responses in 24 hours – which gives us our first ‘statistic’:
1
Vim users love talking about vim
So, here are the results. 1300 of you responded to the survey. Percentages have been rounded down to the nearest integer. Because I said so. Questions where the answers were freeform I’ve done my best to group. But it might not be perfect.
Each question was optional so each question had a number of empty responses, I’ve left these out. This is why most percentages do not add up to 100%
I welcome others to take a look at the raw results: Link to Raw Data
Do you spend most of your time in vim or gvim?
I forgot about macvim, and had never heard of vim-qt – but thankfully most of you seemed to work out that my intention was – gui or terminal.
vim 946 72%
gvim 347 26%
How often do you use vim?
daily 1227 94%
weekly 54 4%
monthly 11 0.8%
The obligatory operating system question – Which do you use most?
The world is full of programmers using macs in coffee shops right?
linux 807 62%
os x 347 26%
windows 131 10%
freebsd 19 1%
One person said they used “your face” which is an odd choice as my face has very poor memory management.
What key do you have mapped as your leader key?
I’ve always just stuck with the default, althought I was aware that alot of people use ‘,’.
Using space was new to me, but according to this it’s one of the most popular keys
comma 470 36%
backslash 309 23.7
none/ don’t use, never heard of leader 220 17%
space 169 13%
semicolon 13 1%
forward slash 12 0.9%
esc 11 0.8%
colon 8
underbar 5
ctrl 4
alt/ caps lock 2
How often do you make changes to your vimrc file?
montly 603 46%
weekly 459 35%
yearly 138 10%
daily 60 4%
no customisations 21 1%
When did you first use vim?
2010’s 613 47%
2000’s 505 38%
1990’s 162 12%
How many plugins do you have installed at the moment?
This is the question I really wanted answered.
Thankfully it seems like most of us would rather have less plugins
Your favourite plugin?
ctrlp 171 13%
syntastic 59 4%
nerdtree 59 4%
youcompleteme 50 3%
fugitive 41 3%
airline 39 3%
surround 30 2%
vundle 30 2%
pathogen 24 1%
Do you remove plugins you no longer user?
yes 1009 77%
no 229 17%
What is your average typing speed?
61 – 80 391 30%
41 – 60 360 27%
> 80 344 26%
20 – 40 wpm 121 9%
< 20 wpm 8 1%
Roughly how many different computers will you use in a particular week?
6 – 10 143 11%
2 – 5 877 67%
> 10 122 9%
1 132 10%
Have you written a plugin?
no 1006 77%
yes – publicly available 121 9%
yes – kept it private 148 11%
Give me a single piece of advice
Some of it is great, some is crazy, you’re welcome to take a look at the raw results:
I spent a while trying to work out how to construct this question, what I was trying to work out is if the editor war really existed, I don’t use emacs, but I did, and there are some things I think it does better than vim, and some things it doesn’t.
I spent ~2 years using emacs before moving to vim, I wanted to get a broad idea of how emacs is view by the vim community. The results were a suprise
That’s it
That’s all there is nothing more. I’m not great with statistics so I encourage others to take a look at the raw data and come up with a much better ‘analysis’.
If anyone feels that the above is in some way worthy of a small tip here’s my btc address:
Update: So I’ve learned that there’s actually some work to be done here around sandboxing before it’s actually ready for public use. Until then I’ve removed the server, but the code is still available if you want to play around with it in a trusted setting.
One of the great things about go is the whole environment you get with it, gofmt, godoc, go get. It’s all good stuff.
There’s also a playground for you to try things out on.
The docs also have runnable examples in them based off a similar idea:
http://golang.org/pkg/encoding/json/#example_Unmarshal
It’s a great idea, it means you can actually try the code out while you’re reading the docs. But it only works on code from the stdlib. But that makes sense doesn’t it? You don’t want to allow anyone to import some arbitrary package.
But it occured to me that when someone develops a package, the author might want to include runnable examples for their own code.
The source for something similar to the playgrond is incuded in the go release (misc/goplay) I’ve modified it very slightly to allow /compile to be called from any location.
I’ve called it goplaas (It’s not really as a service, but that’s the standard suffix these days isn’ it? And it got your attention!)
The idea is you install it on your own server, and it will have access to anything installed on your GOPATH. This means you can control what packages can be imported, and also the version.
All you need to do is put some javascript and a textarea into a webpage, point it to your server and you’re done.
Here’s an example using the loggo package from launchpad.net. Try it
1
I've removed it - sorry
I’ll try to keep my server running with goplaas for the next few weeks. If it suddenly stops working, it’s likely that I’ve pulled the server down. You can experiment yourself by running
12
go install github.com/mattyw/goplaas
./goplaas -http :9999 #Listen on 9999
goplaas includes a -compile flag for sending compilation instructions to another server, full instructions are in github.
A good next step would be to package this all up into a docker container to allow anyone to install it with relative ease
Today I finally got round to trying out something i’d been meaning to look into for nearly a year, and seeing as clojure and editors seems to be a topic again I thought it would help add some diversity.
Acme is an editor unlike any other. It was designed by Rob Pike for Plan9 (although it’s available for unix-like systems using Plan9 from User Space).
It’s designed to be an editor and shell. Unlike emacs and vim which encourage you to keep your hands on the keyboard, acme requires use of a 3-button mouse, and it supports something called mouse chording. A rough guide would be:
Button 1: Select Text
Button 2: Execute Command
Button 3: Search/ acquire text (or load file)
Button 1-2 Cutting
Button 1-3 Pasting
Button 1-2,3 Cut+Paste (Copy)
My introduction to acme was via Russ Cox’ video, which is a great starting point:
You can probably summarise the whole thing by saying that in acme – text is data. Once you type some text, it can be saved into a file, executed as a command or passed as an argument to a command.
Obviously this resonated with the clojure part of my brain. It occured to me that being able to select some clojure code in acme and evaluate it would make acme a great editor for clojure.
Thanks to nrepl the job is really quite easy. I wrote a very very simple nrepl client in go. You can find it on my github
All it does is read clojure code from stdin and send it to nrepl. This is really all there is to do. In acme you just need to type your command in one of the scratch areas:
Take a look at the below screenshot. You can see:
In yellow: selected clojure code with mouse button 1
In red: selected command with mouse button two
Release the buttons and the window in the bottom right contains the evalutated code
You’ll need a command line tool that can take clojure code on stdin and send it to nrepl. I used my own gonrepl tool- To use that you need to install go then run:
tl;dr: Juju is aimed at deploying and scaling hundreds of services. But it’s also a handy alternative to buying a VPS.
I don’t have a VPS, I have in the past but I’ve never used them enough for me to be able to justify the cost long term.
Juju provides a perfect solution to this use case.
I have a very simple charm I call devenv, it doesn’t do much more than install stuff I use for doing development, like installing tmux and grabbing my configuration files. I’ve blogged about it previously here.
This week I was having trouble getting a client library to connect to a simple server program I’d written, I was lost with debugging but luckily the library author volunteered to help me out.
I already had a binary of the server that could be used to generate the error in the client library, but I don’t have a VPS, so what’s the quickest way for me to make the server public so the library author can play around with it?
All I needed to do was to add the following line to the start hook of my devenv charm:
1
./app &
And copy the app binary to my charm directory.
It’s sort of an anti pattern – but because juju copies the whole charm dir when it deploys, it will grab any binaries in the charm dir as well, it’s not a great idea to make use of this for serious deployments, but for me it was great
From deciding I wanted to deploy the binary – to having it deployed was under 5 minutes. 20 minutes later the library was fixed and the server was taken down
1
juju destroy-environment
That was it. All in all it cost around $0.02 (that’s just over 1p). Much better than having a VPS that would largely sit idle.
Of course the other advantage with using juju is that I’m not tied to a particular cloud provider, so I can choose which one I want depending on the pricing at that moment. And since juju now supports local mode you can do your charm development offline to save even more money!
Great, how does this help me?
If you’re thinking of getting a VPS – but don’t feel you can justify the cost because of the amount of use you’d get out of it, then you really should try out juju. Once you’ve got it installed there’s a handy charm that just deploys ubuntu server, it’s a great starting point for writing a charm to setup your own development environment.
Don’t write macros is a pretty good rule, but when you’re learning a language it’s quite a fun little exercise.
One such exercise I like to try is this:
Write a function so that I can call system commands without having to wrap them in quotes.
So rather than doing this:
cmd([“grep”, “-rin”, “foo”, “./”])
I can do this:
cmd([grep, -rin, foo, ./])
Clojure
In lisps like clojure, the solution is very elegant, thanks to the minimal syntax, you end up with something like:
Not too much magic (as far as macros are concerned) here. Make a unique symbol with (gensysm) to keep our macro hygienic.
We syntax quote our whole list using ` to not evaluate anything unless we say so. That means we have to do some magic with our args parameter: ‘~args. ~ here means
evaluate args (so we get the symbols we passed in) but then we quote with ’ so that we don’t evaluate these symbols. That’s the magic that lets us not need quotes around our arguments.
The rest of the code just puts them in the right place by mapping our args list against the str function and applying it to sh.
This is how you use it:
12
(prn (mcallls-l))(prn (mcallgrep-rinnot "./")); We still need quotes around ./ because it's not valid syntax
Elixir is a lovely little language I’ve been playing with recently, so naturally I wanted to try out the macros.
I needed to ask a few questions on the irc channel (thanks ericmj and true_droid!), but I got it cracked:
Again, not much magic, map_join maps our list using the Macro.to_string function then joins at the end, this is done outside of the quote since we will work it out at compile time.
We then pass this to System.cmd in a quote block – which means don’t evaluate this yet, that will happen when we call mcall at runtime.
12
IO.inspectMacroTest.mcall([ls,-l])IO.inspectMacroTest.mcall([grep,-rin,def,"./"])# We still need quotes around ./ because it's not valid syntax
We’re going to use a raspberry pi and juju to setup a wordpress blog on aws.
The raspberry pi is an amazing little gadget. Low power enough that I don’t feel guilty about leaving it turned on overnight, but flexible enough that I can shove leds into it to grab my attention if needed.
Juju is ubuntu’s answer to setting up services in the cloud. Once installed you simply setup your credentials with an existing cloud provider like aws or hp cloud and then deploy things into it. Setting up wordpress for example is as simple as
12345
juju bootstrap # Setup the cloud environmentjuju deploy wordpress # Start an instance and install wordpress on itjuju deploy mysql # Start an instance and install mysql on itjuju add-relation wordpress mysql # Connect wordpress and mysql togetherjuju expose wordpress #Allow wordpress to be accessed from a public ip
The Plan
Install go onto the raspberry pi
Install juju
Deploy a wordpress blog
We’re going to be installing the latest version of juju which is written in go.
Installing Go
Dave Cheney has written an excellent article on Installing go on the raspberry pi. On my model B (1.0 revision) I had to update raspi-config so that I could select the 240M/16M memory split, but you might have to see how you get on here. I choose to skip the tests and just ran ./make.bash to build go rather than all.bash.
I was unable to build juju using go 1.0.3. I ended up building go 1.1.1 from source, which seemed to work fine. From memory I believe you need to setup a folder to be your GOPATH before you can install the latest version of go.
Installing Juju
Getting the latest tip of juju-core is easy using the go get command, it grabs all the dependencies for you, it does mean we’re grabbing tip, but there are ways to fix that later if we need to. juju-core is hosted on launchpad.net which uses bazaar for source control. This means you need to install it before you can run go get. I was able to install it on raspian using
12
sudo apt-get update
sudo apt-get bzr
Without doing the update I was unable to grab all of bzr’s dependencies, so I recommend running the update first.
Once bzr is installed you can install juju using go get:
12
sudo apt-get install libcurl4-gnutls-dev
go get launchpad.net/juju-core/...
When it’s finished you should find a juju binary installed. Mine ended up in my GOPATH, I think because I had something setup wrong somewhere. But if you’ve got this far I’m sure you’ll be able to find it.
Setting up Juju
There’s much better instructions for setting up juju here The basic steps come down to
Setting up a public key
Generating an environments.yaml file
Filling the file in with the data for your cloud provider
I configured juju for aws. My ~/.juju/environments.yaml file looked like this:
Once you’ve done this, try running juju bootstrap. If it finishes without error wait a minute or so and run juju status, If all goes well you should see something like this
Now we can start the business of getting a blog up and running:
1234
juju deploy wordpress
juju deploy mysql
juju add-relation wordpress mysql
juju expose wordpress
You might have to wait a few minutes for everything to start up, keep taking a look at the output of juju status until you see all the agent-states running and expose true on wordpress.
When it’s done you should see something like this in your juju status output:
Erlang was the first functional language I ever used. When I first started reading Programming Erlang by Joe Armstrong I was hooked, I was fascinated in a language where “=” didn’t indicate an assignment, and the idea of pattern matching immediately struck a chord with me, I loved the one module per ‘process’ thing. Erlang was set to become my favourite language, there was only one problem.
I didn’t like the syntax
It’s a completely personal view, I know people who love the syntax, I just didn’t. To me it just didn’t have what Christopher Alexander in “The Timeless way of building” calls “quality without a name”. I suppose I just didn’t find it fun to type. So after re writing a couple of internal libraries in erlang to see what they looked like (they were much shorter and quite easy to read) I stopped using it, and started playing around with other languages like haskell and clojure.
Clojure was my first lisp dialect and I enjoy it alot, the syntax, the whole homoiconicity thing, macro’s aren’t something you should use all the time – but it’s comforting to know they’re there to fall back on if you need them.
Parallel to this functional language experimentation was another path – languages I used at work, which has seen me recently move from writing mostly in Python to Go. Go’s channels and go routines are great, but when I started writing go code seriously (about 8 months ago) there was something that immediately stuck me.
1234
thing,err:=someFunction()iferr!=nil{//handle the error}
Now, I’m not going to start complaining about go’s error handling, because I like it. But sometimes I would like to be able to make use of pattern matching and do something like this:
1
{thing,nil}=someFunction()
And have my goroutine just dies if it fails. I know that’s not go’s philosophy with errors, and that’s fine. I’m just saying.
Work and Fun Collide
I’ve spent the past 6 months saying to people that I would like a lisp dialect to be built onto go. What I really meant was I wanted a language that would let me do go style concurrency and play around with a repl, macros and all that functional programming fun stuff.
It looks to me that elixir provides exactly that.
Just look at these two programs. Both call a 10 second sleeping function on a seperate ‘thread’ and wait for the response. The first in go, the second in elixir.
They’re not too different, elixir has spawn where go has ‘go’ and elxir has receive where go has select. There’s no denying that they’re completely different languages, in some rather crude tests I’ve done go does seem to be largely faster, but elixir let’s me do meta-programming whilst forgetting about types.
I expect to be blogging alot more about elixir in the future
My first post on my new blog, so let’s keep it simple.
In sql databases you can specify unique constrains on certain columns, like this example from postgres:
12
createuniqueindexidx_UniqRefonFooBar(UniqRefASC);
The question is: Is it possible to do this in a schema-less database like mongo?
Lets’ start writing some code that we want to fail. For this I’m going to use go and Gustavo’s mgo library
+packagemain++import(+"labix.org/v2/mgo"+"log"+)++typeFooBarstruct{+Namestring+Ageint+UniqIdint+}++funcmain(){+session,err:=mgo.Dial("127.0.0.1")+iferr!=nil{+log.Fatal(err)+}++coll:=session.DB("test").C("uniq_test")+coll.DropCollection()++a:=FooBar{"Foo",1,1}+b:=FooBar{"Bar",2,1}++err=coll.Insert(a)+iferr!=nil{+log.Fatal(err)+}+err=coll.Insert(b)+iferr==nil{+log.Fatal("We wrote something but wanted to fail!")+}++}
Nothing too fancy here:
Create a connection to our collection
If the collection exists drop it (useful for iterating quickly on this test)
Create two structures with a UniqId field duplicated
Insert both, if the second insert doesn’t fail log an error
If we run it as is, we see our failure
12
2013/06/12 13:14:51 We wrote something but wanted to fail!
exit status 1
Now we’re in a position to start looking into a solution, and it turns out we don’t
have to look far. Just like postgres mongo has indexes, we can create one using ensureIndex.
Here’s how it looks in the mgo library:
123456789
//Let's try an indexindex:=mgo.Index{Key:[]string{"uniqid"},Unique:true,}err=coll.EnsureIndex(index)iferr!=nil{log.Fatal(err)}
This creates an index on our uniqid field and forces it to be unique. If we call this code
before we try inserting here’s what we get
There we go, it also supports compound indexes which lets you specify multiple fields which must
be unique, but I’m not going to cover that here. The mongo documention
is a great source for more information