From plaintext to Pandoc, searching for a perfect markup

Sometimes there are issues that seem to come up again and again over a period of months and years. For myself, one of the biggest annoyances of recent years has been finding a form of lightweight markup that I can compose all of my documents in. This kind of thing is a very personal choice, with that in mind I will share the process by which I made the choice I am now happy with and the simple and lazy workflow it gives me.

Why is this so important to me?

I spend most of my time at a computer in front of a text editor, I know the editor inside out and so it is a very efficient environment. Even if I am not on my own system I can always find a text editor and just start writing whatever environment I am on, Unix, Windows, even my old Psion. So, a plaintext solution for all of my documents is a great timesaver.

Consider a few of the documents I deal with on a day to day basis which each need a different application to work with:

• Webpages need to be HTML
• Emails need to be plaintext
• Documentation and application forms need to be richtext
• Manuals need to be .pdf
• Python documentation needs to be RestructuredText
• Wiki/Trac entries need to be in their wiki format

Frequently I need several different formats at once; HTML, pdf and richtext for user documentation is a typical example so writing one master document and being able to generate several outputs is a great timesaver and makes document revisions easier to manage.

Why is it so difficult to find a solution?

I have two very different requirements, firstly a markup syntax that I feel comfortable with, secondly a set of unified tools that enable easy conversion to other formats.

The path to the final solution

At first I started writing everything in HTML since I used it so often, but as expected this was too verbose and difficult to read and needed lots of tools to convert too and from different formats.

Next I discovered Markdown by John Gruber, it has a very simple plaintext format that is easily readable and converts to very clean HTML, however it was lacking (intentionally) a few little things I used frequently, most notably definition lists and simple tables. Having said that, it was a pleasure to work with and made a lot of sense so I used it a lot.

The aforementioned shortcomings meant that over time I researched other Markdown varieties and came to Multimarkdown by Fletcher Penney. This solved a lot of my problems and gave me a broader variety of output formats, but meant that if I used other markdown tools it didn’t display quite as I would have liked.

Around this time I had begun using Python as my coding language of choice, so it seemed that reStructuredText was the obvious way to go; incredibly full-featured with lots of supporting tools and the documentation language for Python. I tried and really tried with it but for some reason the syntax just seemed a little ugly and I didn’t seem to be saving much time between the syntax and the various different tools I had to use to output different formats from it.

So eventually I went back to using a flavor of markdown again and made a great discovery...

And the winner is… Pandoc

Pandoc is a well thought out and well executed piece of software that I now find indispensable. Written in Haskell it is extremely fast, but more than that, it has a huge amount of features that fulfill 99% of my requirements, such as:

• Markdown compliant
• generates document fragments or whole documents
• reads HTML and simple reStructuredText
• writes HTML, reStructuredText, RTF, pdf

My lazy document workflow

Firstly I created a folder containing a CSS stylesheet for HTML output, some custom headers and footers for documents and a logo. Next I created a simple shellscript which when passed a filename will create a folder (based on the name) and convert the file into every possible output format based on the input file type.

An input file foo.txt will generate the following files in a couple of seconds:

foo/
 foo.html
 foo.s5.html
 foo.rtf
 foo.xml
 foo.la.tex
 foo.con.tex
 foo.pdf

This is of course more than I usually need, but the process is so fast and easy that I just grab the ones I need and then delete the rest of the folder.

Finally

Finally I have a toolset that does almost everything I need and a whole lot more all in one integrated bundle and lets me use my favorite markup syntax.

I recommend that you take a look at Pandoc if you have similar needs.

Amusing Fortran book (no really)

I recently found this fine example of a politically correct computer programming guide: COMPUTER PROGRAMMING FORTRAN 7th edition 1981

Whilst it has several noteworthy entries this is probably the one which made me chuckle most, taken from page 68, Communication and Arithmetic chapter:

"Once the program has been written, the actual punching of the cards is a straightforward clerical operation which can be performed by keypunch operators--usually girls--who know little or nothing about programming."

Sony Ericsson K770 plugin for iSync

Having held off a contract phone until I could find one which did everything I really wanted I finally took the plunge when I saw the Sony Ericsson K770i.
The only problem I have with the phone was that it was not supported by Apple’s iSync utility.

Since I knew from reviews that the internals are basically the same as the K800 and K810 I thought that it should be possible to make a plugin based on the K800 plugin that comes with iSync.

A quick search of the contents of the iSync package soon found what I needed:

iSync.App
|
\Contents
|
 \Plugins
   |
    \ApplePhoneConduit.syncdevice
      |
       \Contents
         |
          \Plugins
            |
             \SonyEricsson-K800.phoneplugin

I copied this directory and renamed it SonyEricsson-K7700.phoneplugin, then did a search for K800 in all of the .plist files within the directory, replacing them with K770.

After a quick trip to Photoshop editing com.sonyericsson.K800.tiff to represent the K770 and saving the file as com.sonyericsson.K770.tiff I was ready to try it out.

Creating a new folder named PhonePlugins in my local Library directory and copying the SonyEricsson-K800.phoneplugin directory to it did the trick and my new phone synced perfectly.

You can download a zipped copy of the SonyEricsson-K800.phoneplugin from the file link above, use it at your own risk.

Lifehacking my journey to work. Part 1, the problem

About four months ago I started a new job in Brighton. This meant for the first time in a good while I had to commute by train. I have always enjoyed the train and in the past have found it a great place to get a little work on my personal Python projects done before and after the office PHP grind. The journey itself takes between 15 and 20 minutes depending upon on whichtrain I take, just enough time to write a little section of code, or indeed a blog post. Initially the trains were quite pleasant and uncrowded but then for some reason it all changed for the worse.

Here is a breakdown of the typical journey before I decided to start hacking it:

Phase 1. The mount

Jostle to get to the front of the platform as the doors open in the hope of getting a window seat.

Phase 2. Staking a claim

Find a seat, remove laptop from bag and sleeve, wait whilst laptop wakes up and wait for it to ask if I want to join the locked network on the train so I can say no and close the dialog.Suddenly squeeze up closer to window place bag on lap, place laptop on top of bag, tuck elbows into ribcage and fold legs back underneath seat as a couple resembling Wildebeests, bred from the genetic stock of Hagrid decide that the seats around me are just perfect for them.

Phase 3. The business

Attempt to type something with hands held like a begging dog due to the straitjacketed elbows and non-ergonomic position of laptop on top of lap and now on top of bag too. Turn up volume on iPod to drown out the various wheezing, grunting and caveman-like noises that pass as conversation between the aforementioned genetic throwbacks in an attempt to think. Realise I am going to get nothing done and prepare to disengage.

Phase 4. The dismount

Long before the train begins to slow, engage in an elbow battle whilst forcing the laptop back into it's sleeve and bag. Politely say excuse me and stand up as my hillbilly travel companions tuck their legs slightly to the side leaving me a one inch gap between their knees the size of volleyballs and the edge of the table as I attempt a Houdini act to free myself from the chair. Dash off train just in time breathing a sigh of mingled relief and frustration.

This had been happening so frequently lately that I was probably averaging 10 words each direction. Time to try a little lifehacking, so Part Two will reveal the solution.