Markdown vs HTML – hoedown or showdown?

On a mailing list recently, a friend asked: should my workflow use an HTML editor or markdown? There is, of course, no easy answer. It depends what trade-offs you want to make. At Fire and Lion we use markdown for book production, and I know very smart people who think that’s crazy. They’d pick an HTML editor any day.

What worries me about working in an HTML editor is that it must make assumptions about what the user intends. That is, HTML editors abstract what you see from what you’re storing. What you see is what you hope you get. The real source format is what the user types, but HTML editors effectively discard it, jumping straight to rendered output and hiding its structures from view. Non-technical users often have no idea what they’re actually storing, and very little control over it.

And those assumptions are the root of all editor evil: they inevitably lead to a hidden mess of legacy markup. As users edit, and especially as they paste from other sources, their editing software has to make guesses about what formatting and what HTML elements the user wants to keep and what it can discard. You only have to glance at the source of a heavily edited WordPress page to find a teeming mass of unnecessary spans, redundant attributes and inline CSS.

Over time, the HTML gets messier, and that mess is swept under the rug of the WYSIWYG view. And as it gets messier, it becomes less portable, and conversion tools become less useful. Suddenly I can’t just reuse my HTML somewhere else without unpredictable results. For any given reuse I lose hours to cleaning up my HTML, effectively creating a whole new fork of my project, and losing the ‘single-source master’ feature of my workflow. I’m sure every new HTML editor aims to solve that problem, but I haven’t found one that’s solved it yet.

So that’s why I remain a champion of markdown-based workflows: there is no abstraction in the editor, because I’m only ever working in plain text. The simplicity of plain text means my content stays clean as I go, because there is no rug to sweep a mess under.

The bare bones of markdown have other spin-offs, too:

  • Markdown is more portable. By ‘portable’ I mean between people and between machines. For non-technical people, markdown is more open than HTML: it’s instantly readable and copy-pastable. A format that’s useful without a developer in the room is exponentially cheaper to work with, especially when you have to move it between machines.
  • The contraints of markdown force us to keep document structures simpler, sticking to fewer, standardised elements.
  • And diffs of plain-text markdown (in Git especially) are easy to read. We can use them in editorial workflows as is.

However! Those who prefer HTML editors are right that markdown has serious constraints. Or, rather, that HTML5 (like many markup languages) provides more features than markdown can provide natively. For instance, markdown can’t produce tables with merged cells, create plain divs and spans, or manage nested snippets for things like figures. They’d argue rightly that the markdown editing experience can be clunky, especially to those accustomed to Word-like UIs. And that non-technical users mostly don’t share my concerns about messy underlying markup: they just want an editor that looks great and is easy to use.

Like tabs versus spaces, I don’t expect this debate will ever be resolved. What matters is that we each pick our own trade-offs, and respect the trade-offs others make.

 

I love you, InDesign, but it’s time to let you go

I love you, InDesign, but it’s time to let you go. We just can’t be together in a multi-format world.

InDesign is expensive, so I can’t have my whole team working in it. It’s so powerful that it takes years of experience to use it without making a mess. And it’s fundamentally incapable of producing both print PDF and ready-to-use HTML from a single master file, despite some amazing hacks.

Adobe has tried valiantly to turn this page-based, hot-lead-replacement into a multi-format tool, but its roots in print are just too deep. Making books in InDesign and converting them to high-quality ebooks and websites is a rocky journey that leaves even the smartest typesetters bloodied and broke.

At Fire and Lion we make a lot of books for screen and paper (mostly for publishing companies and non-profits). To our clients, what matters most is that the books are well-crafted in every format, and that working with us is problem-free. Behind the scenes, we have to do something special to make that possible.

So the first thing we do is avoid using InDesign for setting everything but the most heavily illustrated books. We don’t do page-by-page layout and convert to HTML later. In fact, we do exactly the opposite: we make each book as a little website, and then output to PDF.

To put it another way: Fire and Lion makes responsive websites that respond not only to screen sizes but to the pages of a book. And we do it so well that, looking at the finished product, you can’t tell the difference between our books and those you’d get from a typesetter working in InDesign.

We’ve been lucky to work with clients who’ve let us make their books with this cutting-edge toolset. You have to be brave to accept a GitHub repository as your open files, rather than an InDesign package; but it’s brave people like that who move our industry forward.

Nothing we’re doing is a secret: our workflow is open. So when we’re not making books, we’ll be talking about how we make them. If you’re working with similar tools, or curious about ours, let us know.

Eleven ways to join the resistance from your desk

If you’re feeling bewildered, you’re not alone. So, once you’ve been thoroughly nauseated on Twitter and are keen to actually do something about this shit-show, here are a few non-profit projects that I wholeheartedly believe in, and whose people I’d trust with my life and my money. Volunteer, give money, say nice things – it all helps.

Majal runs several critically important web platforms for activists, migrants, musicians, and LGBT people, especially in the Middle East. Each of their projects has different needs, but you won’t do better than giving money straight to Majal. Kickass human: Esra’a El Shafei.

CASH Music – no one works harder for musicians and what they do for society than these guys. And no one quite believes what CASH does: they make and give away free software and services to musicians, only because it’s the right thing to do. It’s so unusually altruistic that many funders actually don’t like giving them money. But we can, because we’re not assholes. Kickass human: Jesse von Doom.

The Debt Collective. If you’re in the US, these amazing people are among very few directly challenging predatory lending – and winning. For the rest of us elsewhere, let’s learn from them and apply their strategies in our own countries. Predatory lending, especially student debt, is a plague that won’t go away on its own. Kickass human: Astra Taylor.

Africa Check, and any good fact-checking organisation right now, are doing what journalism used to do by default. And they’re only the seeds of what we’re going to need in the coming years. Every cent you give them literally buys truth and buries lies. Kickass human: Peter Cunliffe-Jones.

Safecast. Simply put: get a portable Geiger counter from these guys and help grow the world’s biggest, public-domain database of radiation data. They’re also trialling devices for measuring air quality. This is just the beginning: what really matters is that Safecast proves that we don’t need governments or big organisations to quality-control earth. We can just do it our big-brained-primate selves. Kickass human: Sean Bonner.

Fight for the Future. If we’ve learned anything this last year, it’s that the Internet is a battleground for hearts and minds. FFTF have already saved your Internet bacon more than once and you probably don’t even know it. There is a lot to learn from them about Internet activism when you take your own stand, big or small. Kickass human: Tiffiniy Cheng.

ContentMine. If you’re not in academia, you might not know there’s a pitched battle going on for ownership of new scientific knowledge. No kidding, there are actually big companies that own and control the science we pay for with our taxes. WTF. ContentMine have a plan to counter that. Apart from donating to their work, if you have technical skills you can use their open tech to make scientific facts public. Kickass human: Peter Murray-Rust.

The Open Knowledge Foundation. The first task of dictators is to hide information from the public. It’s OKFN’s job to stop that from happening. OKFN have a network of groups around the world, which do things like make government information easy for people to find and understand. Joining one might be the closest you’ll come to joining a rebel assault on the Scarif databank. Except it’s legal and you won’t get killed.

Bettercare (I’m a co-founder and board member) publishes open-access learning material for nurses and midwives in badly resourced clinics and hospitals. This stuff helps save lives every day, especially the lives of women and children. Every incoming cent helps to create new material and keep existing material up-to-date and openly available. If you have a medical background, you can also help by using and reviewing the material. Talk to me for details.

Book Dash (I’m a co-founder and board member) gathers volunteer designers, writers and illustrators to create free, high-quality children’s books that anyone can download, translate, print or distribute. Developers can contribute to the open-source Book Dash apps, or help improve the Book Dash website. Talk to me for details.

MyConstitution.co.za (I’m the founder and project maintainer) is a community project to put the South African constitution, in all official languages, in the public domain –here in South Africa, you would not believe how hard it is for citizens to read their own constitution. The project needs managers, the tech needs developers, and the content needs legal experts and experienced translators. Talk to me for details.

All of these projects are close to my heart, and have structures in place for volunteer contributions or donations. I’ve kept the list short so you’d read it, but I’d also vouch for any others funded by the Shuttleworth Foundation, and a bunch I’ll kick myself for forgetting here.

Do the right thing. Sleep better. Wake up stronger.

Three things every editor should know about digital publishing

A while ago I gave this talk at the Cape Town Professional Editors Group. Here are my speaking notes.

Today, every passage you edit will sooner or later be read on screen. This digital world desperately needs our craft and high standards, but what does that mean for our daily work? In this talk I’ll pick out three big, important issues, and talk about some of the tools we’re using to tackle them. The first is text-only editing. That is, the end of word processing as we know it. The second is real-time, collaborative editing. And third, automagical pagination: how do we edit when there’s no such thing as ‘page two’?

So what does this digitisation thing really mean for editors? I think, basically, it means you’re editing text that will be read on a screen. Importantly, you’re editing text that will be read on a screen and on paper.

Now if you’re going to edit for the screen, the single most important thing is to actually read on a screen yourself. If you aren’t reading on screen, you simply cannot edit for the screen. Just like you can’t fix a car if you’ve never ridden in one.

That said, we are all busy people and there is an infinite amount to learn about computers: the rate at which the technium evolves far outstrips the rate that we can understand it. Even the greatest minds in computing readily admit that the Internet is now bigger and more complex than any one person understands.

So the trick is to not try too hard to learn it. Rather, just start using web- and screen-oriented tools and the learning will come when you need it. No one went to a whole seminar on how to use email before they sent an email.

In the next thirty minutes or so I’ll pick out three big, important developments and talk about some of the tools we’re using to tackle them. This is basically show and tell.

The first is text-only editing. That is, the end of word processing as we know it.

The second is real-time, collaborative editing.

And third, I’ll talk a little about automagical pagination: how do we edit when there’s no such thing as ‘page two’?

Text-only editing

First, what is text-only editing? Text-only editing is editing in plain-text files. When you do this, you’ll probably be using a particular writing structure called Markdown. For instance, let’s use Stackedit to write plain-text markdown. Type on the left, and on the right Stackedit turns our plain text into formatted HTML.

On the left, I type plain text in a markdown structure. On the right, formatted HTML.

On the left, I type plain text in a markdown structure. On the right, formatted HTML.

What we’re seeing here is the separation of content (which is structured text and image-references only) from formatting and design.

What are the big advantages of text-only editing?

  1. Smaller, faster files.
  2. Computers need perfect consistency (the digital age is a wonderful place for obsessive copy editors). Here the tools force our hand, and we learn to be less sloppy.
  3. Text-only means fewer copy-paste messes (when you copy paste into a new document and the fonts go all weird), because I’m getting only and exactly what I’m seeing. Plain text. We do have learn some new tricks like unicode glyphs (there is no ‘Insert symbol’ font or formatting gimmicks, like superscripting an o for a degrees symbol). This is actually a good thing, even if it seems like more work at first while we learn its tricks.
  4. Less file corruption, because there is simply less going on – less code to go wrong.
  5. Better version control, especially if you learn to use a tool like Git.

Collaborative editing

Collaborative editing has literally changed the way I write, edit and deliver documents.

What is collaborative editing? In short, me and someone else editing the same online document at the same time. The biggest tool for this is Google Docs.

What are the major pros of collaborative editing?

  1. It lets others watch while you work. And you can watch while others work. Publishing is weird because it’s always been a team sport played by lonely freelancers from their own home offices. Collaborative editing instantly makes the team aspect real and useful.
  2. You can use commenting for feedback and discussion. Track changes just isn’t the same as actual live annotation. No more emailing documents with increasing repetitions of the word ‘final’ in the file name. (Also, see Hypothesis.)
  3. Instant delivery of work and real-time review. As soon as you’re ready for your client to check something, share the doc and the ball’s in their court. So much editing is problem solving, and collaborative editing means the publisher-editor-designer are basically always in the room together at the same time.

I cannot believe that Google Docs has been around for years and people are still editing in MS Word. I promise, promise, promise you want to move all your writing and editing into Google Docs. (You could also use something similarly cloud-based with live collaborative editing but, for better or for worse, most people are familiar with Google and already have Google accounts).

Automagical pagination

Lastly, what is automagical pagination? Well, on screen, our software and screen size are going to decide how much text is on the ‘page’, the visible area in front of us. On screens we might refer to this as the ‘viewport’. If you’ve used Kindle, iBooks, or Google Play Books you probably know what this looks like.

There are a few key issues that arise when text flows into a viewport. And very importantly, when you’re editing the same text for both that viewport and also print output.

  1. Hyphenation and non-breaking spaces. Of course you never want to put a hard hyphen into a line because that line will be made and remade in countless different lengths in its life, and you don’t want your hyphens turning up in the middle of a line. You also don’t want the space in a number like 100 000 breaking over a line, so you need to learn how to insert a non-breaking space. And there are several other glyphs that have similar complications, like ellipses and en dashes.
  2. Cross-references. That is, referring to other places in the document. On screen, you can’t say ‘see page twenty’, because ‘page twenty’ is completely different on my computer and on my phone. You can’t say ‘Click here to go to the figure’, because in print there is nothing to click. And you can’t say, ‘in the figure below’, because on screen the figure might shift position. Common solutions are to introduce numbering systems for sections and figures, or to completely rephrase cross references. (Some smart digital-first workflows let you insert a variable that becomes a page reference in print, and is a hyperlink on screen.)
  3. Elements that appear on screen but not in print. For instance, let’s say you want to include a YouTube clip in an ebook, but you can’t have the clip in the print version. In some systems, it is possible to mark certain elements to appear in one version but be completely hidden in another.
  4. Minimalist courage. For maximum compatibility with unknown reading systems, you have to use fewer, more carefully-chosen features. You can’t have ten different variations on headings or boxes. Pick very few features, and treat them consistently with the same styling rules. Make no single-instance exceptions (e.g. never say “I’ll make this one heading smaller because otherwise it’ll look funny here.”)
  5. Strict content hierarchies. You have to place every feature of the book in a hierarchy, as if your whole book was a tree of trunk, branches and leaves. Computers need hierarchy.

There is lots more we could go into here but there isn’t enough time and we’d bore half the room. And as I suggested at the start, it doesn’t matter how much you try to stuff in your head now, when the only way to make it useful and make it stick is to deal with issues as they come up in your work.

I hope that you have some concrete questions, though, so we can spend some time dealing with those real issues that you’ve already come up against.

Uncapped: Can humans handle a blank cheque for data?

I remember the first time I used uncapped Internet. I was sitting in a friend’s coffee shop in Bristol, on a trip from South Africa, and my brain exploded. Till then, every time I’d clicked a link I’d worried about how much data it would use. I’d had to think carefully about every video, every download, every large page. And that anxiety was a constant force against browsing freely. That was just the way the web worked: you had a data allowance and, no matter how big it got, you had to use it wisely.

So uncapped Internet was not just fun, it was a revelation. Revolution, even. Suddenly nothing stood between me and anything my heart desired. I could indulge my curiosity at will. Games, video, porn, software, music, and learning. So. Much. Learning. That hour in a cafe in Bristol literally changed my life.

And now so many of us — almost anyone at the wealthier end of middle class — just take it for granted. Once we cross from limited to unlimited — from finite to infinite — we easily forget what it’s like to manage with limited resources. Our behaviour on one side of that line is very different from our behaviour on the other.

Doug Hoernle, founder of mobile-education business Rethink Education, tells me their young, low-income users are relentlessly careful about the data they consume on their phones. When a class is doing research, one student will open Wikipedia, screenshot the page, and WhatsApp the image to everyone else. This isn’t just to save data, but to ration it: they don’t know how big the Wikipedia page will be, but they can tell exactly how big a screenshot is before they download it. Similarly, before they decide to download a free app, they’ll check its size and calculate its cost in data: its real price. And even then, they’ll weigh up carefully whether that app’s size in their cheap phone’s memory is worth all the photos they could save in its place. For them, there is no such thing as ‘free’.

As much as I love unlimited data, it has a real danger: without a budgetary constraint on our browsing, there is far less pressure to choose carefully. We just open the Internet firehose and let it run. We’ll curate later, we think, and then we complain that there’s so much crap on the web and we never have any time for ourselves or our work. And then, perversely, we turn that firehose back on the web and upload our own stuff — often half-baked — for others to deal with.

Humans have a lot to learn about managing the firehose. I may learn a few tricks in my lifetime, and I’ll pass them on to my son, who’ll learn a few more. It’ll take generations for us to be comfortable, confident, happy dealing with an unlimited supply of information.

For some of those lessons, we should look to those who’re still capped, for whom every byte counts. How do they make their decisions? Who influences them? What sites and apps really matter? Are their constraints teaching them — at least till they cross that great divide — how to be more discerning people? Maybe they have something to teach us about priorities.

(This post was first published on Medium.)