Markdown vs. HTML – What Makes the Most Powerful Notes Editor?
TheBrain Blog

Harlan Hugh
Aug 1, 2020

Markdown vs. HTML – What Makes the Most Powerful Notes Editor?

The all-new custom-built notes editor is the main feature of TheBrain 11. One of the key changes is that it uses Markdown instead of HTML as its underlying format.

Why did we choose Markdown over HTML? What are the benefits and what does it mean for the future? If you'd like to know, read on.

In a recent post responding to one of our customers about why we developed Markdown editor for TheBrain, I rather bluntly stated:

HTML was never intended as a medium for composing text in and the fundamental design of it has severe limitations that make it extremely clumsy to adapt to that purpose.

This was justifiably met with some rather civilized disagreement as being inaccurate and overly broad.

I started composing a response and wound up with a rather lengthy discussion on the drawbacks of using HTML to write notes and the benefits of Markdown which I decided to convert into this blog post. While the overwhelming majority of feedback on the new editor has been positive, I think it's not completely clear to everyone why we went with Markdown as the underlying format. I hope this will be enlightening in that regard.

HTML-Markdown-TheBrain comparison

Why HTML Sucks for Writing Notes

I love HTML and it's truly a pillar of the modern world. But for writing notes, it is less than ideal to put it mildly. To clarify my original statement: HTML does not work well for the casual user quickly composing notes. To illustrate, let's consider the two ways you can create HTML - manually in a text editor or via a WYSIWYG editor.

While I'm sure most people would not advocate it generally, writing HTML by hand is painstaking and laborious. It may be obvious but a brief discussion of the reasons is worthwhile. The tags that specify paragraphs, line breaks, headings, boldface, lists, images, links, and just about everything else beyond plain text are a major distraction at best. They are also error-prone as they are sometimes lengthy and require significant effort to type with the correct syntax. Also burdensome is the need to match opening and closing tags… I don't think anyone wants to write HTML by hand - they do it on occasion when making a website - but not for the content that makes up the articles on the site. If you attempt to do this in many cases you will wind up spending as much time or more fiddling with the HTML as you do writing the actual content. And then there is the fact that what you compose looks almost nothing like the final output - as a manual HTML author you must constantly switch between a display view and composing view.

HTML Editing vs Output

For all the above reasons and more, WYSIWYG editors for HTML are now the norm and there are many commercial and open-source options. Unfortunately they don't work very well for our purposes. There are fundamental issues with the nature of HTML that make is unsuited for use as an underlying format for a WYSIWYG editor.

It is difficult for the editor to translate what looks like cleanly formatted text when rendered into cleanly formatted HTML. It's not impossible, but it is difficult. For example, looking at the HTML output from Word will make your head explode and the output from most other WYSIWYG editors is not that far behind. One reason is because HTML is just too flexible. There are many ways you can express the same thing. Tags can be added that serve no purpose yet are difficult for all but the most sophisticated parsers to detect their redundancy. HTML can specify just about anything - and this is a problem as a WYSIWYG editor cannot function in every case. Even if you do manage to get clean HTML when typing in a WYSIWYG editor, pasting HTML content into it from other sources will inevitably end up as a mess.

The user experience when typing, editing, and moving the cursor in general in a WYSIWYG HTML editor is a major problem and one we spent many moons trying to solve. While at first blush everything seems fine the more time you spend writing in an HTML editor the more you come to loathe it. Small little niggles, like the inability to return the cursor to its original place by placing pressing the down arrow an equal number of times you pressed the up arrow, build up into a catalog of hundreds of things that you expect to work but just don't. Many of these seemingly tiny issues boil down to intractable problems like the fact that the cursor position cannot be tracked and specified accurately.

In addition to the above, I'll mentioned a few other dragons that IMO cannot be slain when using a browser-based HTML editor: unpredictable and overly aggressive undo/redo; poor results when pasting HTML; high CPU and memory usage; slow loading/saving/initialization; inability to work at 60 FPS in certain environments, problematic interfacing between the browser Javascript and native code… I could go on, but I imagine you're getting the gist.

I'm speaking from experience on this topic. TheBrain versions 4 through 10 (more than a decade) utilized a WYSIWYG HTML editor. Most recently we used TinyMCE in TheBrain 9 and 10, spending over 4 years tweaking and customizing it to our needs. Before selecting TinyMCE we experimented with many many others - they all suffer from the fact that they are built on the contenteditable attribute and Javascript to manipulate and understand the browser DOM. We documented hundreds (maybe more than a thousand but I don't have the exact count handy) of bugs that were virtually unassailable because at their root they relied on a poor foundation.

Markdown to the Rescue

In light of all these issues, we made what at the time was a risky decision to abandon off-the-shelf editors completely and rewrite everything from scratch. We chose Markdown as the basis for the underlying document format.

Markdown exists thanks to John Gruber, who created the original parser and specification. As he said here:

The overriding design goal for Markdown’s formatting syntax is to make it as readable as possible. The idea is that a Markdown-formatted document should be publishable as-is, as plain text, without looking like it’s been marked up with tags or formatting instructions.

This approach has resulted in a format that is very easy to read, to type, and to parse. I believe that is why it has widespread adoption as the format of choice for many writers and publishing tools (blogs, content management systems, etc.).

Editing Markdown vs Output

Markdown was intentionally limited to not include the ability to specify fonts, colors and sizes. As shown by history, these abilities tend be abused over time and lead to the content getting mixed up with the presentation. Also, divorcing presentation from content allows consistent representation meaning in the document enabling automatic outlines, collapsing of sections and more.

We have extended Markdown in ways that make it well-suited as a replacement for an almost WYSIWYG editor. Certain features that users expect but were intentionally excluded from the original spec, like the ability to use different fonts and colors are compromises that I think work out well in both matching former WYSIWYG user expectations and keeping a clean underlying textual representation. We have deemphasized the UI for these features with the intention of encouraging users to use overall style settings generally instead.

It bears mentioning that the visibility of tags is somewhat of a dividing feature/bug.

One benefit of seeing tags during composition is that it allows one to type exactly what one wants without having to resort to command keys and/or UI actions with the mouse. As a corollary you can also delete, copy, and paste them with ease. The power of this for an experienced writer should not be underestimated. It's like being able to use every application that uses markdown as an expert user instantly.

Some people (most proponents of Markdown) appreciate the concrete presence of tags so they can see exactly where things are. For example, when typing a word in italics and the cursor is at the edge of the word it is immediately obvious if the next character typed will be inside the tag or outside. If anyone remembers WordPerfect's "reveal codes" feature, they can relate to the utility of this. Others find them distracting and would prefer never to see them.

Most Markdown editors are essentially plain-text editors with perhaps some color coding and sizing added for clarity paired with a synchronized preview window that shows what the final output will look like. This is less than ideal and feels like going back into the 19th century for those at home with Microsoft Word.

TheBrain's Almost WYSIWYG Markdown Editor

Because TheBrain's Markdown editor is much more than a plain text editor we can do things like show images in-line, hide tags that specify links, fonts, colors, and other conveniences. At the same time we surface representations of these tags so they are evident to the user and can be selected, copied, pasted, etc.

We have designed TheBrain's editor in an attempt to get the best of both worlds. Most tags (but not the ones that are very long and obtrusive such as images and links) are shown while editing in a subdued manner that blends them into the background of the page while still allowing you to see them if you look for them. At the same time the results of the tags are shown exactly as they would be in a WYSIWYG editor. A single click hides the tags completely.

Editing Markdown in TheBrain vs Output

The result is an almost WYSWYG editor that can do everything users expect, where content can be easily exchanged with other applications (whether text-based, Markdown-based, or HTML-basked) while still outputting perfectly clean HTML (or PDF if you like).

We have a lot of things coming up that would never have been possible without the new editor that was introduced as part of TheBrain 11. I'm pleased beyond my highest hopes at the results both as they stand in version 11 and as a basis for the future.

The precise control we have now that the editor is almost 100% our own code is unprecedented. It has already allowed us to do things with the editor like: roll it out nearly identically on Windows, macOS, iOS and Android and the web; offer nearly a thousand fonts that dynamically load as needed; utilize the same editor for small controls like searching and renaming; support drag/drop and copy/paste of images, videos, and thoughts directly into notes; clean pasting of HTML from external sources; and much more.

As you might imagine, we have plans to leverage the editor to a much greater degree in version 12 and there are numerous features that I've been wanting to develop for years that are now within reach.

We know that some people would like to be able to keep tags hidden while editing the document. While we can't yet commit to when it will be released, we did have this in mind when building the editor and it is planned for a future version.

Thanks for reading. If you made it to here I'm glad it was entertaining enough to hold your interest.

Keep thinking,
-Harlan


Tags: Content Management Information Management TheBrain Technologies

Filed under: Feature Focus and How Tos

More posts by: Harlan Hugh