In August, after a well-needed-longer-than-I-had-in-a-long-time ☀️ break, I started at The Things Network.

The first weeks have been both exciting and challenging. Hardware is almost completely new to me, yet it is my very job to make our technology more accessible. So why not start with building the documentation site I desperately felt a need for myself?

Requirements

We wanted the documentation site to meet a number of requirements:

1. Open Source on GitHub.
2. Anyone should be able to run the site locally.
3. Content in Markdown.
4. Working previews on GitHub, including embedded images.
5. Quick corrections via GitHub’s edit feature.
6. Automatically publish on push or merge.
7. Served at https://www.thethingsnetwork.org/docs.

Result

• Website: https://www.thethingsnetwork.org/docs/
• Source: https://github.com/thethingsnetwork/docs/

GitHub Pages

GitHub Pages hosts static websites, sourced from a GitHub repository. This means the site will update automatically when you push or merge and is served from GitHub’s world-wide data centers.

NGINX

GitHub Pages are served from https://<owner>.github.io/<repo>. You can setup a custom domain or subdomain, but we wanted to serve from a path under our main domain at https://www.thethingsnetwork.org/docs. NGINX reverse proxy to the rescue!

One gotcha; GitHub redirects URLs without the debated trailing slash to the github.io URL. We prevented this by rewriting these URL in the proxy configuration:

For the bots and people who somehow do end up on the github.io URL, I included a canonical URL and redirect.

Jekyll

GitHub only serves static websites, but we want our content to be in Markdown, not HTML. So we had to use a generator, but remember that our list required contributors to be able to edit directly on GitHub and the website to update on merge.

Jekyll to the rescue; the only static website generator that GitHub will run on your behalf after pushing or merging changes to the Markdown source files.

Guides

Inspired by Parse’s docs I use front matter to split up guides into sections. This allows us to style each as a block, re-use sections between guides and not have one long file to deal with.

Notice how by keeping the sections and images they use in a folder with the same name as the guide (which will compile to [filename]/index.html, the images also work in GitHub previews. You can even refer to images in other guides this way.

Since guides are not just pages, but configured as collections I have access to meta data which for example allows me to generate a list of guides automatically.

Versioning

As we’re about to formally launch our production preview, whilst also still supporting our current staging environment we needed to be have multiple versions available at the same time.

I dealt with this by introducing another collection called versions. By convention this folder has copies of the guides in subfolders which I use to name the version. For our production preview this is refactor.

I then loop over the versions collection in the guides layout to look up alternative versions for a guide. This might seem quite an effort, but the nice thing with static websites is that this is only computed when the site builds. Once generated, it’s just HTML.

External sources

I started out with all content in the documentation repository, but I recently began moving the API reference guides for our libraries to their own repositories for a number of reasons:

1. Require PRs on libraries to also document the changes.
2. Bundle documentation with each distributed version of the libraries.
3. Provide offline documentation.

I still wanted the documentation to include the API references as well, so I wrote a simple Gulp task to pull them in. Currently this does require me to manually run the task and push changes, but we could trigger this using Travis and GitHub hooks later.

Bootstrap

We already used some of Bootstrap on our website and it was a great fit to keep the documentation codebase free of lengthy custom stylesheets. It makes little sense to reinvent the wheel and style dozens of components Bootstrap provides you with out of the box.

Stylebook

While building yet another website around The Things Network, I started a small side-project to see if we could share styles between them. Quite the challenge if you consider that our websites use different technologies including Django and React, with styles in Sass, Less and Stylus.

The answer was a GitHub repository which pretty much any package manager can depend on directly nowadays. The styles are written in Less, which we automatically transpile to Sass and Stylus. We also include Bootstrap in all three languages and variables to customize it for our brand. Everything is set up in such a way that we can pick and choose.

Navigation Bar

Another element that needed some thought was the navigation bar we use on the main website:

Because we have several independent sites and codebases in different languages it’s hard to get the same result in a DRY way. What makes it even harder is that the the bar features the user state and – if logged in – name and photo.

Luckily I could borrow from my experience at Appcelerator. The documentation site loads a script from https://www.thethingsnetwork.org/navbar.js which is generated dynamically to include the user state and profile. The script writes the HTML for the navigation bar to the documentation body, which is styled using the stylebook.

Currently the forum and wiki still both have their own navigation bars, which we plan to replace with the one of the main website in the same way we now did for the documentation site.

Webpack

To source front-end JavaScript from NPM packages like TocBot and our own stylebook I use Webpack. This module bundler generates a single JavaScript file. Again this helps to keep our codebase on GitHub about content, not presentation.

HTMLProofer

It is very easy to for example break internal anchor links by changing a heading. We also like all images to have alt tags. To check for these kind of issues, I added HTMLProofer. A simple NPM script lets Jekyll build to a temporary folder which HTMLProofer will test for errors.

The Jekyll build itself is also a test for errors in front matter, templates etc.

Markdown-spellcheck

We like all content to use en-US spelling. To help us and contributors in this I use markdown-spellcheck. A simple NPM script allows you to spellcheck all Markdown files. The developer mode even triggers an interactive mode with suggestions to immediately fix errors.

Pre-commit

To prevent us and contributors from committing changes that don’t meet our requirements I added pre-commit. This NPM package automatically installs a pre-commit githook that is configured to make sure changes to JavaScript have been processed by Webpack and npm test passes before adding the packed scripts and doing the actual commit.

Travis

To make sure even Quick Pull Requests for small corrections meet our requirements, I’ve configured the repository for Travis to automatically run npm test. PRs that don’t pass the tests cannot be merged.

CLAssistant

To settle licensing of contributions, we’ve linked up all our open source repositories with CLAssistant. PRs cannot be merged until all contributors have signed the CLA. This only takes a few clicks the first time one does a PR.

Future ideas

There are still things I’d like to improve, like adding a Markdown linter and check for orphan (unused) assets. But right now the focus is on content and so far the current setup has proven to allow me and our contributors to focus on just that!

I hope this detailed walkthrough for our documentation site inspired you and gave you something to pick and choose from for your own site. Let me know if you have ideas for further improvements!