ckunte.net

Chisel

After eight years and two security breaches, I got tired of managing an increasingly complex server setup to run a weblog and content management system. After searching for a headless static site-gen. project with low code base that could be effectively managed with git and published on either github or other web service offering DVCS for effectively killing server security issues, I found Chisel, which I could fork and customise. In this post I describe how I got Chisel running, how I added customisations and handy automations.


setup and run

Python 2.7.x, the required scripting language to run Chisel, comes pre-installed on a Mac OS X, and so you’re ready to dive straight into the following steps. (Use Terminal1 to run commands in steps below.)

  1. Install python package management system, pip.

    sudo easy_install pip
    
  2. Install Markdown (a plain-text to html marking language) and Jinja2 (a template engine in python):

    sudo pip install markdown jinja2
    
  3. Save your copy of chisel under ~/Sites folder. Edit chisel.py and update the settings as below:

    SOURCE = os.environ["HOME"] + "/Sites/www/posts/"
    DESTINATION = os.environ["HOME"] + "/Sites/www/"
    TEMPLATE_PATH = os.environ["HOME"] + "/Sites/chisel/templates/"
    HOME_SHOW = 3
    URLEXT = ".html"
    
  4. With mkdir -p ~/Sites/www/posts ~/Sites/www/images, create a site structure that looks like below:

    ~/Sites
      |
      +-- www
      |   |
      |   +-- posts
      |   |
      |   +-- images
      |   
      +-- chisel
    
  5. Write posts in Markdown and park them in the sub-folder posts , e.g., my-first-post.md. (Separate words with a hyphen in file names in posts, images, files; avoid using spaces or other special characters — they are unfriendly to URLs.)

  6. Post pre-format: All posts require a minimum pre-formatted information for posts to be parsed into their HTML equivalents by Chisel. Following is the structure of a typical post:

    My first post
    2012-12-24
    
    Blah blah..
    
    Blah blah, blah..
    

    Line 1 is the title of the post. On line 2, the date follows the format of month/day/year. Line 3 be blank. Line 4 onward, one starts writing the post content (or the story), with paragraphs separated by a blank line, and so on.

  7. Run Chisel to produce posts in html with the following command:

    python ~/Sites/chisel/chisel.py
    

    Run chisel

  8. Run a local server, and load web log in the browser:

    cd ~/Sites/www
    python -m SimpleHTTPServer & open http://localhost:8000
    

Anvil for Mac

If you prefer a simpler interface for a web server, then Anvil for Mac, which is powered by pow, is a great tool to serve (multiple) static sites on your computer. Point it to a folder, and it auto-assigns a URL based on folder name, and you can then load it up on your browser. (For example, if my static site folder is ckunte, then Anvil serves it as ckunte.dev. If the folder is ckunte.io, then it serves it as ckunte.io.dev.)


nginx server setup

If you prefer a full featured server instead (of a tool like Anvil for Mac or SimpleHTTPServer python module), then the following is a step-wise process of how to set-up a local web server environment using nginx on a Mac OS X.

  1. Download and install rudix, which is a collection of pre-built UNIX software delivered as packages for Mac OS X.
  2. Update /usr/local/etc/nginx/nginx.conf with server_name and location directives (see below). Ensure /etc/hosts file contains this following line:

    127.0.0.1 localhost ckunte.dev
    

    Run dscacheutil -flushcache, if necessary, so the /etc/hosts file is refreshed. Here are all the contents of my nginx.conf file:

    worker_processes 1;
    events {
        worker_connections 1024;
    }
    http {
        include mime.types;
        default_type application/octet-stream;
        sendfile on;
        keepalive_timeout 65;
        gzip on;
        server {
            listen 80;
            server_name ckunte.dev;
            root /Users/ckunte/Sites/ckunte.github.io;
            charset utf-8;
            location / {
                index index.html index.htm;
                try_files $uri.html $uri/ =404;
                error_page 404 = /404.html;
            }
        }
    }
    
  3. Test the thus updated nginx.conf with sudo nginx -t. With syntax ok, and the test successful, run sudo nginx (or if it’s already running, reload it with sudo nginx -s reload).

  4. Load http://ckunte.dev in a browser. (I quite like .dev, because it hints that it’s local and offline for development, testing, and proof-reading.)
  5. nginx is mapped to an alias in my ~/.zshrc as below:

    alias srv="sudo nginx"
    alias srvstop="sudo nginx -s stop"
    

    This allows me to run nginx only when I need it, and stop when I don’t.


Autopair underscores and backticks

Sublime Text 2 in Markdown mode does not feature autopairing underscores or backticks, which I use for emphasis and for inline code respectively. To get around this, I wrote a .json file to enable this. Select all text from this file and add it under Preferences → Keybindings - User.


Automatic image link

Typically, all the images I (plan to) embed are stored in a folder called images, and I use Markdown syntax to add a relative URL into the post I write or edit.

In the past, I’d look up an image to embed, say, filename.jpg in the Finder, and then type the following in TextMate — my text editor. While this was fairly straightforward, typing a filename at times felt like a chore, i.e., a good case for automation.

Here’s the workflow:

  1. I open images folder in Finder, and select an image of choice. (Left window.)
  2. Then in TextMate, I type ;img at line I am looking to embed an image in the post, and Markdown-formatted image link for the selected image appears. (Right window.)

Obviously, I am using a couple of TextExpander snippets to get the desired outcome. Here’s how it works:

First snippet (named imgfn), an Applescript variety, gets the filename of the selected file from Finder, which is as follows:

--- Get filename of the file selected in Finder
tell application "Finder" to get "/images/" & the name of (selection as alias)

The second snippet (a plain text variety, named ;img) pulls the filename of the selected image file in Finder, formats it in Markdown and prints it in the text editor. The second snippet looks like below:

![](%snippet:imgfn%)

For screenshot images in this new vanilla theme, I have chosen to apply a class called screen so such images not only do not spread end to end, but also because screenshots are smaller in width (restricting to paragraph width). To automate this, a second snippet is created with a trigger ;im5, which works just like the above, but spits out html code instead of the Markdown code. The snippet is as below:

<img class="screen" src="%snippet:imgfn%" alt="">

While doing this, positioning windows side by side for better visibility, as seen above, works for me.


Validating markup (Sep 2014)

PyRSS2Gen worked fine as a stop-gap for the last couple of years, despite the fact that it kept generating invalid XML markup for my RSS feed. Thankfully, feed readers seemed more tolerant than the FEED Validator. Then, on a whim this weekend, I decided to fix the invalid markup, and so in one session, I plucked parts of PyRSS2Gen out from the generator script, which I had introduced two years ago, and then rewrote some of its code to generate an RSS feed natively. I needed a template in Jinja 2, which I further created for this purpose. (In a minute.) A couple of run-ins after with the validator — using various tags and markup escapes, the generated XML markup finally began to validate. Here’s the part I got rid of from the script:

import PyRSS2Gen, datetime

RSS = PyRSS2Gen.RSS2(
    title = "ckunte.net log", # title
    description = "ckunte.net", # description
    link = BASEURL + "rss.xml", 
    lastBuildDate = datetime.datetime.now(),
    items = [])

@step
def gen_rss(f, e):
    for file in f[:3]:
        RSS.items.append(PyRSS2Gen.RSSItem(title=file['title'], link=BASEURL + file['url'], description=file['content'], author="ckunte", guid = PyRSS2Gen.Guid(BASEURL + file['url']), pubDate=datetime.datetime(file['year'], file['month'], file['day'])))
    RSS.write_xml(open(DESTINATION + "rss.xml", "w"))

which I replaced with the following:

RSSDATE_FORMAT = "%a, %d %b %Y %H:%M:%S %z"

def write_feed(url, data):
    path = DESTINATION + url
    with open(path, "w") as file:
        file.write(data.encode('UTF-8'))

@step
def gen_rss(f, e):
    write_feed('rss.xml', e.get_template('feed.html').render(entries=f[:HOME_SHOW]))

The feed.html template is as follows:

<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
   <title>ckunte.net</title>
   <description>ckunte</description>
   <atom:link href="http://ckunte.net/rss.xml" rel="self" type="application/rss+xml"/>
   <link>http://ckunte.net/</link>
   <language>en-us</language>
    {% for entry in entries %}
    <item>
      <title>{{ entry.title | escape }}</title>
      <link>http://ckunte.net/{{ entry.url }}</link>
      <guid>http://ckunte.net/{{ entry.url }}</guid>
      <pubDate>{{ entry.rssdate }}GMT</pubDate>
      <description>{{ entry.content | escape }}</description>
    </item>
    {% endfor %}
</channel>
</rss>

There’s a bit of unintended appendage — see the GMT part? The time zone is supposed to be generated by python (%z), but for some reason it wasn’t. So, I plugged it with a quick and dirty manual tag, filing the problem away for later. At this point, the feed was validating, and so I stopped, since it solved the problem.

Buoyed by the idea, I thought I could take on another validation exercise. This one occurred in the footnotes. This wasn’t because of my plain text posts, but because of markdown in python. So, hoping for some improvement, I upgraded it to the latest version with the following:

sudo pip install --upgrade markdown

There was a new version (2.5) alright, but it broke the script. (Yes, these things happen.) Errors suggested it was to do with markdown and mdx_smartypants — a third party extension that played well with an earlier version of markdown in python. Just to re-check I wasn’t seeing stars, I uninstalled the latest version of markdown in python and replaced it with the last working version (v 2.4.1):

sudo pip uninstall markdown
sudo pip install markdown==2.4.1

That got it working again. To be sure, I looked through 2.5’s release notes to realize that some call instructions to parse were different. For instance, I had this line:

FORMAT = lambda text: markdown.markdown(text, ['smartypants','footnotes'])

The release note advocated the following instead:

FORMAT = lambda text: markdown.markdown(text, extensions=['markdown.extensions.smarty','markdown.extensions.footnotes'])

Upon further checking through supported third party extensions, I realized mdx_smartypants did not make the list, which meant I needed to use the official extensions within markdown as above. So, I uninstalled mdx_smartypants, and replaced the above to get it working. The official smarty extension turned out to be different in the way it encoded emdash and endashes. (With mdx_smartypants, it was two dashes to produce an emdash, and one for endash. The official smarty extension requires three dashes to produce an emdash, and two for endash.) So, batch search and replace to the rescue:

perl -pi -w -e 's/ -- / --- /g;' *.md

Now all is well again; the update to the new markdown 2.5 did not fix the validation errors concerning footnotes; so posts with footnotes still do not validate. But I am happy to note that from this exercise, the line count got reduced by two. :-)


Optimizing external fonts (Sep 20)

I use three external fonts for this site. Three functionally, with the fourth limited only to serving a small subset of English letters in caps. Given my specific need, loading the fourth seemed like an excess, and I began looking to find a way to reduce its footprint. Engineers at Google Fonts, as it happens, have already thought through this problem. (I learned this from Dr. Levien.) The fourth font is used only for titles and navigation links on the page, so to avoid repeating characters, I wrote a tiny script to generate my list of unique characters:

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
"""
unique-letters.py

Finds unique set of letters in a sentence. Helps me minimize 
external font overhead: http://goo.gl/9unwL .

Created by ckunte on 2014-09-16.
"""

def main():
    x = "THIS IS AN ENGINEERS PERSONAL LOG COLOPHON ARCHIVE HOME"
    print "".join(list(set(x)))
    pass

if __name__ == '__main__':
    main()

x in the above is the list of words I use, which I’d like my forth external font to display. Instead of filtering it manually, the above script reads it and print a unique string of non-repeating characters out.

$ python unique-letters.py 
A CEGIHMLONPSRTV

Using the list above, I construct the URL to import in to my CSS, as below, thereby limiting font-character overhead in site’s load times by 92%.

@import url(http://fonts.googleapis.com/css?family=Lato:400,900&text=ACEGIHMLONPSRTV);

URL + title with Text Expander (Sep 27)

I love the fact that Markdown does not force me to have links inline, thereby keeping it clean and very much readable as a plain text file. Whenever I create a link, I’d also like to include its title text more often than I actually do. This is because doing it manually is an extra step, and when words are flowing, there is no time to pause. But it does not have to be this way, and I realized TextExpander can handle this well; it’s just that I need to break it down into a couple of Applescript snippets, before calling them both in a third plain text snippet. Here’s how:

Applescript snippet 1 ;u1:

tell application "Safari" to get the URL of the front document

Applescript snippet 2 ;u2:

tell application "Safari" to get the name of front document

Plain text snippet 3 ;uri, which calls both these above in Markdown format:

%snippet:;u1% "%snippet:;u2%"

When I hit ;uri, I get the URL of the link, followed by its title in quotes.


Homepage image roll (Oct 19)

For the last three years, the homepage has been showing a static image; and I wondered if I could cook-up a simple javascript to random-rotate images from the stories I write here. The weekend exercise resulted in the following code for image rotation (the number of images in the code below are just examples. Actually I have a longer list, which are cut out for brevity):

var images='4711.jpg%andamansea.jpg'.split('%');
var desc='From our visit to <a title="4711" href="/2007/cologne.html">Cologne</a> in 2007.%Our boat hit something, <a title="Andaman Sea" href="/2013/andaman-sea.html">knocking the propeller out</a>.'.split('%');
function getRandomImage(b,c,a){a=a||"/images/";var d=Math.floor(Math.random()*b.length);document.write('<img src="'+a+b[d]+'" alt=""><p>'+c[d]+'</p>');document.close()};

I call this script in the head section like this:

<script type="text/javascript" src="/css/rotimg1.js"></script>

Now to actually display a random image within the body section, I add this following:

<script type="text/javascript">getRandomImage(random_images_array);</script>
<noscript><img alt="" src="/images/img.jpg"></noscript>

The image rotation occurs only when the page loads or is refreshed — this is deliberate; I did not want it to be a slide-show. The noscript is handy when a browser has javascript switched-off, which results in showing the static image that’s been on display until recently. Now with different images of different size and aspect ratios (since I am trying to use a common pool of images used in blog posts and not a separate set of images for this purpose), the next challenge was to find a way to display them 1:1 like cover images (without cropping them permanently). After a while, I learned I could do this by applying object-fit:cover;overflow:hidden; directive to the image that the above script would randomly display.

But as with browsers, there’s always a regret, and in this case, it’s only a WebKit thing (i.e., it only works in Google Chrome and Apple Safari), while the image shows stretched in Firefox, Internet Explorer (and likely in Opera; I haven’t checked this.) There was a promising polyfill hack I tried — hoping to play nice with the ones that hadn’t had a support for object-fit yet, but that’s not working for me at the moment; and I do not want to convert it into a random background-image as a workaround.


Copyright notice (Oct 22)

I was planning to add a copyright notice, but that didn’t make sense for a non-commercial, personal website like this. Here’s the code that works:

Copyright &copy;<script type="text/javascript">document.write('2010&ndash;',new Date().getFullYear().toString().substr(2,2))</script> C.Kunte. All rights reserved

I have to resort to javascript — credits to Martin at Stackoverflow, since this is a static website, without server-side scripting.


Responsive design (Nov 16)

The site is now responsive, by which I mean, on a handheld, the type is much readable than before. In a single .css file I employed a couple of @media directives to serve type and layout for the desktop, tablet and phone.

The following is for the phone:

@media screen, handheld {
    ...
}

This following is for devices that are not handhelds:

@media screen and (min-width:36em) {
    ...
}

The minimum width is closer to but smaller than the #container width of this layout.


Sitemap (Nov 21)

I haven’t found the need to generate a sitemap for this site, since I exclude it from being indexed by search engines. But here’s a working solution, as I played with the template engine today.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <url>
    <loc>http://ckunte.net/</loc>
    <changefreq>weekly</changefreq>
    <priority>1.0</priority>
   </url>
   {%- for entry in entries %}
   <url>
    <loc>http://ckunte.net/{{ entry.url }}</loc>
    <changefreq>weekly</changefreq>
    <priority>0.9</priority>
   </url>
   {%- endfor %}
</urlset>

I added a couple more lines to the generator so the above template is used to actually generate the sitemap.xml.gz file:

import gzip

def write_sitemap(url, data):
    path = DESTINATION + url
    with gzip.open(path, "wb") as file:
        file.write(data.encode('UTF-8'))

@step
def gen_sitemap(f, e):
    write_sitemap('sitemap.xml.gz', e.get_template('sitemap.html').render(entries=f))

The above combination now generates a full sitemap. I chose not to add lastmod tag, since it’s optional, and stayed with only the priority tag.


Markdown filter for pages (Jan 2015)

Thanks to jinja2-markdown extension (hat tip to Daniel Chatfield), I can now generate my static page (viz., the colophon) also in Markdown. I had to modify a line defining env to enable this:

env = jinja2.Environment(loader=jinja2.FileSystemLoader(TEMPLATE_PATH), extensions=['jinja2_markdown.MarkdownExtension'], **TEMPLATE_OPTIONS)

Front matter for future exports (May 2017)

A number of static site generators use front matter. This script would be handy to update the frontmatter, should one decides to move on from chisel to some other static site generator. Put this is a .sh file and run it.

# Frontmatter
# Prepends `title =` to 1st line in all files
for f in *;
  do sed '1i\
  title = ' < $f > tmp_$f ;
  mv tmp_$f $f ;
done
# Prepends `date =` to 2nd line in all files
for f in *;
  do sed '2i\
  date = ' < $f > tmp_$f ;
  mv tmp_$f $f ;
done

Post navigation (Jul 2017)

I’ve closed this issue on @dz’s chisel repository now, which I’d opened in 2015.

It has been over five years since I began using chisel to produce this site, and it’s fast approaching the longest time I used only one other before. All the refinements I added over time to chisel have all been on my time, without being jolted out to. So taking this road less traveled has been well worth it. Still there was this nagging issue of missing post navigation on this site until recently — talk about bare necessities! Thanks to Ankur Tyagi, I closed that gap this year. Now, each individual post page offers a previous and a next post link below (i.e., if next and / or previous exist in the archive, of course).

Post navigation

While writing this post, I made one change in that code to ensure consistent behaviour, since urls, but not titles, are unique on my site:

From:

{% set curr = entry.title %}

To:

{% set curr = entry.url %}

Subsequently once more in an if loop.

From:

{%- for entry in entries %}
  {%- if entry.title == curr %}
    {%- if loop.index0 > 0 %}
    ...

To:

{%- for entry in entries %}
  {%- if entry.url == curr %}
    {%- if loop.index0 > 0 %}
    ...

  1. For a prettier Terminal, see zsh