NOTE: #275 reduced the default number of velocity/position solver iterations. If this causes an issue, you can manually set the solver iterations back to 10 and 10 which were the old defaults using Settings.setVelocityConstraintSolverIterations and Settings.setPositionConstraintSolverIterations. This change was made based on testing where it was found that a lower number of iterations reduces jitter and the overall system energy without significant fidelity loss. The performance improvement was a side effect.

NOTE: #287 will now wake the body when certain changes are made to it. Please review the issue for the details. If you relied on the body staying at rest in these situations, just use setAtRest(true) after making the change.

See the release notes for all the detail.

See the release notes for all the detail.

A major change from previous versions are the settings for motors and springs. Before, you would disable these features by setting their values to zero or some default value. Now, these features are enabled/disabled using specific set{feature}Enabled methods. You can still set those settings to zero/defaults, which effectively disables them, but you lose the original value. For example, if you want your spring stiffness to be 10.0 and you want to toggle it on or off based on user interaction, you have to store the 10.0 somewhere, then set it to zero to disable the spring, then set it back to 10.0 to re-enable the spring with the correct value. These new methods allow you to set it to 10.0 and then toggle the behavior on/off separately.

Another major change from previous versions is on the WheelJoint and PrismaticJoint. These joints have had their body arguments reversed so that the given axis of allowed motion is fixed to the first body rather than the second. This shouldn’t impact existing code too much, but generally you’d need to reverse the first and second body and potentially negate the axis you had before.

Apart from these breaking changes, many of the joints see new features to help reduce the number of joints between bodies to achieve a desired effect. For example, before you would need to use both a WeldJoint and a RevoluteJoint if you wanted an angular spring with limits - this is now possible with only a WeldJoint. Another example is if you wanted a linear spring with prismatic motion, you’d need to use both a PrismaticJoint and a DistanceJoint - now you can just use the PrismaticJoint.

The class documentation for all joints has been revised to be clearer and reflect all the changes described above.

As always with a major release, all deprecated APIs have been removed.

See the release notes for all the detail.

Part of the problem is the translation from a complex shape to a collection of convex pieces. There are many algorithms that take simple polygons and break them down into convex pieces (called convex decomposition), but often the output is not ideal for realtime simulation. It’s common for small and/or thin convex shapes to be generated on top of the shear number of shapes. That said, it’s not so much a problem with decomposition, but more with the input to these algorithms.

To help with this problem, version 4.2.0 of dyn4j added a new feature called polygon simplification. This is the process of altering the input to the convex decomposition to produce better results.

# Concept

The concept here is simple: to improve the quality of collision shapes by adding a pre-processing step before the convex decomposition process. This pre-processing step, called polygon simplification, reduces the number of vertices in the source polygon. The goal of this process is NOT to keep the input the same, but rather to alter the shape to remove insignificant features.

Consider the following example in Figure 1. For the most part it’s a pretty standard shape, but there’s a little piece of it that sticks out. A possible decomposition of the shape is also shown: two shapes, one triangle and one polygon.

While this is a perfectly valid decomposition, it’s not great for physical simulation. The green triangle is small and thin which can cause this object to behave oddly with other objects. Things like sticking together during collision come to mind.

The question is whether the green bit is even necessary for realistic collision. It might be, but often it’s not, so what happens if we just ignore the green triangle? You could post-process the convex decomposition and remove problem shapes based on area, length, or other criteria, but you run the risk of removing shapes internal to the overall decomposition.

Consider figure 2 of a non-convex polygon that we’ve decomposed. If you simply remove the purple triangle you’ll leave a gap which is probably worse than leaving the problem triangle in. Is the purple triangle providing any tangible benefit to physical simulation? No, it’s just noise. We need a way to pre-process the simple polygon to simplify it further, then decompose it.

NOTE: While this is a poor example of a convex decomposition, it’s a perfectly valid one. Let’s imagine the purple and green shapes were combined, does the additional information of that extra feature provide value to physical simulation? Again, it’ll depend on your use-case, but I’d argue that it has no value and only slows down the simulation.

NOTE: While polygon simplification and convex decomposition are great, it’s no substitute for manually designed collision shapes.

# Algorithms

There are a number of algorithms out there to do polygon simplification but I’ll be focusing on three in particular:

• Vertex Cluster Reduction
• Douglas-Peucker
• Visvalingam

All of these attempt to remove insignificant features of an input simple polygon. Consider the input of these algorithms an ordered list of vertices of the simple polygon in clock-wise or counter clock-wise winding.

# Vertex Cluster Reduction

By far the simplest algorithm to implement and understand. The basic concept is to remove adjacent vertices that are too close given a minimal distance. The algorithm is \(O(n)\) complexity and can be used as a first pass before employing another algorithm.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
// NOT INTENDED TO BE SOURCE CODE - MORE PSUEDO CODE
List<Vertex> reduce(List<Vertex> vertices, double minDistance) {
    List<Vertex> simplified = new ArrayList<Vertex>;
    int size = vertices.size();

    Vertex ref = vertices.get(0);
    simplified.add(ref);
    for (int i = 1; i < size; i++) {
        Vertex v0 = vertices.get(i);
        if (ref.distance(v0) > minDistance) {
            // set the new reference vertex
            ref = v0;
            // add it to the simplified polygon
            simplified.add(v0);
        }
        // if it's smaller, then we ignore the vertex
        // and continue
    }
    return simplified;
}

If we applied this to our second example above, we’d remove one of the red points and end up with the following simple polygon and it’s respective decomposition:

Nice! But the algorithm can’t catch cases like figure 4 where the vertices are not clustered. Vertices P1, P3, P4, and P6 provide little value to the overall structure of the shape. Unfortunately, decomposing this will produce an unnecessary amount of elements. For this we need something more complex.

# Douglas-Peucker

The Douglas-Peucker algorithm is a \(O(n log_2 n)\) complexity algorithm that actually operates on polylines instead of polygons, but can be adapted for polygons. The basic concept is to draw a line from one point to another, check the distance to the line for all points in between, if all the points are less than a minimum distance, then remove all those points. If there’s one point that’s more than or equal to the minimum distance, subdivide the polyline into two, and perform the same steps on each one.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
// NOT INTENDED TO BE SOURCE CODE - MORE PSUEDO CODE
List<Vertex> dp(List<Vertex> polyline, double max) {
    int size = polyline.size();
    Vertex first = polyline.get(0);
    Vertex last = polyline.get(size - 1);

    double maxDistance;
    int maxVertex;
    for (int i = 1; i < size - 1; i++) {
        Vertex v = polyline.get(i);
        // get the distance from v to the line created by first/last
        double d = distance(v, first, last);
        if (d > maxDistance) {
            maxDistance = d;
            maxVertex = i;
        }
    }

    if (maxDistance >= max) {
        // subdivide
        List<Vertex> one = dp(polyline.sublist(0, maxVertex + 1));
        List<Vertex> two = dp(polyline.sublist(maxVertex, size));
        // rejoin the two (TODO without repeating the middle point)
        List<Vertex> simplified = new ArrayList<Vertex>();
        simplified.addAll(one);
        simplified.addAll(two);
        return simplified;
    } else {
        // return only the first/last vertices
        List<Vertex> simplified = new ArrayList<Vertex>();
        simplified.add(first);
        simplified.add(last);
        return simplified;
    }
}

If we run through this process until completion, we end up with vertices P1, P2, P3, and P6 removed - nearly half the verticies. This results in almost half the convex shapes in the decomposition. In addition, the parts that are left are nice and chunky too.

NOTE: The number of vertices reduced will be highly dependent on where the splits are since there’s no re-processing of those vertices where the splits occur. It shouldn’t be a problem since these vertices are the farthest from the line, but it’s possible this could be an issue for the initial split.

The first image is an example of one of the stages of the algorithm where it removed vertices P3 and P4 since they were not sufficiently far from the line between P2 and P5 (not labeled).

# Visvalingam

Visvalingam is an alternate \(O(n log_2 n)\) algorithm that takes a similar, but slightly different approach. Instead of computing the distance between the points and a line and recursively subdividing, it looks at the triangular area of each vertex, sorts them, and removes them until the only ones left are above a certain minimum area. For example, using same shape as before, but highlighting all the triangular areas:

We can see from this picture that the small blue, purple, black, and red areas would clearly be elimitated due to their area, whereas the large green, black, red, and purple areas would remain. The areas are removed by removing the vertex that creates the triangle.

One benefit of this algorithm is that it’s simpler - no subdivision or recombining is necessary nor recursion. The algorithm goes something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
// NOT INTENDED TO BE SOURCE CODE - MORE PSUEDO CODE
List<Vertex> visvalingam(List<Vertex> vertices, double minArea) {
    // iterate on all input vertices O(n) and compute the area
    // and put them in a priority queue to sort them by area O(log n)
    Queue<VertexWithArea> queue = new PriorityQueue<VertexWithArea>();

    int size = vertices.size();
    for (int i = 0; i < size; i++) {
        Vertex v0 = vertices.get(i == 0 ? size-1 : i-1);
        Vertex v1 = vertices.get(i);
        Vertex v2 = vertices.get(i+1 == size ? 0 : i+1);
        
        double area = getTriangleArea(v0, v1, v2);
        VertexWithArea vwa = new VertexWithArea(v1, area);
        queue.add(vwa);
    }

    // iterate over the priority queue and eliminate vertices
    do {
        VertexWithArea vwa = queue.poll();
        if (vwa.area < minArea) {
            // remove this vertex

            // NOTE: you need to recompute the area of the adjacent vertices
            // and have them get resorted in the priority queue (just remove them
            // and re-add them back or something)
        }
    } while (!queue.isEmpty());

    // what's left is the simplified polygon
}

This would yield the same result as the Douglas-Peucker algorithm above, but that’s not guaranteed. Another nice attribute of this algorithm is that it’s ever evolving the result, to the point that it could be use for other purposes like zoomable maps.

The downside of this algorithm is the selection of the minimum area. It’s much harder to select that properly than a distance metric.

# Self-Intersection

The last topic I want to mention is that of self-intersection. All these algorithms will reduce the simple polygon by removing vertices. The removal of a vertex is not guaranteed to prevent self-intersections in the resulting simplified polygon. For example, figure 7 shows a simple polygon where the smallest triangle (green area) contains another vertex (P5). If we remove P2 (and therefore the green triangle) we’d create a self-intersection - the segment from P1 to P3 would intersect the segments connecting to P5.

We could handle this in a straight forward way by testing all segments against the segment we’re adding (P1-P3), but that’d be pretty inefficient and take our computational complexity up to \(O(n^2)\). If you squint really hard though, you can probably see that this is just a collision detection problem and we’ve dealt with it before.

Recall that for global collision detection we have a phased approach that starts with a broadphase which eliminates those pairs that cannot intersect, but allows some pairs that may not be intersecting proceed to the next phase. The next phase, the narrowphase, performs the accurate collision detection. We can apply the same concept here, in fact, you could even employ the very same algorithm and implementation. Dump all the individual segments into your broadphase and then everytime you detect a vertex to remove, check the resulting line segment for collisions with the broadphase. Then, check those potential collisions using a simple segment-vs-segment intersection test. You can even stop the whole collision detection process when you detect the first collision. Nice!

We’ve now detected a self-intersection, but what do we do about it? Well, the choice is yours really. I raised the white flag and just left the vertex in the resulting polygon and continued processing other vertices. You could try to generate a new point which removes the feature, but doesn’t create the self-intersection too.

# Results

To top things off I’d like to share some results of how well this process can work for complex simple polygons.

# Final Remarks

In summary, polygon simplification can be used to reduce the complexity of simple polygons to support better decompositions which supports better physical simulation. These algorithms can be used in realtime simulations to cleanse user input, but will modify the input. Choice of the minimum distances/area is the remaining hurdle to using these without user input. One thought I have for that would be to analyze the original simple polygon and come up some metrics to automatically determine the minimum values. Ideally all of this would be part of a toolchain where you can specify these parameters to achieve the most optimal result for your scenario, likely tweaking the result further manually.

# References

• Reference implementation for this post
• Vertex Cluster Reduction
• Douglas-Peucker (Wikipedia)
• Douglas-Peucker
• Visvalingam
• Visvalingam with self-intersection avoidance
• Visvalingam Paper
• A source with a lot of information/links on line simplification
• Coastline Paradox
• Fractal Dimensions

If you’ve ever done performance analysis or improvements to a piece of code, you are aware that improving performance by saving one object allocation, for example, is nice, but often not very impactful. Naturally, if you’re able to find lots of those instances, these can add up, but the most impactful performance improvements come from algorithms. Choosing algorithms that are better suited to the problem, more efficient wrt. complexity, better cache locality, etc.

Specifically, this performance improvement was in the Broadphase collision detection system and how it handles detection from frame to frame.

# Background

First a bit of background. dyn4j includes a system for detecting collisions between objects. This system is broken down into three main phases:

• Broadphase
• Narrowphase
• Manifold Generation

Why the 3-phase approach? The narrowphase algorithms are expensive to run, so if we can reduce the possible set of colliding pairs, then we save a lot of processing time. If we didn’t do this, we’d need to test every object with every other object. That’s \(O(n^2)\) tests - very inefficient especially when it’s more common to not be colliding than they other way around.

Thus, the broadphase is where we detect all the potential pairs of colliding objects. More specifically, it returns a set of collision pairs that may be colliding. Those pairs that are not included in this set are definitely NOT colliding. The pairs are handed over to the narrowphase where we find out if they truly are or aren’t colliding. Now, how does it do this? And how does it do it more efficiently than the narrowphase?

# Broadphase Collision Detection

Broadphase collision detection algorithms center around the idea of choosing a simple bounding shape that encompasses the object. For example, let’s imagine I have a simple square shape and I want to create a bounding shape for it. I can use any type of shape I want, but clearly the choice will determine the efficiency of the algorithm. In the below picture you see three different possibilities (these are the more common choices):

The first is a bounding circle. This bounding shape has very quick collision detection - compare the squared distance between the two objects against the sum of their radii. The downside is that long shapes create huge bounding circles. If you could guarantee that your shapes were all compact, this might be a good choice, but not for a generic collision detection system that has no idea what objects will look like.

The second is an Axis-Aligned Bounding Box, AABB for short. It’s a rectangle that’s always aligned to the x and y axes. This solves the problem that the bounding circle had where long objects would create large bounding shapes. However, what happens if that long object is rotated about it’s center - it’s the same issue as the circle bounding shape. AABB does better, but maybe not the best. To test for collision between two AABBs we only need to compare their min/max values - 4 comparisons.

The last is an Oriented Bound Box, or OBB for short. It’s a rectangle that’s always oriented with the shape. If the shape is rotated at 45 degress, then the bounding rectangle is rotated at 45 degrees. Nice! This solves our problem of rotated long objects generating huge bounding shapes. Not so fast - there’s a problem with this approach. To check for collision between two OOBs you have to use more sophisticated routines. In fact, one of those routines is effectively a narrowphase algorithm - we don’t get any benefit out of a broadphase that’s as expensive as the narrowphase.

You may have noticed that the bounding shapes in the image above didn’t exactly fit the object. This was mainly to show the differences in the bounding area, but as we’ll see soon, this is actually one of the sources of the performance enhancement being described in the post.

Having an easier collision detection proceedure isn’t all the broadphase does - we still need to contend with that \(O(n^2)\) problem. We could brute force it by testing everything against everything, but that’ll be horribly inefficient. What can we do? The answer is to subdivide the problem into smaller and smaller chunks so that when we test the higher level chunk and it’s negative, we move on - there’s no reason to do any tests against objects inside that chunk. The image below highlights this hierarchy of chunks:

For example, if I’m testing an object’s AABB against the purple region in the image above and I detect no collision, there’s no reason to test the interior regions since they can’t possibly be colliding if the purple region isn’t. If we could build a data structure that would organize the scene in this fashion we could reduce the number of collision tests significantly. For example, if we could organize the scene into a binary tree structure, the number of tests would then be \(O(n\log_2{n})\).

There are other partitioning schemes as well: grid-based, quadtree, BSPs, etc.

# Bounding shape Expansion

Creating a data structure to do less collision tests isn’t a trivial task. In addition to the creation of the initial state, we also have to keep it up-to-date as those objects move and rotate over time. It must be efficient at updating so that we don’t trade one performance issue for another. To avoid going down the rabbit hole of describing various data structures that can be used in broadphase collision detection, let’s just assume that you have one and updates to that data structure aren’t cheap.

Enter bounding shape exapansion. If you recall from the section above where we laid out a couple of options for bounding shapes, you may have noticed that the bounding shapes for the square didn’t fit around it tightly. This was primarily to highlight the different bounding shapes, but also to draw attention to the fact that they don’t have to fit tightly. Imagine I expand the bounding shape for a square and put that expanded bounding shape into the broadphase. The effect is that when the square moves/rotates, I don’t have to update the expanded bounding shape unless it leaves that expanded region.

In \(T = 2\), the square only moved one unit to the right and is still fully contained in the bounding AABB. There’s no reason to update the bounding shape because it would not effect the end result.

Let’s review what we’re trying to do here - we want to send less pairs to the narrowphase, but we also want to reduce the number of updates to our data structure. Sending less pairs to the narrowphase means the expensive collision detection there is run less often. Less updates to our broadphase data structure means we do less work per frame. If we choose tighly fitting bounding shapes we get the smallest number of collision tests in the broadphase, but for every one of those objects that’s moving/rotating, the data structure will need to be updated every frame. If we choose super fat bounding shapes we end up doing more collision tests in the broadphase, but we don’t have to update the broadphase data structure as often (as described above).

The key then is to strike a balance between expanding a lot and not expanding at all. This is a configurable property in dyn4j.

# The New Optimization

At this point, we have a broadphase collision detection algorithm that uses a data structure to avoid the \(O(n^2)\) number of collision tests, a collision detection test that is far quicker than the narrowphase, and we’ve chosen an appropriate bounding shape expansion to avoid rebuilding te data structure every frame. What more can be done?

To take a step back, think about how a simulation evolves over time:

• Detect collisions
• Resolve collisions
• Move objects to new positions

Or something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
public void runSimulation() {
    while (true) {
        simulationStep();
    }
}

public void simulationStep() {
    // get ALL broadphase collision pairs
    List<Pair> pairs = broadphase.getPairs();

    // filter broadphase pairs using the narrowphase
    Iterator<Pair> iterator = pairs.iterator();
    while (iterator.hasNext()) {
        Pair pair = iterator.next();
        if (!narrowphase.detect(pair)) {
            // then remove it
            iterator.remove();
        }
    }

    // resolve collisions
    // ...

    // move objects to new positions
    // ...
}

This flow would be called for every frame of the simulation. Thus, for every frame we have to retest everything with everything. this seems like a lot of work, but it makes sense - we need to make sure that new collisions are detected and old collisions are no longer handled. The magic for the improved performance in dyn4j 4.0.x lies in this process.

Imagine that we have the following configuration on frame 1 of our simulation:

We have three squares with expanded bounding shapes (AABBs in this case). The purple and teal expanded bounding shapes are overlapping. The orange object’s expanded bounding shape is offset because the orange object moved last frame. Also imagine the purple object is moving as well, but the teal object is stationary. In this setup we have 3 objects and therefore 3 collision tests to do in the worst case (brute-force). Purple vs. Teal, Purple vs. Orange, Teal vs. Orange.

We don’t test Purple vs. Teal and Teal vs. Purple as this would give the same result.

Next, imagine the simulation proceeds to frame 2:

The purple and orange object have moved two units to the left while the teal object stayed stationary. Notice that the purple object’s expanded bounding shape remained the same, but the orange object’s expanded bounding shape changed. Now, think critically here - from the last frame to this frame, what do we really need to test? Everything again?

You’ve probably already guessed it, but no we don’t. The collision between the purple and teal objects hasn’t changed - no reason to test it again. That said, the orange object’s expanded bounding shape did change, so we have to test that. That means in frame 2 we only do 2 collision tests! Orange vs. Teal and Orange vs. Purple. Therefore, the performance enhancement we can make is to track, over time, the collisions between objects in the broadphase and only re-test those that have had their expanded bounding shapes updated.

KEY: Therefore, the performance enhancement we can make is to track, over time, the collisions between objects in the broadphase and only re-test those that have had their expanded bounding shapes updated.

Enhancing our code from before:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// ...
List<Pair> pairs = new ArrayList<Pair>();

public void simulationStep() {
    // get only the NEW broadphase collision pairs
    List<Pair> bpairs = broadphase.getNewPairs();

    // filter broadphase pairs using the narrowphase
    for (Pair pair : bpairs) {
        if (narrowphase.detect(pair)) {
            // then track it in our cross frame set of pairs
            pairs.add(pair);
        }
    }
    // ...
}

Ok, great - this is cool and all, but how do we handle frame 3 of the simulation:

In this frame, we see that the teal and orange object’s bounding shapes have not moved, but the purple object’s has. As a result, we only test Purple vs. Teal and Purple vs. Orange and find neither is colliding. If the contract for our broadphase is to only return potentially colliding pairs, it would return nothing. But how would we know that the collision between Purple and Teal is no longer happening?

It turns out the answer is pretty simple, as you go through ALL the stored pairs, remove those who’s bounding shapes are no longer colliding:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
// ...
List<Pair> pairs = new ArrayList<Pair>();

public void simulationStep() {
    // get only the NEW broadphase collision pairs
    List<Pair> bpairs = broadphase.getNewPairs();
    
    // add them to the set of pairs from the last frame
    // obviously you'll need to make sure you don't add
    // a pair that already exists in the pair list from 
    // last frame - I'm not doing that here for simplicity
    for (Pair pair : bpairs) {
        pairs.add(pair);
    }

    // filter out those that are no longer colliding
    Iterator<Pair> allIterator = pairs.iterator();
    while (allIterator.hasNext()) {
        Pair pair = allIterator.next();
        if (!pair.isBoundingShapeOverlapping()) {
            // then remove it
            allIterator.remove();
            continue;
        }

        // filter broadphase pairs using the narrowphase
        if (narrowphase.detect(pair)) {
            // then continue processing the collision...
        }
    }
}

You might say at this point, “wait a minute, we’re doing collision tests with the expanded bounding shapes again - isn’t that even worse?” It turns out no, the key being that we’re not doing it for ALL pairs, only those that are currently tracked as potentially overlapping. We’ve changed the algorithm from \(O(n\log_2{n})\), for example, to \(O(m\log_2{n})\), where \(m\) is the number of objects who’s expanded AABBs have been updated.

# Evaluation

Let’s evaluate the improvement with some numbers and assumptions. Assume we have a scene with 5000 objects. Of those 5000 objects, 500 are moving every frame. Of those 500 moving objects, 50 of them need to update their expanded shapes every frame. Assume that of all 5000 objects we have 1000 collisions. Finally, assume that our broadphase has complexity \(n\log_2{n}\) to do all tests.

The \(+ 1000\) in the above forumlas account for re-testing of overlap to account for objects moving out of collision (as described in the Frame 3 discussion in previous section).

From scenario 1 to 3 that’s nearly a 100% decrease in the number of tests - wow! Clearly, the numbers I chose were to highlight the improvement, but the gains will be highly dependent on your simulation. You could think of scenario 2 being the worst case for this enhancement and scenario 3 being the best case. These numbers are also dependent on the value used to expand the bounding shapes and the ratio of static vs. moving shapes in your scene. If all 5000 objects were moving and we chose an expansion value that forced an update for all objects every frame, then we’re worse off than where we started. Also keep in mind that the storage and maintenance of the frame to frame collision data has overhead.

dyn4j saw anywhere between 30-40% raw performance increase implementing this simple enhancement (measuring the same simulations before/after the enhancement)

A positive side effect of tracking the collisions from frame to frame is that we can reduce allocation by storing the results and reusing those objects for the next frame. In addition, we can build new API surface that enables callers to query this information outside of the standard listener pattern giving them more opportunity to do things in a simpler manner.

To start, I wanted the site to be something like what I had before - I liked the layout of the landing page and blog portion of the site. If you were less concerned about what the site looked like you could install an off-the-shelf layout/theme. GitHub Pages actually has a few to choose from to give you a kickstart. I also wanted it to be fast and low maintenance which meant using CDNs for any linked libraries, the number of linked libraries should be low, and use of liquid templates to compartmentalize components for reuse. Probably not surprising, as the move progressed I found issues, enforced new constraints, and had to learn a lot.

Full disclosure - I’ve been a web developer for 15 years so a lot of the basic concepts are familiar to me - Jekyll, Ruby, Liquid templates, etc. were all new to me. I was already familiar with markdown as well.

The dyn4j.org website is a public repo on GitHub. Reference the source for an additional resource when going through your conversion.

# Getting Started

To get started you’ll need to install the jekyll tooling which includes ruby. For development of the site I went with Visual Studio Code which has good support for markdown, yml, css, sass, JavaScript, and so on. I ended up using all of these in some way and having syntax highlighting was really important.

After that I went through the step by step tutorial to get a better understanding of the platform. I’ll confess, I didn’t go through all of it right away, I focused mostly on the inital build of the site, index.html, includes, layouts, and front matter.

The key with Jekyll and GitHub pages is that it’s a framework for building static web sites. You build out your content, pages, JavaScript, etc. and Jekyll combines everything to make static html pages. This is what makes Jekyll sites really fast.

I also followed the GitHub guides to get a repo setup properly and cloned it locally.

Refer to the GitHub Page documentation to understand what things are not supported. For example, not all Jekyll plugins are supported.

# Build the Skeleton

My first goal was to build the basic skeleton of the site and the landing page. I wanted to choose an existing template, but they all seemed OK - nothing really wrong with them, just not what I wanted so I decided to download a few example jekyll themes: (no-style-please and creative theme) as references. I also decided on Bootstrap 5 for the basic UI components and then set after it.

This stage quickly devolved into fixing all the things I didn’t like about the old site and feverishly testing on all different screen sizes. I spent a lot of time here. I cleaned up verbiage, images, links, pretty much everything. I’m happy I spent the time there, but it was a huge time suck.

For the animation I used AOS since it had the capability to wait for the user to scroll before animating. I also included the MathJax library since I have a few blog posts that are math heavy.

So at this point I had the landing page done and a decent pattern to follow for the rest of the site.

# Build the Blog

The next step was to get the blog portion of the site going. For that I referenced the no-style-please theme to understand how to display them, where to put the posts, and so on. I made it my own and modeled it after the existing site with a tag cloud, grouped by year, and category links sections. There are some limitations here that I didn’t really like, but it was a small price to pay. For example, to create a link to a specific tag or category - not possible - instead you can create a page with all your posts grouped by tag/category and link to that page with an anchor to the correct location on the page. Not huge deal - no one reads these anyway do they? The other thing that wasn’t straight forward was the paging for the main blog landing page - I didn’t want a hundred blog posts to all be on the same page that the user lands on.

I built it out while using the same posts from the no-style-please theme first to get all the includes and layouts the way I wanted them.

# Moving the Blog Posts

For this step, I needed something to export all the posts from the WordPress site and convert them to Jekyll markdown. For that I installed the Jekyll Exporter plugin into my WordPress site. A couple of good things about this is that it also exported pages and assets (images). The conversion was pretty good too.

That said there were some problems. The first was the date front matter data point. It was in the format ISO format that you’d expect, but Jekyll needs the date formatted like YYYY-MM-DD HH:MM:SS +/-TTTT. This only matters if you want your posts sorted properly… (lot’s of eyerolling ensued). The second problem was that the exported/converted posts left a lot of HTML in them and I really wanted to move to entirely markdown. I understand why it couldn’t convert those parts, they were complex HTML code, so I decided to roll my sleeves up and update all the posts. This was very time consuming and I was very VERY bored. I decided while I was modifying each post to review them as well. This, while still boring, was very satisfying. For example, going back through the constrained dynamics posts, I found a number of minor mistakes and added more clarifying content. I also decided to change all the hard coded images of math to Latex for better maintainability. Another issue was that the exported content didn’t properly change the path to images and those needed to be updated and migrated to the site’s assets folder manually. There are couple of things you can do here to make it easier to references images. I also reviewed all links and cleaned those up.

TIP: The date format from the Jekyll Exporter does not match what Jekyll is looking for. That said, as far as I can tell, GitHub Pages doesn’t care what I put in for the timezone offset - it just completely ignores it and considers the datetime is in UTC…

All in all, I spent WAY too much time here. I’m happy with the end result, but can’t imagine what I would have done if I had more than 60 posts (and most of my posts were short announcements for dyn4j releases - fixing the larger posts was brutal).

TIP: There’s a gotcha here. The Jekyll Exporter tool exports your posts as markdown files with front matter. as part of the front matter there’s an id field which I’m assuming was the original id from WordPress. However, when you try to do post.id you will get the slugified name of the post - the id property of the post object cannot be overridden.

# Moving the Comments

I spent a lot of time thinking about whether to move comments or not. GitHub Pages, being a static site host, meant that I wouldn’t be able to accept more comments unless I developed a different solution. I had around 500 comments total - all moderated so they were quality. I decided to move them over and make the decision whether to accept new comments later. Jekyll has a way to generate static content from data. This is the mechanism we’ll use for comments.

To save you some time, I probably went the long way. What I describe below is what I did. After all that work I found this. Somehow I overlooked this, so I don’t know what the output is like, but I’d probably start with that tool if I were to do it over again.

Moving the comments was a lot harder. There wasn’t a nice exporter like there was for posts so I decided to just export them myself using the WordPress APIs. To do so, I signed into WordPress admin and executed the following script(s):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
let promises = [];

promises.push(fetch('/wp-json/wp/v2/posts?per_page=100&page=1').then(function(response){
    return response.json().then(function(data) {
        return data.map(function(item) {
            return {
                id: item.id,
                guid: item.guid
            };
        });
    });
}));

for (var i = 1; i < 7; i++) {
    promises.push(fetch('/wp-json/wp/v2/comments?per_page=100&page=' + i).then(function(response) {
        return response.json().then(function(data) {
            return data.map(function(e) {
                return {
                    id: e.id,
                    post: e.post,
                    parent: e.parent,
                    date: e.date_gmt,
                    content: e.content.rendered,
                    author: e.author_name,
                    avatars: e.author_avatar_urls
                };
            });
        });
    }));
}

Promise.all(promises).then(function (dataSets) {
    let posts = dataSets[0];
    var postsById = {};
    for (var i = 0; i < posts.length; i++) {
        postsById[posts[i].id] = posts[i].guid.rendered;
    }

    let n = dataSets.length;
    let result = [];
    for (var i = 1; i < n; i++) {
        result = result.concat(dataSets[i].map(function(item) {
            item.post_guid = postsById[item.post];
            return item;
        }));
    }
    console.log(JSON.stringify(result));
});

The WordPress APIs have a limit of 100 records per page.

You’ll need to tweak it for your scenario. For example, I knew how many posts and comments I had so I just hard coded the number of API calls I needed to execute. The script generated a nice JSON string in the console which I just copied and pasted into a file.

Not shown here, I also exported all posts to JSON in the same way for the next step.

Jekyll acutally supports a few different formats for “data”. One of which is JSON and at first I used the JSON generated in the previous step to feed the posts with the appropriate comments (by filtering them on the post_guid property). But after making the decision to implement a solution for new comments, I decided to change how I was storing the comments. I took this JSON file, read it into memory with a C# console application and converted it into a list of yml files, placed in folders named with the slugified name of the parent post (more on the why later). Here’s the code for that:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
class Program
{
    static void Main(string[] args)
    {
        string json = File.ReadAllText($"posts.json");
        List<Post> posts = JsonConvert.DeserializeObject<List<Post>>(json);
        Dictionary<int, Post> postMap = posts.ToDictionary(p => p.Id);

        json = File.ReadAllText($"comments.json");
        List<Comment> comments = JsonConvert.DeserializeObject<List<Comment>>(cj);

        var converter = new ReverseMarkdown.Converter();
        string baseDir = Directory.GetCurrentDirectory();

        foreach (Comment comment in comments)
        {
            Post post = postMap[comment.Post];
            string path = Path.Combine(baseDir, post.Slug);
            Directory.CreateDirectory(path);

            string avatarUrl = comment.Avatars.OrderBy(a => a.Key).LastOrDefault(a => !string.IsNullOrEmpty(a.Value)).Value;
            string avatarProp = "";
            if (!string.IsNullOrEmpty(avatarUrl))
            {
                avatarProp = $"\navatar: {avatarUrl}";
            }

            // my wordpress content had all kinds of unicode characters I wanted to remove
            string message = comment.Content.Trim()
                .Replace("&#8220;", "\"")
                .Replace("&#8221;", "\"")
                .Replace("&#8217;", "'")
                .Replace("&#8230;", "...")
                .Replace("&#8211;", "-")
                .Replace("\\*", "*")
                .Replace("\"", "\\\"")
                .Replace("“", "'")
                .Replace("”", "'")
                .Replace("&gt;", ">")
                .Replace("&lt;", "<")
                .Replace("\n", "\\n");

            // match the date format Jekyll wants
            string date = comment.Date.ToString("yyyy-MM-dd HH:mm:ss zz00");

            string content = $"id: {comment.Id}\ndate: {date}\nauthor: {comment.Author}\nparent: {post.Slug}{avatarProp}\nmessage: \"{message}\"";
            File.WriteAllText(Path.Combine(path, $"{comment.Id}.yml"), content);
        }
    }
}

I then copied+pasted the folders/files to the data/comments folder in my Jekyll site and updated my templates, includes, layouts to use these. I based my templates off of these. The import of these had some problems - Jekyll warned about a couple of things and I just manually fixed those.

One interesting thing here is that I wanted to convert these to markdown too. I tried to use the ReverseMarkdown NuGet package, but the generated markdown didn’t render properly, so I just left the comments as is.

I also considered going through all the comments and fixing links, images, formatting, etc. and quickly gave up on that idea. The output was good enough so I just let it all as is - there are just too many to go through.

# Moving Pages

In WordPress you can create pages as well as posts/comments. In dyn4j, there were only a few and these were converted via the Jekyll Exporter tool mentioned earlier, but just like posts, they still had a lot of HTML embeded in them. Another issue is that they were horribly out of date. So I decided to build these mostly from the ground up. I used the existing pages for a good outline for content, but pretty much rewrote all of them in markdown. I think the result is so much better - up to date documentation, more code samples, better references to classes in the library and so on. I was also able to consolidate some pages and skip the conversion entirely. I converted 4 pages of 5.

# Code Highlighting

One thing I spent a lot of time on was code highlighting. It was very hard to understand how this works. Basically, you write your code comments in single or triple tick marks and Jekyll parses the content and emits HTML ready for highlighting (using Rouge). But to actually highlight the code you must add in a CSS file. You can find a bunch of options. The process is described in detail here, but the key that I kept missing was that you run a command to generate the CSS of choice, then you just include that in your <head>.

# Custom Domain

No what’s really cool about all this is that you can configure GitHub Pages with a custom domain. You can even enable HTTPS! I decided at the start of this process that I would do this, but it turned out to be really frustrating for a number of reasons. The first reason is due to the way DNS works - you change something and everything is broken for a while (a hour for example) until the change propagates. I think in total dyn4j.org was probably down for 10 hours while I fiddled with this. The second reason was due to my host through which I purchased the domain long ago - their tools are just so buggy. Pages not loading, not allowing me to do things but not telling me why, etc. A lot of this was trial and error.

I just realized I went on a rant and you don’t care - you just want to know what to do. The first thing you should do is read the official GitHub Pages documentation and follow that. It really is good documentation.

The first step is to configure your GitHub repo with the custom domain name. It won’t work, but that’s ok. You need to do this first so that another repo doesn’t steal your domain when you finally point your domain at GitHub IPs. Once that’s complete, then you need to log into the host for your domain and update DNS. I followed exactly what was in the GitHub Pages documentation and it nearly worked. For my host, I had to create A records (step 7 of the official docs) but I also had to create A records using “@” as the source. I also had to use the apex domain in GitHub pages, NOT the www prefixed domain.

TIP: If you want both the apex domain and www subdomain to work with HTTPS, set the repo custom domain to the apex domain. For example, if you own example.com, set the repo’s custom domain to example.com. Then setup the CNAME and A records as defined in the docs.

TIP: Every host may have different ways to setup DNS. In my case I had to add 4 more A records using “@” instead of my apex domain pointing to GitHub IPs.

TIP: Be patient and expect your site to be down for a while. DNS takes time to propagate and you want to make sure you are testing accurately. After making changes in DNS, give it at least an hour (that was the default TTL on my host).

See here for more help with custom domain setup.

This was the final setup (not sure if I needed both the @ and dyn4j.org A records, but I was pretty frustrated at this point and decided to leave it alone since it was working)

# Accepting New Comments

Like I’ve mentioned before, the whole point of Jekyll is to produce a lightning fast static site generated from non-static data (like posts, comments, pages, templates, etc.) So how can we accept new comments? It was a great question and I found a solution. It’s actually quite straight forward:

• Create a comment form that will post data to an Azure Function
• Setup an Azure account ($200 free credit)
• Create an Azure Function to accept comments
• Add the necessary code to the function to create a branch on your site repo, create a comment yml file, and submit a pull request

You may recall from above that I left the “why” out when I was discussing moving away from a giant JSON file for comments - this is it. If you have a giant JSON file all PRs will modify that file and merging could become really troublesome. Instead, if you break out each comment into it’s own file, then each comment is independent and you can approve PRs in any order without conflicts.

It’s by no means a perfect solution, but it actually turns out to be really nice. A commenter submits the form, you get it in PR form where you can modify it if needed, you can use the PR as a moderation avenue (be sure to manually delete the source branch). When you approve the PR and it get’s merged, GitHub automatically regenerates your site and deploys it and the comment is now visible! Cool!

TIP: There’s a nice cost calculator for Azure you can use to project cost. For something like dyn4j.org I calculated the cost at less than $1/month and probably $0 for most months.

Here’s the function code, adapted from the link above for .NET Core 3.1:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
public static class JekyllBlogCommentsAzureFunction
{
    struct MissingRequiredValue { } // Placeholder for missing required form values
    static readonly Regex validPathChars = new Regex(@"[^a-zA-Z0-9-]"); // Valid characters when mapping from the blog post slug to a file path
    static readonly Regex validEmail = new Regex(@"^[^@\s]+@[^@\s]+\.[^@\s]+$"); // Simplest form of email validation

    [FunctionName("PostComment")]
    public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Anonymous, "post")] HttpRequestMessage request, ILogger logger, ExecutionContext context)
    {
        var form = await request.Content.ReadAsFormDataAsync();

        // Make sure the site posting the comment is the correct site.
        var commentSite = Environment.GetEnvironmentVariable("CommentWebsiteUrl");
        if (!string.IsNullOrEmpty(commentSite) && !AreSameSites(commentSite, form["comment-site"]))
        {
            logger.LogInformation($"{commentSite} and {form["comment-site"]} do not match");
            return request.CreateErrorResponse(HttpStatusCode.BadRequest, "Please make sure you post this to your own Jekyll comments receiever.");
        }

        if (TryCreateCommentFromForm(form, out var comment, out var errors))
            await CreateCommentAsPullRequest(comment);

        if (errors.Any())
            return request.CreateErrorResponse(HttpStatusCode.BadRequest, String.Join("\n", errors));

        if (!Uri.TryCreate(form["redirect"], UriKind.Absolute, out var redirectUri))
            return request.CreateResponse(HttpStatusCode.OK);

        var response = request.CreateResponse(HttpStatusCode.Redirect);
        response.Headers.Location = redirectUri;
        return response;
    }

    private static bool AreSameSites(string commentSite, string postedCommentSite)
    {
        return Uri.TryCreate(commentSite, UriKind.Absolute, out var commentSiteUri)
            && Uri.TryCreate(postedCommentSite, UriKind.Absolute, out var postedCommentSiteUri)
            && commentSiteUri.Host.Equals(postedCommentSiteUri.Host, StringComparison.OrdinalIgnoreCase);
    }

    private static async Task<PullRequest> CreateCommentAsPullRequest(Comment comment)
    {
        // Create the Octokit client
        var github = new GitHubClient(new ProductHeaderValue("PostCommentToPullRequest"),
            new Octokit.Internal.InMemoryCredentialStore(new Credentials(Environment.GetEnvironmentVariable("GitHubToken"))));

        // Get a reference to our GitHub repository
        var repoOwnerName = Environment.GetEnvironmentVariable("PullRequestRepository").Split('/');
        var repo = await github.Repository.Get(repoOwnerName[0], repoOwnerName[1]);

        // Create a new branch from the default branch
        var defaultBranch = await github.Repository.Branch.Get(repo.Id, repo.DefaultBranch);
        var newBranch = await github.Git.Reference.Create(repo.Id, new NewReference($"refs/heads/comment-{comment.id}", defaultBranch.Commit.Sha));

        // Create a new file with the comments in it
        var fileRequest = new CreateFileRequest($"Comment by {comment.author} on {comment.post_id}", new SerializerBuilder().Build().Serialize(comment), newBranch.Ref)
        {
            Committer = new Committer(comment.author, comment.email ?? Environment.GetEnvironmentVariable("CommentFallbackCommitEmail") ?? "redacted@example.com", comment.date)
        };
        await github.Repository.Content.CreateFile(repo.Id, $"_data/comments/{comment.post_id}/{comment.id}.yml", fileRequest);

        // Create a pull request for the new branch and file
        return await github.Repository.PullRequest.Create(repo.Id, new NewPullRequest(fileRequest.Message, newBranch.Ref, defaultBranch.Name)
        {
            Body = $"avatar: <img src=\"{comment.avatar}\" width=\"64\" height=\"64\" />\n\n{comment.message}"
        });
    }

    private static object ConvertParameter(string parameter, Type targetType)
    {
        return String.IsNullOrWhiteSpace(parameter)
            ? null
            : TypeDescriptor.GetConverter(targetType).ConvertFrom(parameter);
    }

    /// <summary>
    /// Try to create a Comment from the form.  Each Comment constructor argument will be name-matched
    /// against values in the form. Each non-optional arguments (those that don't have a default value)
    /// not supplied will cause an error in the list of errors and prevent the Comment from being created.
    /// </summary>
    /// <param name="form">Incoming form submission as a <see cref="NameValueCollection"/>.</param>
    /// <param name="comment">Created <see cref="Comment"/> if no errors occurred.</param>
    /// <param name="errors">A list containing any potential validation errors.</param>
    /// <returns>True if the Comment was able to be created, false if validation errors occurred.</returns>
    private static bool TryCreateCommentFromForm(NameValueCollection form, out Comment comment, out List<string> errors)
    {
        var constructor = typeof(Comment).GetConstructors()[0];
        var values = constructor.GetParameters()
            .ToDictionary(
                p => p.Name,
                p => ConvertParameter(form[p.Name], p.ParameterType) ?? (p.HasDefaultValue ? p.DefaultValue : new MissingRequiredValue())
            );

        errors = values.Where(p => p.Value is MissingRequiredValue).Select(p => $"Form value missing for {p.Key}").ToList();
        if (values["email"] is string s && !validEmail.IsMatch(s))
            errors.Add("email not in correct format");

        comment = errors.Any() ? null : (Comment)constructor.Invoke(values.Values.ToArray());
        return !errors.Any();
    }

    /// <summary>
    /// Represents a Comment to be written to the repository in YML format.
    /// </summary>
    private class Comment
    {
        public Comment(string post_id, string message, string author, string email = null, Uri url = null, string avatar = null)
        {
            this.post_id = validPathChars.Replace(post_id, "-");
            this.message = message;
            this.author = author;
            this.email = email;
            this.url = url;

            date = DateTime.UtcNow;
            id = new { this.post_id, this.author, this.message, this.date }.GetHashCode().ToString("x8");
            if (Uri.TryCreate(avatar, UriKind.Absolute, out Uri avatarUrl))
                this.avatar = avatarUrl;
        }

        [YamlIgnore]
        public string post_id { get; }

        public string id { get; }
        public DateTime date { get; }
        public string author { get; }
        [YamlIgnore]
        public string email { get; }

        [YamlMember(typeof(string))]
        public Uri avatar { get; }

        [YamlMember(typeof(string))]
        public Uri url { get; }

        public string message { get; }
    }
}

Now, you do lose spam filtering, but like the source link mentioned, a simple “Are you sure?” dialog seems to do the trick and like previously mentioned you have the moderation via PRs anyway. I really wanted to put in place some networking restrictions on the Azure side, but you can’t. The comment form is submitted from the client and it goes directly to Azure.

Another cool thing I saw being done on the comment form is the automatic generation of avatar URLs for GitHub, Gravitar, and Twitter. I’m not sure what their solutions were, but mine was just a simple (vanilla) JavaScript file that takes the username field and generates an avatar URL based on the format. I’d forgotten how annoying it is to write vanilla JavaScript…

# Summary

While the conversion took more effort than I was expecting, I’ve very happy with the results. It actually forced me to go back and revise pages and posts that were in need of some updates. It also allowes me to maintain the site entirely out of GitHub and drop the hosting I was paying for as well. Other benefits include speed and full control of the UI. All this while retaining the capability to blog and receive comments.

To be clear, with dyn4j.org I was looking for both a nice project website to highlight the maturity that I strive for in library and to serve as a blogging platform for some of the difficult features of the library.

# Thorn #1: Hosting

Hosting of WordPress should be easy and straight forward. When I first took the deep dive into the platform, it was. My host has nice GUI tools to install it and get a domain pointed to it - great! Overtime, however, is where things went down hill.

The real nail in the coffin for this move, was more a host problem than a WordPress problem. They make no attempts to keep you off of aging infrastructure and then charge you to move to equivalent, better supported infrastructure. In my particular case, I’d just renewed for a few years when I realized that my hosting was on some ancient infrastructure that was stuck on PHP 5.x or something and that I’d have to renew again to move. What?! I couldn’t believe it. The whole reason I found out about this was because WordPress was asking me to upgrade, but that upgrade required a newer version of PHP.

Another thing to mention is the ubiquity of more robust solutions and hosting (read: cloud) out there these days. For example, in my day job I frequently use the Microsoft stack, deploying to Azure - their tools are really nice to work with. Want to move to .NET Core 3.1? No problem, create a new App Service Plan and move everything over - no need to deal with support.

# Thorn #2: Upgrades

It’s probably already clear by now, but upgrades were also a nusiance. Tons of warnings about “Take a backup because everything could be broken after this” never gave me much hope, but generally speaking it worked well. The issue is that WordPress has moved on past the PHP version supported the infrastructure I was on. Sticking around on this old infrastructure and old PHP version just gave me the chills.

# Thorn #3: Security

One really strange issue that has occurred, on a number of occasions, has been where someone modified (still not sure how) my .htaccess files and redirected incoming traffic to lovely pharmaceutical sites. As a maintainer, you’d never notice unless you google-searched a page for the site and tried to click the google result - it only redirected in those scenarios. Needless to say, that was very frustrating and I spent A LOT of time hardening my security setup (changed passwords, added MFA, changed FTP usernames and passwords, etc.). This isn’t how I want to be spending my evenings.

# Thorn #4: Security

Yet another cost if you host yourself is SSL. It’s not like dyn4j.org needs it for anything (no there aren’t any custom made t-shirts or hats for you to buy), but there’s been a lot of talk from browsers about showing nasty messages when you navigate to a site without HTTPS. Paying for this just so a browser doesn’t tag my site as a scam, just seemed like an internet tax.

# Thorn #5: Customizability

WordPress is super customizable, has tons of OOB themes, free themes, paid themes, plugins, etc. It really is a nice ecosystem. But this strength is also a great weakness - I really don’t want to spend my precious free time trying to understand the internals of WordPress just so I can customize a page or two. This made it feel very rigid - you either stuck with the plugins/themes you have installed or you learn WordPress development - and I feel like this made it really stagnant.

# Thorn #6: Performance

This is more a knock on my hosting provider - it just always felt slow for what was really a static site with a few infrequent blog posts. I was on the basic of basic plans to reduce cost, but even then there really isn’t that much going on with this site to warrant the lack of performance.

# Replacement Decision

I just kept asking myself, “Why do I keep paying for (putting up with) this…?” The only answer I could think of was a lack of time to make the move. I finally decided, after my hosting provider told me, “You need to pay up if you want new infrastructure” that now was the time to move - even if I my sleep would suffer.

Research initially focused on a replacement host, but the thought of doing the same thing, just on a different host, made me start searching for alternatives. It was then that I discovered GitHub Pages, albeit much later than the rest of the internet it seems…

GitHub Pages offers a really neat way of hosting a site. You build your site using a public repo and GitHub will use whatever branch you choose to serve it up. There are two primary offerings, a static site built however you want, and a Jekyll based solution.

The key concern, in either offering, was that the site is effectively static - i.e. no user interaction or other server components would be possible. Thinking about the dyn4j.org use-case though, the only user-interaction “needed” (though GitHub Discussions might be an interesting alternative) was post commenting. At first, I considered whether I needed commenting at all, but as a I went through the exercise of designing the new site, I found a really cool work around. At this point I was pretty much sold, GitHub Pages would allow me to do what I’m already doing but would solve all of my thorns:

• Hosting - GitHub is the host and it’s free! Give some money to a charity or your choosing!
• Upgrades - You can build whatever you want and it’s all static - you only need to support browser capabilities which have an incredible history of backwards compatibility.
• Security - GitHub doesn’t host anything, they only host the repo’s statically generated content.
• SSL - GitHub will issue a cert for you - unbelievable…
• Customizability - You can build a site with HTML, CSS, and JavaScript or go down the Jekyll path. If you’ve done web development in your career it’ll be nothing. Jekyll did have a bit of a learning curve, but nothing like WordPress.
• Performance - GitHub only publishes a static site and it is lightning fast. I can’t even tell you how much more professional a site looks when it’s fast.

There is one caveat here with Upgrades when using Jekyll. While GitHub pages supports Jekyll natively, it only supports a limited subset of plugins and features. At the time of this writing I installed the latest version of the tooling and this worked fine, but only time will tell. You can avoid this risk, compromising some maintainability, by building a static site with just HTML, CSS, and JavaScript.

You can see the end result at dyn4j.org and it’s accompanying repo. I’ll do another post on the technical elements of the move and the various issues I ran into.

That said, it definitely has some breaking changes. All deprecated API has been removed as well.

See the release notes for all the detail.