DevBlog: An Engine story (tech post)

Hey guys,

I wanted to drop a little blog post to share with you what's going on this month, why it's happening and what are the short and long term implications. This post is going to focus exclusively on technical stuff.

So - as some of you may know, Transylvania is created by using the Twine tool (https://twinery.org/) and more specifically the Harlowe story format. You can think of Twine as the editor and Harlowe as the engine (this will be important in a moment).

One of the main challenges that I've been facing development-wise for the last 2 years is that Transylvania has grown beyond the intended scope of the Twine editor and in its current state of almost 2000 passages, there is some considerable lag while working on the project. For some time I've been pestering the creator of Twine (https://www.patreon.com/klembot ) to improve performance on larger projects, and finally this has given results. Twine 2.4 was released earlier this month, and it features lots of very substantial changes to the editor and pretty much a core re-writing of the entire tool. This was not done only for the purpose above of course, but one of the consequences of the new version is much improved handling of project scope.

So far so good. In a completely separate and parallel line, the Harlowe format also received a very major update, coming with a great deal of changes. These are not simple additions of new functions, but core modification to the engine.

Coincidentally, the new version of Twine, also comes with the new version of Harlowe. Naturally, I was very excited to update and see how Transylvania is handling in a brand new editor with a brand new engine version.

And then everything broke.

In a situation like this with so many variables changing at once, it can be difficult to pin-point where an issue is coming from. There were some relatively easy things where code simply needed to be updated to conform to the new version. On the flip side, there were also some very serious problems, since Transylvania is also using some third party extensions to enhance the experience.

Most notably, the sound stopped working, because the external Harlowe Audio Library the project is using was not compatible with the new version. Of course sound is not yet an officially released feature, but we are working on it in parallel, so having it break was a bit of a bother. Additional third party things that broke include the notifications system and the save import/export system, as well as passage transitions and some animations.

To avoid making this post exceedingly long, here's what happened with each of these systems:

Sound: I managed to rally other creators that were also using HAL and to bring the importance of the issue to Harlowe's creator, Leon. (post is here if you're interested: https://foss.heptapod.net/games/harlowe/-/issues/265 ) With the kind assistance of Leon as well as Chapel (HAL's creator), the issue was patched out and a new compatible version of HAL was released.

Notification system: This uses external JavaScript layered on top of Harlowe, and the way the two interact was changed in the recent release, which is why it broke. A fellow developer, Hituro, has been helping me to troubleshoot this and make the necessary changes to make it compatible again. There's considerable progress and success is on the horizon, but work remains to be done.

Save Export/Import: This was written by another creator, HiEv, and I'm not sure when  and if it will be upgraded for compatibility. I will try to pull some strings there but there's a high likelihood that this feature will need to be temporarily cut out to move forward.

Passage transitions and animations: I've reported the issue to Harlowe and a patch has been released today that fixes the problem.

This is not an exhaustive list, and there have been other minor things undergoing similar efforts of digging into, troubleshooting and reporting minor bugs or issues on the new versions of Twine and Harlowe. Overall everyone has been super responsive and I'm very thankful to be part of such a great open-source community.

-----------------

So where does this leave us: 

I do believe very strongly that the new version is the future and will bring some very long-needed improvements to the development process, as well as new tools and features to play around with and improve the game. At the same time this has been a particularly rocky update with lots of issues. I was hoping to completely transition the project to the new version this month, but as it stands it will not be the case. I have forked into 2 branches - one working on the old version and adding content for patch 0.3.15 at the end of the month, and one clone which has been migrated to the new version and issues are slowly being ironed out to make sure that there is full compatibility. If everything goes well, I will merge them in August and officially move on to the new versions of Twine and Harlowe.