Connor Clark's BlogWorking on Lighthouse, writing about web developer tooling.2022-04-29T00:00:00Zhttps://hoten.ccConnor ClarkDeceitful Java: How I Won A Free Thingy2016-06-12T00:00:00Zhttps://hoten.cc/blog/deceitful-java-how-i-won-a-free-thingy/<p>There are many stories about obsufcated code. Brilliant programmers have created programs that, at first glance (and second...and third...) seem absolutely harmless.<sup class="footnote-ref"><a href="https://hoten.cc/blog/deceitful-java-how-i-won-a-free-thingy/#fn1" id="fnref1">[1]</a></sup> But through some trick or another, be it compiler settings or little known language features, these programs deceive even the most meticulous code audits.</p>
<p>This is not one of those stories.</p>
<p>This code challenge came up during Mo' Code Movember,<sup class="footnote-ref"><a href="https://hoten.cc/blog/deceitful-java-how-i-won-a-free-thingy/#fn2" id="fnref2">[2]</a></sup> a small hackathon that took place in Nov. 2014. A Microsoft representative (shout out to Paul DeCarlo)<sup class="footnote-ref"><a href="https://hoten.cc/blog/deceitful-java-how-i-won-a-free-thingy/#fn3" id="fnref3">[3]</a></sup> had some goodies to give away. He came up with this challenge:</p>
<blockquote>
<p>Every participant will be assigned a unique number (from 1 to n, the number of participants). Each person must submit a program, in the language of their choice, that generates a random number from 1 to n. Whomever's number comes up the most wins. The catch - everyone can review your code. If your code is identified as anything other than a fair number generator, you get disqualified. Code reviewers cannot run the program. Incorrect accusers are also disqualified.</p>
</blockquote>
<p>So, with the problem at hand, and assigned the number 5, I decided to use Java. I fired up Netbeans and created a new project. This was my main file, and what I showed to my code reviewers:</p>
<pre class="language-java"><code class="language-java"><span class="token keyword">package</span> <span class="token namespace">random</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">public</span> <span class="token keyword">class</span> <span class="token class-name">Random</span> <span class="token punctuation">{</span><br /><br /> <span class="token keyword">public</span> <span class="token keyword">static</span> <span class="token keyword">void</span> <span class="token function">main</span><span class="token punctuation">(</span><span class="token class-name">String</span><span class="token punctuation">[</span><span class="token punctuation">]</span> args<span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">int</span> numPeople <span class="token operator">=</span> <span class="token number">7</span><span class="token punctuation">;</span> <span class="token comment">// n = 7</span><br /> <span class="token keyword">int</span> answer <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token keyword">int</span><span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token class-name">Math</span><span class="token punctuation">.</span><span class="token function">random</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">*</span> numPeople<span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token number">1</span><span class="token punctuation">;</span><br /> <span class="token class-name">System</span><span class="token punctuation">.</span>out<span class="token punctuation">.</span><span class="token function">println</span><span class="token punctuation">(</span>answer<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span></code></pre>
<p>No one challenged my code. When it came time to run my program, I was certain it would return 5.</p>
<p>If you're familiar with how Java's packaging system works, you probably already know how I made the deception. Within the same package as my main file (random), I had a class called Math. Math is already a builtin collection of static methods, and is imported by default. However, the Math in my random package gets higher priority than Java's built in Math class. As icing on top of the cake, Java automatically imports all files within the same package. All I had to do was write a faulty random() method, and be sure to collapse the "random" package in Netbean's project explorer.</p>
<pre class="language-java"><code class="language-java"><span class="token keyword">public</span> <span class="token keyword">class</span> <span class="token class-name">Math</span> <span class="token punctuation">{</span><br /><br /> <span class="token keyword">public</span> <span class="token keyword">static</span> <span class="token keyword">double</span> <span class="token function">random</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token number">4.0</span> <span class="token operator">/</span> <span class="token number">7</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span></code></pre>
<p>The contest results ended up something like this:</p>
<table>
<thead>
<tr>
<th>Number</th>
<th>Frequency</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>0</td>
</tr>
<tr>
<td>2</td>
<td>1</td>
</tr>
<tr>
<td>3</td>
<td>2</td>
</tr>
<tr>
<td>4</td>
<td>0</td>
</tr>
<tr>
<td>5</td>
<td>3</td>
</tr>
<tr>
<td>6</td>
<td>0</td>
</tr>
<tr>
<td>7</td>
<td>1</td>
</tr>
</tbody>
</table>
<p>With the contest rigged ever so slightly in my favor, I won the free thingy. The prize was a purple pair of Beats headphones, branded with Visual Studio's logo. They're not my favorite headphones, but I enjoy the convenience. They're a good fit for my laptop go-bag since they fold up nicely.</p>
<p><img src="https://hoten.cc/images/beats-vs.jpg" alt="My Free Thingy" /></p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p><a href="http://www.ioccc.org/" target="_blank" rel="noopener noreferrer">http://www.ioccc.org/</a> <a href="https://hoten.cc/blog/deceitful-java-how-i-won-a-free-thingy/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn2" class="footnote-item"><p><a href="https://github.com/CougarCS/mo-code-movember-2014" target="_blank" rel="noopener noreferrer">https://github.com/CougarCS/mo-code-movember-2014</a> <a href="https://hoten.cc/blog/deceitful-java-how-i-won-a-free-thingy/#fnref2" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn3" class="footnote-item"><p><a href="https://github.com/toolboc" target="_blank" rel="noopener noreferrer">https://github.com/toolboc</a> <a href="https://hoten.cc/blog/deceitful-java-how-i-won-a-free-thingy/#fnref3" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
Test your site on a mobile device locally2017-04-29T00:00:00Zhttps://hoten.cc/blog/test-your-site-on-a-mobile-device-locally/<p>When first starting out building responsive web sites, you may find testing your sites on mobile devices to be cumbersome.</p>
<p>The problem lies with Chrome's developer tools (and others like it). It's a simulator, not an actual device, and a pretty poor one at that. You can trust it just as far as you can throw it (which you can't ... because it isn't a physical object ...).</p>
<p>So you gotta use an actual device. The cumbersome part for new developers is that they actually go through the entire deployment process while testing their mobile device. The workflow might look like:</p>
<ol>
<li>Make some changes to the site</li>
<li>Commit, push, deploy to production</li>
<li>Open the production site on a mobile device. Debug, make changes.</li>
<li>Rinse, repeat.</li>
</ol>
<p>Pretty poor process. Not only are you mucking around in production just to test changes- this takes a long time!</p>
<p>Instead, you can just run the site locally, as usual. If your mobile device and your computer are connected to the same network, you can simply connect to your computer using its local IP address.</p>
<p>In your terminal, type <code>ifconfig</code> or <code>ipconfig</code> (depending on your OS).</p>
<p>You want your computer's LAN IPv4 address.</p>
<h3 id="windows-(ipconfig)" tabindex="-1">windows (<code>ipconfig</code>)</h3>
<pre>
> ipconfig
Windows IP Configuration
Ethernet adapter Ethernet:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Wireless LAN adapter Local Area Connection* 2:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Wireless LAN adapter Local Area Connection* 3:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Ethernet adapter Ethernet 2:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Wireless LAN adapter Wi-Fi:
Connection-specific DNS Suffix . :
Link-local IPv6 Address . . . . . : fe80::d17:173:178a:7e92%8
IPv4 Address. . . . . . . . . . . : <u><b>192.168.1.166</b></u>
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 192.168.1.1
Ethernet adapter Bluetooth Network Connection:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
</pre>
<h3 id="osx-(ifconfig-or-grep-inet)" tabindex="-1">osx (<code>ifconfig | grep inet</code>)</h3>
<pre>
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet6 fe80::843:dbd7:dea3:1b85%en0 prefixlen 64 secured scopeid 0x4
inet <u><b>192.168.1.89</b></u> netmask 0xffffff00 broadcast 192.168.1.255
inet6 fe80::50da:4dff:fe7f:208c%awdl0 prefixlen 64 scopeid 0x9
inet6 fe80::79de:34c1:b4de:f94b%utun0 prefixlen 64 scopeid 0xa
</pre>
<p>On your phone's browser, use that IP address as the hostname, followed by the port your server is using.</p>
<p>Example: <code>http://192.168.1.89:3000</code></p>
<p>You should now see your site! If you don't, ensure that 1) both devices are on the same network and 2) your firewall is allowing the connection.</p>
A gentle introduction to answer set programming2017-07-13T00:00:00Zhttps://hoten.cc/blog/a-gentle-introduction-to-answer-set-programming/<p>I was recently introduced to answer set programming (ASP) as a powerful tool for <a href="https://pdfs.semanticscholar.org/1b7b/1908173a360a10e4a4b9fa97a5359be2e4bc.pdf" target="_blank" rel="noopener noreferrer">generating procedural content for games</a>.</p>
<!-- Excerpt Start -->
<p>ASP is a programming paradigm that solves combinatoric problems given a set of rules. Users of Prolog will recognize the syntax and logic involved, but for everyone else, oh boy, <em>you got some learnin' to do</em>.</p>
<!-- Excerpt End -->
<p>In terms of terseness, answer set programming lies somewhere between C++ and <a href="https://www.youtube.com/watch?v=a9xAKttWgP4" target="_blank" rel="noopener noreferrer">APL</a>.</p>
<p>To begin dabbling in ASP, you need a solver. I'm using <a href="https://github.com/potassco/clingo/releases/tag/v5.2.0" target="_blank" rel="noopener noreferrer">clingo</a>.</p>
<p>I'll try my hand at introducing some of the concepts to you. At the end of the post, you can find some materials for going further with ASP.</p>
<blockquote>
<p>Sidenote: If you happen to be well-versed in ASP, and you notice I am using some terminonlogy incorrectly, please drop me a message! I'm still picking it up, and have a lot left to learn.</p>
</blockquote>
<h1 id="learn(asp)." tabindex="-1">learn(asp).</h1>
<hr />
<p>An AnsProlog program is made up of rules:</p>
<pre class="language-prolog"><code class="language-prolog"><span class="token operator"><</span>head<span class="token operator">></span> <span class="token operator">:-</span> <span class="token operator"><</span>body<span class="token operator">>.</span></code></pre>
<p>If the head is empty, the <code>:-</code> symbol is dropped. Rules with only a body are called <code>facts</code>.</p>
<pre class="language-prolog"><code class="language-prolog"><span class="token function">letter</span><span class="token punctuation">(</span>a<span class="token punctuation">)</span><span class="token operator">.</span><br /><span class="token function">letter</span><span class="token punctuation">(</span>b<span class="token punctuation">)</span><span class="token operator">.</span><br /><span class="token function">letter</span><span class="token punctuation">(</span>c<span class="token punctuation">)</span><span class="token operator">.</span></code></pre>
<p>This can be shortened to:</p>
<pre class="language-prolog"><code class="language-prolog"><span class="token function">letter</span><span class="token punctuation">(</span>a<span class="token operator">;</span> b<span class="token operator">;</span> c<span class="token punctuation">)</span><span class="token operator">.</span></code></pre>
<blockquote>
<p>Sidenote: <code>letter</code> is a user-defined <code>predicate</code>. <code>Predicates</code> can contain any number of <code>atoms</code> (a, b, 1, 2, tom, etc.)</p>
</blockquote>
<p>This ASP program contains only one model in its answer set. Saving as <code>model.lp</code> and running <code>clingo model.lp 0</code> yields:</p>
<pre><code>clingo version 5.2.0
Reading from model.lp
Solving...
Answer: 1
letter(a) letter(b) letter(c)
SATISFIABLE
Models : 1
Calls : 1
Time : 0.006s (Solving: 0.01s 1st Model: 0.00s Unsat: 0.01s)
CPU Time : 0.016s
</code></pre>
<p>The one answer is <code>letter(a) letter(b) letter(c)</code>, which is simply enumerating every fact which was defined.</p>
<blockquote>
<p>Sidenote: an individual answer is a <code>model</code>, and the collection of all models define the <code>answer set</code></p>
</blockquote>
<h1 id="island-generator" tabindex="-1">island generator</h1>
<hr />
<p>Let's create a program to search a simple design space: grids of 10x10 cells, where each cell is one of two types: water or land.</p>
<pre class="language-prolog"><code class="language-prolog"><span class="token function">row</span><span class="token punctuation">(</span><span class="token number">1.</span><span class="token operator">.</span><span class="token number">10</span><span class="token punctuation">)</span><span class="token operator">.</span> <span class="token comment">% the same as row(1). row(2). ...</span><br /><span class="token function">col</span><span class="token punctuation">(</span><span class="token number">1.</span><span class="token operator">.</span><span class="token number">10</span><span class="token punctuation">)</span><span class="token operator">.</span></code></pre>
<p>Let's call each item (ex: <code>row(2)</code>) in a model a <code>term</code>. Terms can be implicitly defined with a <code>proposition</code>.</p>
<pre class="language-prolog"><code class="language-prolog"><span class="token function">cell</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">row</span><span class="token punctuation">(</span>X<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">col</span><span class="token punctuation">(</span>Y<span class="token punctuation">)</span><span class="token operator">.</span></code></pre>
<p>You can read this as "output a term <code>cell(X, Y)</code> if <code>row(X)</code> and <code>row(Y)</code> exists for some <code>X</code> and some <code>Y</code>"</p>
<p>Similarly, "for every combination of row and col terms, bind their values to <code>X</code> and <code>Y</code>, and output a term <code>cell(X, Y)</code>"</p>
<blockquote>
<p>Sidenote: <code>cell(1..10, 1..10)</code> would be more succint</p>
</blockquote>
<p>Let's generate terms for whether something is water or land. This is the first taste of the power of ASP. Everything before created just one single model, but no more.</p>
<pre class="language-prolog"><code class="language-prolog"><span class="token punctuation">{</span> <span class="token function">water</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">:</span> <span class="token function">cell</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token punctuation">}</span><span class="token operator">.</span></code></pre>
<p>This construct is called a <code>choice rule</code>. Like a <code>proposition</code>, it will output the term on the LHS (left hand side) when conditions on the RHS are met. The difference is that the emitted terms are grouped as a <em>set</em>, and each possible subset of terms will be emitted in a different model.</p>
<p>If you ran <code>clingo</code>, oops! There are 100 cells, and each cell can be in one of two states. You've given clingo marching orders to generate 2^100 = 1,267,650,600,228,229,401,496,703,205,376 models. You can run <code>clingo model.lp 1</code> to instruct clingo that you only want a single model.</p>
<p>The solver is deterministic, so it will always be the same model. You can modify this behavior by using additional arguments: <code>clingo blog.lp 1 --sign-def=rnd --rand-freq=1 --seed=123</code>. Providing different values for the seed will affect the solution you get.</p>
<p>Define a complementary term for land:</p>
<pre class="language-prolog"><code class="language-prolog"><span class="token function">land</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">cell</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token operator">not</span> <span class="token function">water</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">.</span></code></pre>
<p>For every X, Y that there exists a term cell(X, Y) and no term water(X, Y), a land(X, Y) will be emitted.</p>
<p>At this point, you should create a script to visualize the output to the program. You can find the python script I used for my visualizations at the bottom of this posting. I used <code>▓▓</code> for water and <code>..</code> for land.</p>
<p>clingo outputs in json when given the option <code>--outf=2</code>. For every visualization, I run this command: <code>clingo model.lp 1 --sign-def=rnd --rand-freq=1 --seed=123 --outf=2 | python visualize.py</code></p>
<p>Feeding data into the visualizer, I get:</p>
<pre><code>▓▓▓▓..▓▓▓▓..▓▓▓▓▓▓..
..▓▓▓▓▓▓▓▓....▓▓▓▓▓▓
..▓▓....▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓..▓▓......▓▓..▓▓
....▓▓▓▓▓▓▓▓▓▓....▓▓
▓▓..▓▓..............
▓▓▓▓▓▓..▓▓..▓▓▓▓..▓▓
▓▓..▓▓..▓▓▓▓........
▓▓..▓▓..▓▓▓▓..▓▓▓▓..
▓▓........▓▓..▓▓▓▓..
</code></pre>
<p>Great. We've come up with a program that successfully searches the design space we defined above. Too bad it's not a very interesting space.</p>
<h2 id="constrain!" tabindex="-1">constrain!</h2>
<hr />
<p>Let's introduce some constraints. Islands are interesting, so let's modify the program to only output models where every edge cell is water.</p>
<blockquote>
<p>Sidenote: I've introduced constants at the top of my model.lp: <code>#const width=10. #const height=10.</code></p>
</blockquote>
<pre class="language-prolog"><code class="language-prolog"><span class="token function">water</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">cell</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">.</span> <span class="token comment">% if a term exists matching "cell(1, Y)", then there must exist a term "water(1, Y)"</span><br /><span class="token function">water</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">cell</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token operator">.</span><br /><span class="token function">water</span><span class="token punctuation">(</span>width<span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">cell</span><span class="token punctuation">(</span>width<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">.</span><br /><span class="token function">water</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> height<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">cell</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> height<span class="token punctuation">)</span><span class="token operator">.</span></code></pre>
<pre><code>▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓..▓▓▓▓......▓▓▓▓▓▓
▓▓....▓▓▓▓......▓▓▓▓
▓▓....▓▓..▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓....▓▓▓▓..▓▓
▓▓▓▓▓▓..▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓..▓▓▓▓▓▓
▓▓▓▓....▓▓▓▓▓▓....▓▓
▓▓▓▓..............▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
</code></pre>
<h2 id="connectedness" tabindex="-1">connectedness</h2>
<hr />
<p>Now, let's force all land cells to be connected (reachable by moving in a cardinal direction). This can be accomplished by establishing a starting point, creating a term <code>connected(X, Y)</code> if <code>land(X, Y)</code> is connected to that starting point, and requiring that all land terms are connected.</p>
<p>One way to select the starting land cell is to find the top-left most land. However, it is much simpler to define the middle as always land, and use that as the start.</p>
<p>By definition, the start is connected to itself, so a connected term is emitted there.</p>
<p><code>connected</code> terms are generated for X, Y if land(X, Y) exists, and there is a neighboring connected cell.</p>
<pre class="language-prolog"><code class="language-prolog"><span class="token comment">% make the middle cell always land</span><br /><br /><span class="token function">land</span><span class="token punctuation">(</span><span class="token function">width/2</span><span class="token punctuation">,</span> <span class="token function">height/2</span><span class="token punctuation">)</span><span class="token operator">.</span><br /><br /><span class="token comment">% base condition. the start is connected to itself</span><br /><br /><span class="token function">connected</span><span class="token punctuation">(</span><span class="token function">width/2</span><span class="token punctuation">,</span> <span class="token function">height/2</span><span class="token punctuation">)</span><span class="token operator">.</span><br /><br /><span class="token comment">% land cells are connected to the start if</span><br /><span class="token comment">% neighboring cells are connected to the start</span><br /><br /><span class="token function">connected</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">connected</span><span class="token punctuation">(</span>X <span class="token operator">-</span> <span class="token number">1</span><span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">land</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">.</span><br /><span class="token function">connected</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">connected</span><span class="token punctuation">(</span>X <span class="token operator">+</span> <span class="token number">1</span><span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">land</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">.</span><br /><span class="token function">connected</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">connected</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y <span class="token operator">-</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">land</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">.</span><br /><span class="token function">connected</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">connected</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y <span class="token operator">+</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">land</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">.</span><br /><br /><span class="token comment">% integrity constraint requiring all land to be connected</span><br /><span class="token operator">:-</span> <span class="token operator">not</span> <span class="token function">connected</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">land</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">.</span></code></pre>
<p>That last line is a bit different. It's a rule without a head. This is an <code>integrity constraint</code>. In their simplest form, they look like this:</p>
<p><code>:- not good_condition</code></p>
<p>Conversely,</p>
<p><code>:- bad_condition</code></p>
<p>The second term there (<code>land(X, Y)</code>) acts as a binding variable- for every X, Y for which the term land(X, Y) exists, the good_condition (<code>connected(X, Y)</code>) must succeed.</p>
<blockquote>
<p>Sidenote: I originally forgot that last line, and the first example I looked at for validating actually was all connected! However, it was wrong. It took awhile to realize my mistake, and after trying a second example the error was obvious. It's very, very important to iterate and validate carefully.</p>
</blockquote>
<p>Now, the final output.</p>
<pre><code>▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓..▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓......▓▓▓▓......▓▓
▓▓..▓▓........▓▓▓▓▓▓
▓▓..▓▓....▓▓......▓▓
▓▓▓▓......▓▓▓▓....▓▓
▓▓▓▓........▓▓▓▓..▓▓
▓▓....▓▓▓▓....▓▓▓▓▓▓
▓▓........▓▓....▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
</code></pre>
<h2 id="drum-roll-..." tabindex="-1">drum roll ...</h2>
<hr />
<p>And, for fun, a 50x50 island:</p>
<pre><code>▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓....▓▓..▓▓..▓▓▓▓........▓▓....▓▓▓▓..▓▓......▓▓......▓▓▓▓..▓▓▓▓....▓▓......▓▓......▓▓▓▓▓▓..▓▓....▓▓
▓▓..▓▓▓▓............▓▓▓▓..▓▓......▓▓............▓▓..............▓▓............▓▓▓▓..▓▓..........▓▓▓▓
▓▓..▓▓▓▓..▓▓....▓▓....▓▓..........▓▓....▓▓..▓▓..▓▓..▓▓..▓▓▓▓▓▓▓▓▓▓......▓▓▓▓▓▓........▓▓..▓▓▓▓..▓▓▓▓
▓▓..........▓▓..▓▓......▓▓....▓▓▓▓..▓▓........▓▓....▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓....▓▓..▓▓....▓▓..............▓▓
▓▓......▓▓▓▓..▓▓....▓▓....▓▓....▓▓......▓▓▓▓....▓▓..▓▓..▓▓▓▓▓▓▓▓..▓▓........▓▓....▓▓▓▓..▓▓▓▓..▓▓..▓▓
▓▓..▓▓........▓▓▓▓▓▓▓▓▓▓....▓▓▓▓▓▓▓▓..▓▓▓▓▓▓........▓▓....▓▓......▓▓▓▓▓▓..▓▓....▓▓..▓▓▓▓▓▓▓▓..▓▓▓▓▓▓
▓▓▓▓....▓▓..▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓....▓▓..▓▓........▓▓....▓▓..............▓▓▓▓....▓▓......▓▓▓▓▓▓
▓▓....▓▓▓▓....▓▓▓▓▓▓▓▓....▓▓▓▓▓▓....▓▓▓▓..▓▓....▓▓..▓▓▓▓▓▓▓▓..▓▓▓▓..▓▓▓▓▓▓........▓▓....▓▓▓▓....▓▓▓▓
▓▓▓▓....▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓......▓▓▓▓..▓▓......▓▓▓▓▓▓..▓▓▓▓........▓▓..▓▓▓▓▓▓▓▓..▓▓▓▓....▓▓▓▓▓▓▓▓▓▓..▓▓
▓▓▓▓......▓▓▓▓▓▓▓▓▓▓▓▓....▓▓▓▓▓▓....▓▓▓▓......▓▓▓▓..▓▓▓▓..▓▓............▓▓▓▓..........▓▓▓▓▓▓▓▓▓▓..▓▓
▓▓▓▓....▓▓....▓▓........▓▓▓▓▓▓▓▓▓▓..▓▓..▓▓▓▓....▓▓..▓▓▓▓▓▓........▓▓....▓▓▓▓▓▓......▓▓▓▓▓▓▓▓......▓▓
▓▓....▓▓....▓▓▓▓▓▓........▓▓▓▓▓▓▓▓..▓▓..........▓▓..▓▓▓▓▓▓..▓▓▓▓▓▓........▓▓....▓▓..▓▓....▓▓..▓▓▓▓▓▓
▓▓▓▓..▓▓..▓▓▓▓▓▓▓▓........................▓▓▓▓......▓▓▓▓..........▓▓▓▓..▓▓....▓▓....▓▓..▓▓........▓▓
▓▓▓▓..........▓▓..▓▓▓▓....▓▓▓▓▓▓▓▓▓▓▓▓▓▓..▓▓▓▓..▓▓..........▓▓▓▓..▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓......▓▓..▓▓▓▓▓▓
▓▓▓▓▓▓▓▓..▓▓..........▓▓........▓▓......▓▓▓▓▓▓▓▓..▓▓▓▓▓▓....▓▓..▓▓......▓▓▓▓▓▓........▓▓........▓▓▓▓
▓▓▓▓▓▓▓▓▓▓....▓▓..▓▓....▓▓..............▓▓▓▓▓▓......▓▓..▓▓..........▓▓..........▓▓....▓▓▓▓▓▓....▓▓▓▓
▓▓▓▓▓▓▓▓▓▓....▓▓..▓▓..........▓▓................▓▓........▓▓▓▓....▓▓....▓▓▓▓..▓▓....▓▓..........▓▓▓▓
▓▓▓▓▓▓▓▓......▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓....▓▓..........▓▓..▓▓▓▓▓▓▓▓..▓▓▓▓▓▓▓▓▓▓▓▓......▓▓▓▓▓▓▓▓..▓▓....▓▓▓▓▓▓
▓▓........▓▓▓▓▓▓▓▓▓▓..▓▓▓▓▓▓..▓▓▓▓▓▓▓▓..▓▓........▓▓▓▓▓▓........▓▓....▓▓▓▓......▓▓....▓▓......▓▓▓▓▓▓
▓▓..▓▓▓▓..▓▓......▓▓........................▓▓..▓▓▓▓▓▓▓▓▓▓....▓▓..▓▓..............▓▓......▓▓..▓▓▓▓▓▓
▓▓..........▓▓..▓▓▓▓▓▓▓▓....▓▓▓▓......▓▓▓▓....▓▓..▓▓▓▓▓▓▓▓..▓▓▓▓........▓▓▓▓..▓▓....▓▓▓▓..........▓▓
▓▓..▓▓▓▓▓▓..▓▓..▓▓▓▓▓▓▓▓▓▓..▓▓....▓▓..▓▓▓▓▓▓▓▓......▓▓..▓▓....▓▓▓▓..▓▓....▓▓..▓▓▓▓....▓▓▓▓▓▓▓▓▓▓..▓▓
▓▓▓▓▓▓▓▓▓▓..▓▓..........▓▓..▓▓▓▓▓▓▓▓..▓▓▓▓▓▓....▓▓..........▓▓..▓▓........▓▓....▓▓▓▓....▓▓▓▓▓▓....▓▓
▓▓▓▓▓▓▓▓▓▓..▓▓........▓▓▓▓....▓▓▓▓......▓▓▓▓▓▓▓▓........▓▓▓▓....▓▓..▓▓......▓▓▓▓▓▓▓▓▓▓..▓▓........▓▓
▓▓▓▓..▓▓▓▓......▓▓▓▓▓▓▓▓▓▓..▓▓▓▓▓▓▓▓..▓▓▓▓▓▓..▓▓..▓▓▓▓▓▓..▓▓▓▓......▓▓......▓▓▓▓..▓▓..▓▓..▓▓....▓▓▓▓
▓▓....▓▓▓▓......▓▓▓▓▓▓▓▓▓▓....▓▓..........▓▓......▓▓▓▓▓▓..▓▓..▓▓▓▓▓▓....▓▓..▓▓....▓▓..▓▓..▓▓..▓▓▓▓▓▓
▓▓..............▓▓▓▓▓▓....▓▓▓▓....▓▓..▓▓▓▓▓▓..▓▓..▓▓▓▓........▓▓..▓▓..▓▓▓▓▓▓▓▓▓▓..............▓▓▓▓▓▓
▓▓▓▓▓▓....▓▓▓▓▓▓▓▓......▓▓......▓▓▓▓..▓▓▓▓....▓▓......▓▓▓▓....▓▓............▓▓....▓▓▓▓..▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓....▓▓▓▓▓▓▓▓▓▓▓▓▓▓..▓▓..▓▓▓▓..........▓▓..▓▓▓▓..▓▓..........▓▓▓▓▓▓..▓▓..▓▓..▓▓........▓▓▓▓▓▓▓▓▓▓
▓▓....▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓......▓▓▓▓..▓▓▓▓▓▓▓▓....▓▓..▓▓..▓▓▓▓▓▓▓▓..▓▓......▓▓....▓▓....▓▓..▓▓..▓▓▓▓▓▓▓▓
▓▓▓▓....▓▓..▓▓▓▓..▓▓▓▓..▓▓..▓▓▓▓▓▓........................▓▓....▓▓▓▓......▓▓▓▓..▓▓▓▓▓▓..▓▓..▓▓▓▓▓▓▓▓
▓▓▓▓..▓▓▓▓............▓▓..▓▓........▓▓▓▓▓▓..▓▓▓▓▓▓▓▓▓▓......▓▓....▓▓▓▓▓▓▓▓..▓▓..▓▓▓▓....▓▓........▓▓
▓▓........▓▓....▓▓..▓▓....▓▓..▓▓▓▓▓▓▓▓▓▓....▓▓..▓▓....▓▓▓▓▓▓▓▓....▓▓............▓▓..........▓▓▓▓..▓▓
▓▓........▓▓..▓▓..▓▓▓▓....▓▓▓▓▓▓▓▓▓▓▓▓▓▓....▓▓......▓▓..........▓▓▓▓..▓▓▓▓..▓▓..▓▓..▓▓▓▓..▓▓▓▓....▓▓
▓▓..▓▓....▓▓..▓▓..▓▓....▓▓......▓▓..▓▓..▓▓..▓▓▓▓........▓▓▓▓..▓▓....▓▓▓▓▓▓..▓▓....▓▓▓▓▓▓....▓▓▓▓▓▓▓▓
▓▓..▓▓........▓▓......▓▓▓▓▓▓....▓▓..........▓▓......▓▓..▓▓▓▓......▓▓........▓▓....▓▓..▓▓..▓▓....▓▓▓▓
▓▓▓▓....▓▓..............................▓▓......▓▓..▓▓..▓▓▓▓▓▓▓▓............▓▓▓▓..▓▓..............▓▓
▓▓▓▓▓▓▓▓▓▓..▓▓▓▓....▓▓..▓▓..▓▓▓▓▓▓..▓▓▓▓......▓▓▓▓..▓▓▓▓..▓▓▓▓..▓▓..▓▓..▓▓..▓▓▓▓▓▓▓▓▓▓▓▓..▓▓▓▓....▓▓
▓▓▓▓▓▓......▓▓▓▓▓▓..▓▓▓▓▓▓▓▓....▓▓▓▓▓▓▓▓..▓▓....▓▓........▓▓▓▓......▓▓▓▓▓▓....▓▓▓▓▓▓......▓▓....▓▓▓▓
▓▓......▓▓▓▓▓▓▓▓........▓▓▓▓..........▓▓....▓▓▓▓▓▓▓▓..▓▓▓▓▓▓▓▓▓▓▓▓........▓▓..▓▓▓▓▓▓▓▓....▓▓....▓▓▓▓
▓▓..▓▓▓▓▓▓....▓▓▓▓....▓▓..▓▓....▓▓..........▓▓▓▓......▓▓......▓▓........▓▓....▓▓......▓▓▓▓▓▓▓▓....▓▓
▓▓....▓▓..........▓▓......▓▓▓▓..▓▓▓▓▓▓▓▓▓▓..▓▓▓▓....▓▓▓▓..▓▓..▓▓........▓▓▓▓▓▓....▓▓▓▓▓▓..▓▓▓▓▓▓..▓▓
▓▓..▓▓....▓▓▓▓▓▓....▓▓▓▓▓▓▓▓▓▓▓▓..▓▓▓▓..▓▓..▓▓▓▓....▓▓▓▓▓▓........▓▓▓▓..▓▓▓▓▓▓▓▓............▓▓▓▓..▓▓
▓▓........▓▓▓▓..▓▓..▓▓▓▓▓▓▓▓........▓▓..▓▓▓▓▓▓▓▓▓▓▓▓..▓▓..▓▓▓▓▓▓▓▓▓▓▓▓▓▓..▓▓▓▓..▓▓▓▓▓▓..▓▓..▓▓....▓▓
▓▓....▓▓..........▓▓▓▓▓▓......▓▓▓▓..............................▓▓▓▓..............▓▓▓▓..▓▓..▓▓▓▓..▓▓
▓▓▓▓..▓▓▓▓..▓▓▓▓......▓▓..▓▓......▓▓▓▓▓▓..▓▓..▓▓▓▓▓▓▓▓....▓▓....▓▓▓▓..▓▓................▓▓....▓▓..▓▓
▓▓........▓▓..▓▓..▓▓....▓▓▓▓..▓▓..▓▓..▓▓..▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓........▓▓▓▓....▓▓..▓▓▓▓▓▓▓▓▓▓▓▓▓▓..▓▓..▓▓
▓▓..▓▓..▓▓▓▓........▓▓........▓▓..▓▓......▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓..▓▓▓▓..▓▓▓▓▓▓..▓▓..▓▓▓▓▓▓▓▓▓▓▓▓▓▓......▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
</code></pre>
<p>You know, on second thought, this looks more like a mineshaft ...</p>
<h1 id=":-not-good(r)-resource(r)." tabindex="-1">:- not good(R), resource(R).</h1>
<hr />
<p>Solving a problem with ASP generally follows this process:</p>
<ol>
<li>Sculpt the general shape of the model</li>
<li>Inspect the output, add constraints to remove unwanted models</li>
<li>Repeat 2 until you're blue in the face</li>
</ol>
<p>Basically: design, constrain, iterate.</p>
<p>This development process is outlined in <a href="http://ceur-ws.org/Vol-546/49-63.pdf" target="_blank" rel="noopener noreferrer">A Pragmatic Programmer’s Guide to Answer Set Programming</a>.</p>
<hr />
<p><a href="https://adamsmith.as/" target="_blank" rel="noopener noreferrer">Adam Smith</a> (no, not that one) provides some fantastic examples of how ASP can be utilized in creating content for games. See his paper <a href="https://users.soe.ucsc.edu/~amsmith/papers/tciaig-asp4pcg.pdf" target="_blank" rel="noopener noreferrer">Answer Set Programming for Procedural Content Generation: A Design Space Approach</a> for more, or a more bite sized blog post on <a href="https://eis-blog.soe.ucsc.edu/2011/10/map-generation-speedrun/" target="_blank" rel="noopener noreferrer">length constraints in map generation</a> for less.</p>
<hr />
<p>The python script used for visualizing the island example:</p>
<pre class="language-python"><code class="language-python"><span class="token keyword">import</span> sys<span class="token punctuation">,</span> json<span class="token punctuation">,</span> re<br /><br /><span class="token keyword">def</span> <span class="token function">process</span><span class="token punctuation">(</span>model<span class="token punctuation">)</span><span class="token punctuation">:</span><br /> width<span class="token punctuation">,</span> height <span class="token operator">=</span> find_one<span class="token punctuation">(</span>model<span class="token punctuation">,</span> <span class="token string">"size"</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><br /> grid <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">[</span><span class="token string">' '</span> <span class="token keyword">for</span> x <span class="token keyword">in</span> <span class="token builtin">range</span><span class="token punctuation">(</span>width<span class="token punctuation">)</span><span class="token punctuation">]</span> <span class="token keyword">for</span> y <span class="token keyword">in</span> <span class="token builtin">range</span><span class="token punctuation">(</span>height<span class="token punctuation">)</span><span class="token punctuation">]</span><br /><br /> <span class="token keyword">for</span> _<span class="token punctuation">,</span> <span class="token punctuation">(</span>x<span class="token punctuation">,</span> y<span class="token punctuation">)</span> <span class="token keyword">in</span> find_all<span class="token punctuation">(</span>model<span class="token punctuation">,</span> <span class="token string">"water"</span><span class="token punctuation">)</span><span class="token punctuation">:</span><br /> grid<span class="token punctuation">[</span>y <span class="token operator">-</span> <span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">[</span>x <span class="token operator">-</span> <span class="token number">1</span><span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token string">'▓'</span><br /><br /> <span class="token keyword">for</span> _<span class="token punctuation">,</span> <span class="token punctuation">(</span>x<span class="token punctuation">,</span> y<span class="token punctuation">)</span> <span class="token keyword">in</span> find_all<span class="token punctuation">(</span>model<span class="token punctuation">,</span> <span class="token string">"land"</span><span class="token punctuation">)</span><span class="token punctuation">:</span><br /> grid<span class="token punctuation">[</span>y <span class="token operator">-</span> <span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">[</span>x <span class="token operator">-</span> <span class="token number">1</span><span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token string">'.'</span><br /><br /> <span class="token keyword">for</span> row <span class="token keyword">in</span> grid<span class="token punctuation">:</span><br /> <span class="token keyword">for</span> cell <span class="token keyword">in</span> row<span class="token punctuation">:</span><br /> <span class="token keyword">print</span><span class="token punctuation">(</span>cell <span class="token operator">*</span> <span class="token number">2</span><span class="token punctuation">,</span> end<span class="token operator">=</span><span class="token string">''</span><span class="token punctuation">)</span><br /> <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br /><br /><br /><span class="token keyword">def</span> <span class="token function">find_one</span><span class="token punctuation">(</span>model<span class="token punctuation">,</span> token_type<span class="token punctuation">)</span><span class="token punctuation">:</span><br /> <span class="token keyword">return</span> <span class="token builtin">next</span><span class="token punctuation">(</span>token <span class="token keyword">for</span> token <span class="token keyword">in</span> model <span class="token keyword">if</span> token<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator">==</span> token_type<span class="token punctuation">)</span><br /><br /><br /><span class="token keyword">def</span> <span class="token function">find_all</span><span class="token punctuation">(</span>model<span class="token punctuation">,</span> token_type<span class="token punctuation">)</span><span class="token punctuation">:</span><br /> <span class="token keyword">return</span> <span class="token punctuation">[</span>token <span class="token keyword">for</span> token <span class="token keyword">in</span> model <span class="token keyword">if</span> token<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator">==</span> token_type<span class="token punctuation">]</span><br /><br /><br /><span class="token comment"># parse into a number, if possible. avoid converting "NaN, Inf"</span><br /><span class="token keyword">def</span> <span class="token function">convert_value</span><span class="token punctuation">(</span>value<span class="token punctuation">)</span><span class="token punctuation">:</span><br /> <span class="token keyword">if</span> value<span class="token punctuation">.</span>isalpha<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span><br /> <span class="token keyword">return</span> value<br /> <br /> <span class="token keyword">try</span><span class="token punctuation">:</span><br /> <span class="token keyword">return</span> <span class="token builtin">int</span><span class="token punctuation">(</span>value<span class="token punctuation">)</span><br /> <span class="token keyword">except</span> ValueError<span class="token punctuation">:</span><br /> <span class="token keyword">try</span><span class="token punctuation">:</span><br /> <span class="token keyword">return</span> <span class="token builtin">float</span><span class="token punctuation">(</span>value<span class="token punctuation">)</span><br /> <span class="token keyword">except</span> ValueError<span class="token punctuation">:</span><br /> <span class="token keyword">return</span> value<br /><br /><br /><span class="token keyword">def</span> <span class="token function">parse_token</span><span class="token punctuation">(</span>token_text<span class="token punctuation">)</span><span class="token punctuation">:</span><br /> <span class="token keyword">match</span> <span class="token operator">=</span> re<span class="token punctuation">.</span><span class="token keyword">match</span><span class="token punctuation">(</span><span class="token string">r"(.+)\((.*)\)"</span><span class="token punctuation">,</span> token_text<span class="token punctuation">)</span><br /> <span class="token builtin">type</span> <span class="token operator">=</span> <span class="token keyword">match</span><span class="token punctuation">.</span>group<span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">)</span><br /> values <span class="token operator">=</span> <span class="token punctuation">[</span>convert_value<span class="token punctuation">(</span>value<span class="token punctuation">)</span> <span class="token keyword">for</span> value <span class="token keyword">in</span> <span class="token keyword">match</span><span class="token punctuation">.</span>group<span class="token punctuation">(</span><span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">.</span>split<span class="token punctuation">(</span><span class="token string">","</span><span class="token punctuation">)</span><span class="token punctuation">]</span><br /> <span class="token keyword">return</span> <span class="token punctuation">(</span><span class="token builtin">type</span><span class="token punctuation">,</span> values<span class="token punctuation">)</span><br /><br /><br /><span class="token builtin">input</span> <span class="token operator">=</span> json<span class="token punctuation">.</span>loads<span class="token punctuation">(</span>sys<span class="token punctuation">.</span>stdin<span class="token punctuation">.</span>read<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /><span class="token keyword">for</span> call <span class="token keyword">in</span> <span class="token builtin">input</span><span class="token punctuation">[</span><span class="token string">"Call"</span><span class="token punctuation">]</span><span class="token punctuation">:</span><br /> <span class="token keyword">for</span> witness <span class="token keyword">in</span> call<span class="token punctuation">[</span><span class="token string">"Witnesses"</span><span class="token punctuation">]</span><span class="token punctuation">:</span><br /> model <span class="token operator">=</span> <span class="token punctuation">[</span>parse_token<span class="token punctuation">(</span>token_text<span class="token punctuation">)</span> <span class="token keyword">for</span> token_text <span class="token keyword">in</span> witness<span class="token punctuation">[</span><span class="token string">"Value"</span><span class="token punctuation">]</span><span class="token punctuation">]</span><br /> process<span class="token punctuation">(</span>model<span class="token punctuation">)</span><br /></code></pre>
<hr />
<p>The full island search program:</p>
<pre class="language-prolog"><code class="language-prolog">#const width<span class="token operator">=</span><span class="token number">50.</span><br />#const height<span class="token operator">=</span><span class="token number">50.</span><br /><br /><span class="token function">size</span><span class="token punctuation">(</span>width<span class="token punctuation">,</span> height<span class="token punctuation">)</span><span class="token operator">.</span><br /><br /><span class="token function">cell</span><span class="token punctuation">(</span><span class="token number">1.</span><span class="token operator">.</span>width<span class="token punctuation">,</span> <span class="token number">1.</span><span class="token operator">.</span>height<span class="token punctuation">)</span><span class="token operator">.</span><br /><br /><span class="token punctuation">{</span> <span class="token function">water</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">:</span> <span class="token function">cell</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token punctuation">}</span><span class="token operator">.</span><br /><br /><span class="token function">land</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">cell</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token operator">not</span> <span class="token function">water</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">.</span><br /><br /><span class="token function">water</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">cell</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">.</span><br /><span class="token function">water</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">cell</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token operator">.</span><br /><span class="token function">water</span><span class="token punctuation">(</span>width<span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">cell</span><span class="token punctuation">(</span>width<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">.</span><br /><span class="token function">water</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> height<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">cell</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> height<span class="token punctuation">)</span><span class="token operator">.</span><br /><br /><span class="token comment">% make the middle cell always land</span><br /><br /><span class="token function">land</span><span class="token punctuation">(</span><span class="token function">width/2</span><span class="token punctuation">,</span> <span class="token function">height/2</span><span class="token punctuation">)</span><span class="token operator">.</span><br /><br /><span class="token comment">% base condition. the start is connected to itself</span><br /><br /><span class="token function">connected</span><span class="token punctuation">(</span><span class="token function">width/2</span><span class="token punctuation">,</span> <span class="token function">height/2</span><span class="token punctuation">)</span><span class="token operator">.</span><br /><br /><span class="token comment">% land cells are connected to the start if</span><br /><span class="token comment">% neighboring cells are connected to the start</span><br /><br /><span class="token function">connected</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">connected</span><span class="token punctuation">(</span>X <span class="token operator">-</span> <span class="token number">1</span><span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">land</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">.</span><br /><span class="token function">connected</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">connected</span><span class="token punctuation">(</span>X <span class="token operator">+</span> <span class="token number">1</span><span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">land</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">.</span><br /><span class="token function">connected</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">connected</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y <span class="token operator">-</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">land</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">.</span><br /><span class="token function">connected</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span> <span class="token operator">:-</span> <span class="token function">connected</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y <span class="token operator">+</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">land</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">.</span><br /><br /><span class="token operator">:-</span> <span class="token operator">not</span> <span class="token function">connected</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">land</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> Y<span class="token punctuation">)</span><span class="token operator">.</span></code></pre>
Speed up building a monolithic app by skipping build steps2018-07-08T00:00:00Zhttps://hoten.cc/blog/speed-up-building-a-monolithic-app-by-skipping-build-steps/<!-- Excerpt Start -->
<p>Late Feburary of this year, I worked on improving the speed of Course Hero's frontend build process. I noted the impact of each improvement I made. I don't think the results are suprising, but I'd like to share anyways.</p>
<p>I'll also show how git can be leveraged to skip build steps when the input files haven't changed - this strategy can help keep build times down in a Monolithic app.</p>
<!-- Excerpt End -->
<p>First, some details. At the time, it took about 12 minutes to build and deploy to our development environment. About 4.5 minutes of that time was from building a few dozen TypeScript projects.</p>
<p>There were a handful of obvious improvements to make - we were some major versions behind on some software central to the build process, and our Webpack projects were not using a single configuration.</p>
<p>For each change, I ran the build step a few times to get an average, and these are the results:</p>
<p>4m21s -> 1m56s (44.44% of original)</p>
<ul>
<li>(-1m9s) Upgrading webpack (2.6 -> 4.0), babelify (7.3 -> 8.0), and babel (6.24 -> 7.0)</li>
<li>(-34s) Consolidating webpack projects into one webpack build config</li>
<li>(-28s) Upgrading node (v6.10.0 -> v8.9.3) and yarn (v0.21.3 -> v1.3.2)</li>
<li>(-18s) Utilize docker volume to persist .yarn-cache, greatly speeding up package downloading and linking (<code>-v yarn-cache-monolith-<dev/prod/stage>:/usr/local/share/.cache/yarn/v1</code>)</li>
</ul>
<p>However, the best improvement by far was <em>skipping the entire build step</em>. The frontend build process outputs to a folder that persists across builds, and if it can be determined that none of the input sources have changed, there is no need to build the TypeScript projects again.</p>
<p>Luckily, <a href="http://shafiulazam.com/gitbook/1_the_git_object_model.html" target="_blank" rel="noopener noreferrer">git's internal object model</a> gives us exactly what we need. All of the source files for this build step are in the <code>js/</code> folder, so we just need to ask git about that folder.</p>
<pre><code>> git ls-tree HEAD js | awk '{print $3}'
4bf1053ffb91970dcbd1425084062ef812c44ba6
</code></pre>
<p>That hash changes whenever anything under the <code>js/</code> folder changes. By simply saving this hash, we can determine if this build step can be skipped.</p>
<p><code>js/compile-on-change</code></p>
<pre class="language-bash"><code class="language-bash"><span class="token shebang important">#!/bin/bash</span><br /><br /><span class="token comment"># this script is run from the "js/" folder</span><br /><span class="token comment"># this hash (the script argument) changes whenever anything in "js/" changes</span><br /><span class="token comment"># ./compile-on-change $(git ls-tree HEAD $PWD | awk '{print $3}')</span><br /><br /><span class="token comment"># hash is passed in instead of calculated within script, since that is faster than</span><br /><span class="token comment"># using git in a docker container</span><br /><br /><span class="token assign-left variable">hash</span><span class="token operator">=</span><span class="token variable">$1</span><br /><span class="token builtin class-name">echo</span> <span class="token string">"js/ hash: <span class="token variable">$hash</span>"</span><br /><br /><span class="token keyword">if</span> <span class="token punctuation">[</span> <span class="token operator">!</span> -d ./dist <span class="token punctuation">]</span> <span class="token operator">||</span> <span class="token punctuation">[</span> <span class="token operator">!</span> -f ./hash <span class="token punctuation">]</span> <span class="token operator">||</span> <span class="token punctuation">[</span><span class="token punctuation">[</span> <span class="token variable"><span class="token variable">$(</span><span class="token function">cat</span> ./hash<span class="token variable">)</span></span> <span class="token operator">!=</span> <span class="token string">"<span class="token variable">$hash</span>"</span> <span class="token punctuation">]</span><span class="token punctuation">]</span><span class="token punctuation">;</span> <span class="token keyword">then</span><br /> <span class="token builtin class-name">echo</span> <span class="token string">"change in js/ detected - compiling frontend assets"</span><br /> <br /> <span class="token function">yarn</span> compile<br /> <span class="token keyword">if</span> <span class="token punctuation">[</span> <span class="token variable">$?</span> -eq <span class="token number">0</span> <span class="token punctuation">]</span><span class="token punctuation">;</span> <span class="token keyword">then</span><br /> <span class="token builtin class-name">echo</span> <span class="token variable">$hash</span> <span class="token operator">></span> ./hash<br /> <span class="token keyword">else</span><br /> <span class="token function">rm</span> -f ./hash<br /> <span class="token builtin class-name">exit</span> <span class="token number">1</span><br /> <span class="token keyword">fi</span><br /><span class="token keyword">else</span><br /> <span class="token builtin class-name">echo</span> <span class="token string">"no change in js/ detected - skipping compilation"</span><br /><span class="token keyword">fi</span><br /></code></pre>
<p>Most pushs don't change any files in the <code>js/</code> folder, so this change greatly speeds up most builds. It takes about 2s to determine that the step can be skipped. So, after all these improvements, this build step takes between 0.7% and 44% of the original build time, give or take.</p>
<hr />
<p>An obvious next step is to individually apply this hash checking for every TypeScript project. This requires determining if any changes occurred in a file that a project uses.</p>
<ol>
<li>For each TypeScript project, determine the files that affect the output</li>
<li>Get the git hash for all these files and save to disk</li>
<li>Only build the project if there is a mismatch between the hashes on disk and the current hashes</li>
</ol>
<p>One problem that complicates 1) is that, for a given project, there is no single folder that contains just that project's source files. Some projects import modules from other project folders. Manually keeping track of these dependencies is not a good solution - this must be automated.</p>
<p>Luckily, the TypeScript compiler provides a simple way to determine, given an entry point, all the files that are imported. Below is a sample implementation</p>
<p><code>getDeps.js</code></p>
<pre class="language-js"><code class="language-js"><span class="token comment">/*<br />git ls-tree HEAD $(node getDeps.js src/dashboard/app.tsx)<br /><br />This command will help generate a unique hash for a project's source files.<br />This can be used to check if a project needs to be recompiled.<br /><br />If we can determine the source files are unchanged, we can skip compilation. As an example,<br />the dashboard app takes about 16 seconds to build. Determining if we can skip takes ~1-3 seconds.<br /><br />> time bash -c 'git ls-tree HEAD $(node getDeps.js src/dashboard/app.tsx)'<br />0.8s<br />*/</span><br /><br /><span class="token keyword">const</span> tsc <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'typescript'</span><span class="token punctuation">)</span><br /><br /><span class="token keyword">function</span> <span class="token function">getTsSources</span><span class="token punctuation">(</span><span class="token parameter">entry</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> options <span class="token operator">=</span> <span class="token punctuation">{</span><br /> <span class="token literal-property property">target</span><span class="token operator">:</span> tsc<span class="token punctuation">.</span>ScriptTarget<span class="token punctuation">.</span><span class="token constant">ES2015</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">jsx</span><span class="token operator">:</span> tsc<span class="token punctuation">.</span>JsxEmit<span class="token punctuation">.</span>Preserve<br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">const</span> program <span class="token operator">=</span> tsc<span class="token punctuation">.</span><span class="token function">createProgram</span><span class="token punctuation">(</span><span class="token punctuation">[</span>entry<span class="token punctuation">]</span><span class="token punctuation">,</span> options<span class="token punctuation">)</span><br /> <span class="token keyword">return</span> program<span class="token punctuation">.</span><span class="token function">getSourceFiles</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">filter</span><span class="token punctuation">(</span><span class="token parameter">source</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> path <span class="token operator">=</span> source<span class="token punctuation">.</span>path<br /> <span class="token keyword">return</span> path<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">'node_modules'</span><span class="token punctuation">)</span> <span class="token operator">===</span> <span class="token operator">-</span><span class="token number">1</span> <span class="token operator">&&</span> path<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">'vendor'</span><span class="token punctuation">)</span> <span class="token operator">===</span> <span class="token operator">-</span><span class="token number">1</span><br /> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">map</span><span class="token punctuation">(</span><span class="token parameter">source</span> <span class="token operator">=></span> source<span class="token punctuation">.</span>path<span class="token punctuation">)</span><br /><span class="token punctuation">}</span><br /><br /><span class="token keyword">const</span> deps <span class="token operator">=</span> <span class="token function">getTsSources</span><span class="token punctuation">(</span>process<span class="token punctuation">.</span>argv<span class="token punctuation">[</span><span class="token number">2</span><span class="token punctuation">]</span><span class="token punctuation">)</span><br /><br /><span class="token comment">// changes in any of these files should rebuild everything</span><br />deps<span class="token punctuation">.</span><span class="token function">push</span><span class="token punctuation">.</span><span class="token function">apply</span><span class="token punctuation">(</span>deps<span class="token punctuation">,</span> <span class="token punctuation">[</span><br /> <span class="token string">'yarn.lock'</span><span class="token punctuation">,</span><br /> <span class="token string">'package.json'</span><span class="token punctuation">,</span><br /> <span class="token string">'tsconfig.json'</span><span class="token punctuation">,</span><br /> <span class="token string">'.babelrc'</span><span class="token punctuation">,</span><br /> <span class="token string">'bower.json'</span><span class="token punctuation">,</span><br /> <span class="token string">'build/gulpfile.common.js'</span><span class="token punctuation">,</span><br /> <span class="token string">'build/shim.js'</span><br /><span class="token punctuation">]</span><span class="token punctuation">)</span><br /><br />console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>deps<span class="token punctuation">.</span><span class="token function">join</span><span class="token punctuation">(</span><span class="token string">"\n"</span><span class="token punctuation">)</span><span class="token punctuation">)</span></code></pre>
<p>Some files will trigger a rebuild for everything. For example, if <code>yarn.lock</code> changes, every project should rebuild.</p>
<p>example output from <code>git ls-tree HEAD $(node getDeps.js src/dashboard/app.tsx)</code> (filenames obfuscated):</p>
<pre><code>100644 blob 8b379cdaf6d1eb9507b6ff4c46b75fb9fc612b69 .babelrc
100644 blob 4fd5eea6090476313cb2efebaa14c4ac23ec1e9c bower.json
100644 blob 01b7ae4e7fbf8a30d460d617ccb1a2f95c36698d build/gulpfile.common.js
100644 blob 197c437fa90f8f83cae9bd0c0d39880d19c16058 build/shim.js
100644 blob 2348e843f17ac18a11b6959f69a02bb3a5f70b56 package.json
100644 blob a8b7a3768dadec2e9b9caf75790220a11c1549a2 src/common/***.ts
.
.
.
100644 blob 165dd8ed955284685c3cd294b7924ad45cb01e81 src/user/store/model/***.ts
100644 blob 84dc7f9ffc788b1023c4c59e2b006166fd64222d src/user/store/model/***.ts
100644 blob 6319765c2d88bc95a2227778ff338238ec8898d9 src/user/store/***.ts
100644 blob d86a154b7e3d2154ad5205c5a050b277b7d0ec5f src/utils/***.ts
100644 blob c84a271666d50e5a948ae63979e7b99056af0e8e src/utils/service/***.ts
100644 blob 9787db21be335b63cd85553f4e96cb4c07e27e03 src/utils/service/***.ts
100644 blob cc5574da824e0e3eff2e0d338e13be2e3a560407 src/utils/service/***.ts
100644 blob 4b5081da4b8af80a3b6f066e1d586f796eeb0c06 src/utils/service/***.ts
100644 blob d3af339a37eba53ade1b634e51c4a89908dae2c4 tsconfig.json
100644 blob 114bd1231b4464fb0023c9083f9f43a8f3637c10 yarn.lock
</code></pre>
<p>You could optionally <code>md5</code> the result. This output can be used to determine if the TypeScript project needs to be built at all. Course Hero's build process isn't doing this yet, but the next time we wish to increase build times, this should result in a good win.</p>
Announcing Theia2018-09-28T00:00:00Zhttps://hoten.cc/blog/announcing-theia/<p>We at <a href="https://www.coursehero.com/" target="_blank" rel="noopener noreferrer">Course Hero</a> are announcing the open release of <a href="https://github.com/coursehero/theia" target="_blank" rel="noopener noreferrer">Theia</a>, a framework for building, rendering, and caching React applications.</p>
<p>As all in web know, SEO drives growth. And naturally, a page with seemingly no content (from the perspective of an indexing spider) or one with a deferred render is not great for SEO.</p>
<p>In recent years, React has become one of the <a href="https://w3techs.com/technologies/comparison/js-angularjs,js-react,js-vuejs" target="_blank" rel="noopener noreferrer">most popular frontend frameworks</a> for high traffic sites. At Course Hero, all new projects since 2016 have been in React, leaving our Angular 1.3.x projects around until a large enough feature compels us to refactor.</p>
<p>However, Course Hero doesn't have a Node.JS backend (the company predates the runtime by a few years). This means that logged out, critical landing pages - which must be tuned for SEO - can't have the initial view with React on the server. A painful workaround for this is to render the intial view in an alternative way, using the backend's templating language. The initial view generated in both implementations must be congruent, or else there's risk of page flickering. Even then, it's suboptimal, as React would not be able to <a href="https://stackoverflow.com/questions/46516395/whats-the-difference-between-hydrate-and-render-in-react-16" target="_blank" rel="noopener noreferrer">hydrate</a> from the inital, non-React rendered view, resulting in a longer than necessary bootup time. These reasons have steered the majority of projects at Course Hero touching landing pages away from using React.</p>
<p>We could could call out to an external process from our PHP backend to do this initial rendering, but we had a few reasons to not do that. We wanted to avoid installing Node on all of our web servers. We wanted to be able to scale Node rendering independently from our web servers. We wanted deployments for our React applications separated from main deployments to the entire site backend. And we wanted control over caching. To achieve these goals, I began implementing a microservice, Theia, in November 2017, and today we are announcing its open release.</p>
<p><img src="https://hoten.cc/images/theia-slack.png" alt="Theia integrates with Slack" />
<em>Theia integrates with Slack</em>
{:.image-caption}</p>
<p>When paired with a Node.JS backend, rendering the initial view before the browser loads any JavaScript is trivial. However, new adopters to React very often don't utilize a Node.JS backend. There's a fundamental friction point here. That initial rendering from the server is important, so companies that do adopt React either relegate it to logged-in experiences, or they build a rendering service that integrates with their backend. I know of one large company that has done the latter, as Ben Ilegbodu of Eventbrite told me last July at Node Summit. Hopefully, Theia will save some effort for others solving this initial rendering issue, and others will find its additional features and configurability beneficial to development.</p>
<p>As of this writing, Theia powers Course Hero's new <a href="https://www.coursehero.com/sg/" target="_blank" rel="noopener noreferrer">course study guides</a>, and we are considering adopting it in other parts of the site.</p>
Game of Life2020-04-12T00:00:00Zhttps://hoten.cc/blog/game-of-life/<p>I was saddened to hear of John Conway's passing yesterday.</p>
<!-- Excerpt Start -->
<p>Conway's Game of Life is mathematics at play. A grid of cells, each either alive or bare, paired with a simple rule set that determines when new cells become alive and live cells die off, are all the constructs necessary to create a world of patterns and intrigue.</p>
<!-- Excerpt End -->
<p>If a bare cell has 3 live neighbors, it becomes alive. If a live cell has 2 or 3 living neighbors, it remains alive, otherwise it dies off (you could say it died from isolation or from overpopulation). Stick to those rules, and you discover a <a href="https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life#Examples_of_patterns" target="_blank" rel="noopener noreferrer">world of patterns</a> emerging from noise. Tweak the rules slightly, and you likely end up with just the noise. That, to me, is the intriguing bit.</p>
<p>I recall ignoring many boring lectures my freshman year of college (we were probably covering UML diagrams...) to program my own version of Conway's Game of Life. I chose Dart because it was so much like ActionScript (I started programming with Flash). And the fact that it was typed made it seem like the future of web development. I even demoed my code to a very confused hiring manager at JP Morgan ... thankfully for us both (and the world?), he passed on the opportunity to hire me and bring Dart to your Chase banking portal.</p>
<p>You can find my version of Conway's Game of Life <a href="https://connorjclark.github.io/dart-life/#life" target="_blank" rel="noopener noreferrer">here</a>. Excuse the styling - I refuse to fix it for posterity reasons, but it really is horrid (give me a break, it was 7 years ago). I suggest playing around with the configurable rule set. Can you find another rule set that produces interesting patterns?</p>
<p>Obviously the Game of Life is just one of many fascinating things made by John Conway. I look forward to learning more about him from his biography. You can also find some nice words from the comments section on <a href="https://news.ycombinator.com/item?id=22843306" target="_blank" rel="noopener noreferrer">Hacker News</a>.</p>
Black Lives Matter2020-05-31T00:00:00Zhttps://hoten.cc/blog/black-lives-matter/<style type="text/css">/* card */
.tweet-card {
background: white;
margin: 15px 0;
padding: 20px;
background-color: #fff;
border: 1px solid #e1e8ed;
border-radius: 5px;
font-size: 16px;
line-height: 1.4;
font-family: Helvetica, Roboto, "Segoe UI", Calibri, sans-serif;
color: #1c2022;
max-width: 500px;
text-align: left;
}
.tweet-card:hover {
border-color: #ccd6dd;
}
.tweet-card a {
color: #2b7bb9;
text-decoration: none;
}
.tweet-card a:hover {
color: #3b94d9;
}
/* header */
.tweet-header {
display: flex;
}
.tweet-header .tweet-profile {
margin-right: 9px;
flex-shrink: 0;
}
.tweet-header .tweet-profile img {
border-radius: 50%;
height: 36px;
width: 36px;
}
.tweet-header .tweet-author {
display: flex;
flex-direction: column;
flex-grow: 1;
}
.tweet-header .tweet-author-name {
font-weight: 700;
color: #1c2022;
line-height: 1.3;
}
.tweet-header .tweet-author-name:hover {
color: #3b94d9;
}
.tweet-header .tweet-author-handle {
color: #697882;
font-size: 14px;
line-height: 1;
}
.tweet-header .tweet-bird {
margin-left: 20px;
}
/* images */
.tweet-images img {
width: 100%;
max-height: 250px;
object-fit: cover;
margin-bottom: 10px;
border-radius: 4px;
}
/* footer */
.tweet-footer {
display: flex;
align-items: center;
}
.tweet-card .tweet-footer a {
color: #697882;
font-size: 14px;
}
.tweet-footer .tweet-date:hover {
color: #2b7bb9;
}
.tweet-footer .tweet-like {
margin-right: 15px;
font-size: 15px;
display: flex;
align-items: center;
}
.tweet-footer .tweet-like:hover {
color: #e0245e;
}
.tweet-footer .tweet-like-count {
margin-left: 4px;
}
.tweet-footer .tweet-like-icon {
filter: grayscale(1)brightness(1.4);
transition: filter;
}
.tweet-footer .tweet-like:hover .tweet-like-icon {
filter: none;
}
/* icons */
.tweet-bird-icon,
.tweet-like-icon {
display: inline-block;
width: 1.25em;
height: 1.25em;
background-size: contain;
background-repeat: no-repeat;
vertical-align: text-bottom;
}
.tweet-bird-icon {
background-image: url(data:image/svg+xml;charset=utf-8,%3Csvg%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20viewBox%3D%220%200%2072%2072%22%3E%3Cpath%20fill%3D%22none%22%20d%3D%22M0%200h72v72H0z%22%2F%3E%3Cpath%20class%3D%22icon%22%20fill%3D%22%231da1f2%22%20d%3D%22M68.812%2015.14c-2.348%201.04-4.87%201.744-7.52%202.06%202.704-1.62%204.78-4.186%205.757-7.243-2.53%201.5-5.33%202.592-8.314%203.176C56.35%2010.59%2052.948%209%2049.182%209c-7.23%200-13.092%205.86-13.092%2013.093%200%201.026.118%202.02.338%202.98C25.543%2024.527%2015.9%2019.318%209.44%2011.396c-1.125%201.936-1.77%204.184-1.77%206.58%200%204.543%202.312%208.552%205.824%2010.9-2.146-.07-4.165-.658-5.93-1.64-.002.056-.002.11-.002.163%200%206.345%204.513%2011.638%2010.504%2012.84-1.1.298-2.256.457-3.45.457-.845%200-1.666-.078-2.464-.23%201.667%205.2%206.5%208.985%2012.23%209.09-4.482%203.51-10.13%205.605-16.26%205.605-1.055%200-2.096-.06-3.122-.184%205.794%203.717%2012.676%205.882%2020.067%205.882%2024.083%200%2037.25-19.95%2037.25-37.25%200-.565-.013-1.133-.038-1.693%202.558-1.847%204.778-4.15%206.532-6.774z%22%2F%3E%3C%2Fsvg%3E);
}
.tweet-like-icon {
background-image: url(data:image/svg+xml;charset=utf-8,%3Csvg%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20width%3D%2224%22%20height%3D%2224%22%20viewBox%3D%220%200%2024%2024%22%3E%3Cpath%20class%3D%22icon%22%20fill%3D%22%23E0245E%22%20d%3D%22M12%2021.638h-.014C9.403%2021.59%201.95%2014.856%201.95%208.478c0-3.064%202.525-5.754%205.403-5.754%202.29%200%203.83%201.58%204.646%202.73.813-1.148%202.353-2.73%204.644-2.73%202.88%200%205.404%202.69%205.404%205.755%200%206.375-7.454%2013.11-10.037%2013.156H12zM7.354%204.225c-2.08%200-3.903%201.988-3.903%204.255%200%205.74%207.035%2011.596%208.55%2011.658%201.52-.062%208.55-5.917%208.55-11.658%200-2.267-1.822-4.255-3.902-4.255-2.528%200-3.94%202.936-3.952%202.965-.23.562-1.156.562-1.387%200-.015-.03-1.426-2.965-3.955-2.965z%22%2F%3E%3C%2Fsvg%3E);
}</style>
<style type="text/css">
h2 {
position: sticky;
width: 100%;
height: 50px;
top: 0;
background-color: var(--body-background-color);
border-bottom: 1px black solid;
z-index: 1;
max-width: 100%; width: 500px; min-width: 221px;
display: flex;
align-items: center;
justify-content: center;
text-align: center;
}
</style>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266952531793416192"></a></blockquote>
<p>It's been tough viewing the footage from around the country these past few days. My goal with this page is to be a resource to send folks that may not be very tuned-into social media.</p>
<blockquote>
<p>Let me say as I've always said, and I will always continue to say, that riots are socially destructive and self-defeating. ... But in the final analysis, a riot is the language of the unheard. And what is it that America has failed to hear? It has failed to hear that the plight of the Negro poor has worsened over the last few years. It has failed to hear that the promises of freedom and justice have not been met. And it has failed to hear that large segments of white society are more concerned about tranquility and the status quo than about justice, equality, and humanity. And so in a real sense our nation's summers of riots are caused by our nation's winters of delay. And as long as America postpones justice, we stand in the position of having these recurrences of violence and riots over and over again [–Martin Luther King Jr.]</p>
</blockquote>
<p>Following, you'll find a collection of videos from the past few days. Please, take the time to listen to the unheard; see your fellow Americans protest against a system of oppression and injustice; witness the media, citizens, and even our representatives attacked while they excericse their right to protest. Understand that, although recently magnified, this is just a glimpse into a reality that has existed for hundreds of years in this country.</p>
<p>But first, <a href="https://www.washingtonpost.com/news/powerpost/paloma/powerup/2020/05/28/powerup-racism-and-police-violence-in-spotlight-at-crucial-time-in-2020-race/5eced36688e0fa32f822be79/" target="_blank" rel="noopener noreferrer">learn</a> that unarmed Black people are much more likely to be killed than unarmed whites; <a href="https://mappingpoliceviolence.org/unarmed/" target="_blank" rel="noopener noreferrer">learn</a> that cops rarely see any jail time for murdering unarmed Black people; and <a href="https://www.npr.org/2020/05/29/865261916/a-decade-of-watching-black-people-die" target="_blank" rel="noopener noreferrer">learn</a> about the final moments of the men, women, and children whose lives were taken away by an overzealous, often racist, and at best incompetent cop.</p>
<p>Note: Please consider donating to a cause listed in this tweet (click to see). Check with your employer to see if they will match anything you give. I chose to donate $1000 to the <a href="https://naacp.org/make-donation-naacp/" target="_blank" rel="noopener noreferrer">NAACP</a>, which my employer matched.</p>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266328844056825857"></a></blockquote>
<h2 id="listen" tabindex="-1">Listen</h2>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266921926905286656"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266960975124717568"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1267207824070893568"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266641724568453120"></a></blockquote>
<h2 id="peaceful-protests" tabindex="-1">Peaceful protests</h2>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266911950979706880"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266863303978074112"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1267250256686788608"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266895422397779968"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266815884737249280"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266954595437350912"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266844355148632064"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1267476355433877504"></a></blockquote>
<blockquote class="reddit-card" data-card-created="1590964858"><a href="https://www.reddit.com/r/houston/comments/gto06m/peaceful_protest_on_holman/">Peaceful Protest on Holman</a> from <a href="http://www.reddit.com/r/houston">r/houston</a></blockquote>
<script async="" src="https://embed.redditmedia.com/widgets/platform.js" charset="UTF-8"></script>
<h2 id="journalists-and-recorders-attacked" tabindex="-1">Journalists and recorders attacked</h2>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266384227492335616"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266933018570219520"></a></blockquote>
Note: John Cusack shares video of being attacked by cops for filming.
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266938983575101441"></a></blockquote>
Note: Photojournalist Linda Tirado blinded in Minneapolis protests.
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266618525600399361"></a></blockquote>
<p>Note: Self-declared "southern journalist" <a href="https://twitter.com/BeauTFC" target="_blank" rel="noopener noreferrer">Beau</a> reaches out.</p>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266975560040714241"></a></blockquote>
<h2 id="police-hiding-badges" tabindex="-1">Police hiding badges</h2>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266897442148102150"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266948928966139904"></a></blockquote>
<h2 id="police-violence" tabindex="-1">Police violence</h2>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266928343141752833"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266885769282584576"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266978979212406784"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266884475268616197"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266945268567678976"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266915712343453697"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266921821653385225"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266540710188195843"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266888294115262466"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266559402225647616"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266926795909144576"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266891847000920065"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266883979342417926"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1267002256974479360"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266989439160590336"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266942703369105409"></a></blockquote>
Note: @cmclymer compiles a list.
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266807973780885506"></a></blockquote>
<h2 id="opportunists" tabindex="-1">Opportunists</h2>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266932254682595331"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266598693647638528"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266734357705887748"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266826666141192192"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266939945270210560"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266918612708851713"></a></blockquote>
<h2 id="outside-the-white-house" tabindex="-1">Outside the White House</h2>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266907885952536579"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266898124477534210"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266941784439087105"></a></blockquote>
<h2 id="riot-control" tabindex="-1">Riot control</h2>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266863435163328512"></a></blockquote>
<h2 id="wtf" tabindex="-1">wtf</h2>
<p>Note: Milk and water are used to douse a victim of pepper spray. Cops stand by / actively allow for these supplies to be stolen.</p>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266892603577925633"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266898825563815940"></a></blockquote>
<p>Note: John Cusack shares a scene in NY that looks like The Joker</p>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266976983499579392"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266570793833512961"></a></blockquote>
<p>Note: Two men pull back 70 year old congresswoman Joyce Beatty as cops lose control of a crowd and begin pepper spraying.</p>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266821475698462720"></a></blockquote>
<p>Note: not a cop, private security for news reporters. Kid supposedly gnabbed an AR-15 from a cop vehicle.</p>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266880676176908288"></a></blockquote>
<blockquote class="twitter-tweet"><a href="https://twitter.com/user/status/1266875751552278528"></a></blockquote>
<blockquote class="reddit-card" data-card-created="1590965049"><a href="https://www.reddit.com/r/ActualPublicFreakouts/comments/gtfe3c/man_follows_his_car_after_the_national_guard_take/">Man Follows his car after the National Guard take it from him during the Minnesota Protests.</a> from <a href="http://www.reddit.com/r/ActualPublicFreakouts">r/ActualPublicFreakouts</a></blockquote>
<script async="" src="https://embed.redditmedia.com/widgets/platform.js" charset="UTF-8"></script>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<script>
const observer = new MutationObserver(function(mutations) {
mutations.forEach(function(mutation) {
if (mutation.attributeName === "class") {
// NOTE: changing theme attribute doesn't update twitter embedded. Instead, just reload page if theme changes.
// const isDark = document.documentElement.classList.contains('dark');
// for (const el of document.querySelectorAll('.twitter-tweet')) {
// el.setAttribute('data-theme', isDark ? 'dark' : 'light');
// }
window.location.reload();
}
});
});
observer.observe(document.documentElement, {
attributes: true
});
const isDark = document.documentElement.classList.contains('dark');
for (const el of document.querySelectorAll('.twitter-tweet')) {
el.setAttribute('data-theme', isDark ? 'dark' : 'light');
}
</script>
KB vs KB2020-07-08T00:00:00Zhttps://hoten.cc/blog/kb-vs-kb/<p>TL;DR: As of v6.0, Lighthouse now uses KiB (=1024 bytes) because it is unambigous. Chrome DevTools moved to kB (=1000 bytes), faster than intended, but maybe <a href="https://randomascii.wordpress.com/2016/02/13/base-ten-for-almost-everything/" target="_blank" rel="noopener noreferrer">for the best</a>?</p>
<p>A KiB (kibibyte, 1024 bytes) is the same as kilobyte (KB, 1024 bytes), unless you mean kilobyte (kB, 1000 bytes). Confused? Read more on <a href="https://en.wikipedia.org/wiki/Kibibyte" target="_blank" rel="noopener noreferrer">wikipedia</a>.</p>
<p>On Mac a KB is 1000 bytes. On Windows a KB is 1024 bytes.</p>
<p>Where did the <code>kibi</code> in kibibyte come from? It's a portmanteau–<code>kilo</code> and <code>binary</code>. Cute. It was also chosen due to its proximity to the SI unit “kilo”, guaranteeing confusion.</p>
<p>If you sell things you want to seem bigger, you use kB. If you prefer your maths in base 2, you use KB, and you're probably a computer. If you bought a TB hard drive and find yourself with 68 fewer GBs of storage than expected, you got played.</p>
<p>In an alternate universe, a kibibyte is called a KKB (large kilobyte). Valiant effort, <a href="https://en.wikipedia.org/wiki/Kibibyte#cite_ref-10:~:text=Donald%20Knuth" target="_blank" rel="noopener noreferrer">Knuth</a>.</p>
<p>This is what happens when you define unit prefixes based on the "convenience" of being close to an existing standard. kilo means 1000, unless it actually means 1024, which surely is close enough. The difference starts at 24, grows with every increasing unit, and results in endless confusion.</p>
<p>Given three wishes, I'd first wish to hit reset on all this mess and declare KB = kilobyte = 1000 bytes, and kB = kibibyte = 1024 bytes. My next wish wouldn't matter, because the first was actually two wishes and the genie was speaking in kilowishes, where kilo is derived from "two". It's close enough.</p>
<p>In most developer tooling, a KB is 1024 bytes. If you've used more than one type of computer or software in your lifetime, you'll probably find yourself doublethinking as to what a KB means based on the context you are currently in.</p>
<p>Thanks for reading. If there's one takeaway, let it be this: KiB = 1024 bytes. Please remember that when you see it in Lighthouse (as of 6.0). The unit value isn't really changing–it's just getting an unambiguous label. See the <a href="https://github.com/GoogleChrome/lighthouse/pull/10870" target="_blank" rel="noopener noreferrer">Lighthouse PR</a>.</p>
<p>In a similar vein, Chrome DevTools now uses kB for the base 10 unit. It was a bit of an accident; you can learn more from the <a href="https://docs.google.com/document/d/1TWn4kpXlN-W_LmuZGQ9Iv7pmK4R7zgmhKkZ9gF2CsrE/edit?usp=sharing" target="_blank" rel="noopener noreferrer">design doc</a>.</p>
Porting Zelda Classic to the Web2022-04-29T00:00:00Zhttps://hoten.cc/blog/porting-zelda-classic-to-the-web/<style>
.captioned-image {
display: flex;
flex-direction: column;
justify-content: center;
align-items: center;
margin: 25px 0;
}
.captioned-image img {
/* width: 50%; */
}
img {
image-rendering: pixelated;
}
.captioned-image span {
text-align: center;
}
.sticky {
position: sticky;
top: 0;
font-size: 1.17em;
font-weight: bold;
z-index: 2;
}
</style>
<script>
const stickyEl = document.createElement('div');
stickyEl.classList.add('sticky');
document.querySelector('article').append(stickyEl);
function updateSticky() {
if (window.innerWidth < 900) {
stickyEl.hidden = true;
return;
}
stickyEl.hidden = false;
const elements = [...document.querySelectorAll('h1, h2, h3')];
const closest = elements.reduce((acc, cur) => {
if (document.documentElement.scrollTop - cur.offsetTop + 70 < 0) return acc;
return document.documentElement.scrollTop - cur.offsetTop > document.documentElement.scrollTop - acc.offsetTop ? acc : cur;
});
stickyEl.hidden = closest === elements[0];
stickyEl.textContent = closest.textContent;
}
document.addEventListener('scroll', updateSticky);
document.addEventListener('hashchange', updateSticky);
</script>
<blockquote>
<p>Nov 27, 2023: Much has changed since this article was published. I've become far more involved with ZC development; the name of the program is now ZQuest Classic; our website is <a href="https://zquestclassic.com/" target="_blank" rel="noopener noreferrer">zquestclassic.com</a>; and the web version discussed in this article is now hosted at <a href="https://web.zquestclassic.com/play/" target="_blank" rel="noopener noreferrer">web.zquestclassic.com</a></p>
</blockquote>
<div class="captioned-image">
<img src="https://hoten.cc/images/zc/Mitchfork.png" alt="" />
<span>Mitchfork's winning screenshot from the <a href="https://www.purezc.net/forums/index.php?showtopic=77409" target="_blank">2021 Screenshot of the Year contest</a></span>
</div>
<!-- Excerpt Start -->
<p>I ported Zelda Classic (a game engine based on the original Zelda) to the web. You can play it <a href="https://web.zquestclassic.com/play/" target="_blank" rel="noopener noreferrer">here</a>–grab a gamepad if you have one!</p>
<p>It's a PWA, so you can also install it.</p>
<!-- Excerpt End -->
<p>I've written some <a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#zelda-classic">background information</a> on Zelda Classic, and chronicled <a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#porting-zelda-classic-to-the-web">the technical process</a> of porting a large C++ codebase to the web using WebAssembly.</p>
<blockquote>
<p>Follow <a href="https://twitter.com/zelda_classic" target="_blank" rel="noopener noreferrer">Zelda Classic on Twitter</a>!</p>
</blockquote>
<nav class="table-of-contents"><ol><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#zelda-classic">Zelda Classic</a><ol><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#on-the-web">On the Web</a></li></ol></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#porting-zelda-classic-to-the-web">Porting Zelda Classic to the Web</a><ol><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#getting-it-working">Getting it working</a><ol><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#emscripten">Emscripten</a></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#starting-off">Starting off</a></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#learning-cmake-allegro-and-emscripten">Learning CMake, Allegro, and Emscripten</a></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#allegro-legacy">Allegro Legacy</a></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#starting-to-build-zelda-classic-with-emscripten">Starting to build Zelda Classic with Emscripten</a></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#let-there-be-threads">Let there be threads</a></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#of-mutexes-and-deadlocks">Of mutexes and deadlocks</a></li></ol></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#getting-it-fully-functional">Getting it fully functional</a><ol><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#playing-midi-with-timidity">Playing MIDI with Timidity</a></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#music-working-but-no-sfx">Music working, but no SFX?</a></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#build-script-hacking">Build script hacking</a></li></ol></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#making-it-awesome">Making it awesome</a><ol><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#quest-list">Quest List</a></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#mp3s-and-oggs-and-retro-music">MP3s, and OGGs and retro music</a></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#persisting-data">Persisting data</a></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#gamepads">Gamepads</a></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#mobile-support">Mobile support</a></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#pwa">PWA</a></li></ol></li><li><a href="https://hoten.cc/blog/porting-zelda-classic-to-the-web/#takeaways">Takeaways</a></li></ol></li></ol></nav><h1 id="zelda-classic" tabindex="-1">Zelda Classic</h1>
<a href="https://web.zquestclassic.com/create/?open=quests/purezc/773/r01/1st-mirrored-vertical-and-horizontal.qst&map=0&screen=8" target="_blank">
<div class="captioned-image">
<img style="max-width: min(700px, 100%)" src="https://hoten.cc/images/zc/editor.png" alt="ZQuest editor opened to the starting screen of the original Zelda (but mirrored!)" />
<span>ZQuest, the Zelda Classic quest editor</span>
</div>
</a>
<p><a href="https://www.zeldaclassic.com/" target="_blank" rel="noopener noreferrer">Zelda Classic</a> is a 20+ year old game engine originally made to recreate and modify the original Legend of Zelda. The engine grew to support far more features than what was necessary to create the original game, and today there are <a href="https://www.purezc.net/index.php?page=quests&sort=rating" target="_blank" rel="noopener noreferrer">over 600</a> custom games - the community calls them quests.</p>
<p>Many are spiritual successors to the original, perhaps with improved graphics, but very recognizable as a Zelda game. They range in complexity, quality and length. Fair warning, some are just awful, so be discerning and use the rating to guide you.</p>
<p>If you are a fan of the original 2D Zelda games, I believe you'll find many Zelda Classic quests to be well worth your time. Some are 20+ hour games with expansive overworlds and engaging, unique dungeons. The engine today supports scripting, and many have used that to push it to the limits: it's almost impossible to believe that some quests implemented character classes, online networking, or achievements in an engine meant to create the original Zelda.</p>
<p>However, the most recent version of Zelda Classic only supports Windows... until now!</p>
<h2 id="on-the-web" tabindex="-1">On the Web</h2>
<p>I spent the last two months (roughly ~150 hours) porting Zelda Classic to run in a web browser.</p>
<p>There's a lot of quests to choose from, but here's just a small sampling! Click any of these to jump into the quest:</p>
<style>
.grid-col2 {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 10px;
align-items: baseline;
}
@media only screen and (max-width: 768px) {
.grid-col2 {
display: block;
}
}
</style>
<div class="grid-col2">
<!-- <a href="https://web.zquestclassic.com/play/?quest=bs3.1/NewBS+3.1+-+1st+Quest.qst" target="_blank">
<div class="captioned-image">
<img src="https://hoten.cc/quest-maker/play/zc_quests/bs3.1/image1.png" alt="" width="100%">
<span>BS Zelda 1st Quest</span>
</div>
</a> -->
<a href="https://web.zquestclassic.com/play/?open=quests/purezc/373&name=Links-Quest-for-the-Hookshot-2-Quest" target="_blank">
<div class="captioned-image">
<img src="https://hoten.cc/images/zc/hookshot-title.png" alt="" width="100%" />
<span>Link's Quest for the Hookshot 2</span>
</div>
</a>
<a href="https://web.zquestclassic.com/play/?open=quests/purezc/139" target="_blank">
<div class="captioned-image">
<img src="https://hoten.cc/images/zc/hod.gif" alt="" width="100%" />
<span>Hero of Dreams</span>
</div>
</a>
<!-- <a href="https://web.zquestclassic.com/play/?open=quests/purezc/731" target="_blank">
<div class="captioned-image">
<img src="/images/zc/gollab.png" alt="" width="100%">
<span>Go Gollab: The Conflictions of Morality</span>
</div>
</a> -->
<a href="https://web.zquestclassic.com/play/?open=quests/purezc/751" target="_blank">
<div class="captioned-image">
<img src="https://hoten.cc/images/zc/new-legacy.png" alt="" width="100%" />
<span>Legend of Link: The New Legacy</span>
</div>
</a>
<a href="https://web.zquestclassic.com/play/?open=quests/purezc/152&name=Castle-Haunt-II" target="_blank">
<div class="captioned-image">
<img src="https://hoten.cc/quest-maker/play/zc_quests/152/image0.gif" alt="" width="100%" />
<span>Castle Haunt II</span>
</div>
</a>
</div>
<p>I hope my efforts result in Zelda Classic reaching a larger audience. It's been challenging work, far outside my comfort zone of web development, and I've learned a lot about WebAssembly, CMake and multithreading. Along the way, I discovered bugs across multiple projects and did due diligence in fixing (or just reporting) them when I could, and even proposed a <a href="https://github.com/whatwg/html/issues/7838" target="_blank">change to the HTML spec</a>.</p>
<h1 id="porting-zelda-classic-to-the-web" tabindex="-1">Porting Zelda Classic to the Web</h1>
<p>The rest of this article is an overview of the technical process of porting Zelda Classic to the web.</p>
<blockquote>
<p>If you're interested in the minutia, I've made <a href="https://docs.google.com/document/d/1tOI1k9nSWDxmHXoW-yy4fk3_7AbS6vCk3UUG2iCwS_g" target="_blank" rel="noopener noreferrer">my daily notes</a> available. This was the first time I kept notes like this, and I found the process improved my working memory significantly... and it definitely helped me write this article.</p>
</blockquote>
<h2 id="getting-it-working" tabindex="-1">Getting it working</h2>
<h3 id="emscripten" tabindex="-1">Emscripten</h3>
<p><a href="https://emscripten.org/" target="_blank" rel="noopener noreferrer">Emscripten</a> is a compiler toolchain for building C/C++ to WebAssembly. The very TL;DR of how it works is that it uses <code>clang</code> to transform the resultant LLVM bytecode to Wasm. It's not enough to just compile code to Wasm–Emscripten also provides Unix runtime capabilities by implementing them with JavaScript/Web APIs (ex: implementations for most syscalls; an in-memory or IndexedDB-backed filesystem; pthreads support via Web Workers). Because many C/C++ projects are built with Make and CMake, Emscripten also provides tooling for interoping with those tools: <code>emmake</code> and <code>emcmake</code>. For the most part, if a C/C++ program is portable, it can be built with Emscripten and run in a browser, although you'll like have to make changes to <a href="https://emscripten.org/docs/porting/emscripten-runtime-environment.html#emscripten-runtime-environment" target="_blank" rel="noopener noreferrer">accommodate the browser main loop</a>.</p>
<blockquote>
<p>If you are developing a Wasm application, the Chrome DevTools DWARF extension is essential. See <a href="https://developer.chrome.com/blog/wasm-debugging-2020/" target="_blank" rel="noopener noreferrer">this article</a> for how to use it. When it works, it's excellent. You may need to drop any optimization for best results. Even with no optimization pass, I often ran into cases where some frames of the call stacktrace were obviously wrong, so I sometimes had to resort to printf-style debugging.</p>
</blockquote>
<h3 id="starting-off" tabindex="-1">Starting off</h3>
<p>Zelda Classic is written in C++ and uses Allegro, a low-level cross platform library for window management, drawing to the screen, playing sounds, etc. Well, it actually uses Allegro 4, released circa 2007. Allegro 4 does not readily compile with Emscripten, but Allegro 5 does. The two versions are vastly different but fortunately there is an adapter library called Allegro Legacy which allows an Allegro 4 application to be built using Allegro 5.</p>
<p>So that's the first hurdle–Zelda Classic needs to be ported to Allegro 5, and its CMakeLists.txt needs to be modified to build allegro from source.</p>
<blockquote>
<p>Allegro 5 is able to support building with Emscripten because it can use <a href="https://github.com/libsdl-org/SDL" target="_blank" rel="noopener noreferrer">SDL</a> as its backend, which Emscripten supports well.</p>
</blockquote>
<p>Before working on any of that directly, I needed to address my lack of knowledge of CMake and Allegro.</p>
<h3 id="learning-cmake-allegro-and-emscripten" tabindex="-1">Learning CMake, Allegro, and Emscripten</h3>
<p>Allegro claims to support Emscripten, but I wanted to confirm it for myself. Luckily they provided some <a href="https://github.com/liballeg/allegro5/blob/master/README_sdl.txt#L30" target="_blank" rel="noopener noreferrer">instructions</a> on how to build with Emscripten. My first PRs were to Allegro to improve this documentation.</p>
<blockquote>
<p>I wasted a few hours here because of an <a href="https://github.com/liballeg/allegro5/pull/1319" target="_blank" rel="noopener noreferrer">unfortunate difference</a> between bash and zsh.</p>
</blockquote>
<p>Next I found an interesting example program showcasing palette swapping–encoding a bitmap as indices into an arbitrary set of colors, which can be swapped out at runtime. But, it didn't work when built with Emscripten. To get a little practice with Allegro, I worked on improving this example.</p>
<p>The fragment shader:</p>
<pre class="language-glsl"><code class="language-glsl"><span class="token keyword">uniform</span> <span class="token keyword">sampler2D</span> al_tex<span class="token punctuation">;</span><br /><span class="token keyword">uniform</span> <span class="token keyword">vec3</span> pal<span class="token punctuation">[</span><span class="token number">256</span><span class="token punctuation">]</span><span class="token punctuation">;</span><br /><span class="token keyword">varying</span> <span class="token keyword">vec4</span> varying_color<span class="token punctuation">;</span><br /><span class="token keyword">varying</span> <span class="token keyword">vec2</span> varying_texcoord<span class="token punctuation">;</span><br /><span class="token keyword">void</span> <span class="token function">main</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br /><span class="token punctuation">{</span><br /> <span class="token keyword">vec4</span> c <span class="token operator">=</span> <span class="token function">texture2D</span><span class="token punctuation">(</span>al_tex<span class="token punctuation">,</span> varying_texcoord<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">int</span> index <span class="token operator">=</span> <span class="token keyword">int</span><span class="token punctuation">(</span>c<span class="token punctuation">.</span>r <span class="token operator">*</span> <span class="token number">255.0</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>index <span class="token operator">!=</span> <span class="token number">0</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> gl_FragColor <span class="token operator">=</span> <span class="token keyword">vec4</span><span class="token punctuation">(</span>pal<span class="token punctuation">[</span>index<span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token keyword">else</span> <span class="token punctuation">{</span><br /> gl_FragColor <span class="token operator">=</span> <span class="token keyword">vec4</span><span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Allegro passes a bitmap's texture to the shader as <code>al_tex</code>, and in this program that bitmap is just a bunch of numbers 0-255. Attached to the shader as an input is a palette of colors <code>pal</code>, and at runtime the program swaps out the palette, changing the colors rendered by the shader. There were two things wrong here that results in this shader not working in WebGL:</p>
<ol>
<li>It lacks a precision declaration. In WebGL, this is not optional. Very simple fix–just add <code>precision mediump float;</code></li>
<li>It uses a non-constant expression to index an array. WebGL does not support that, so the entire shader needed to be redesigned. This was more involved, so I'll just link to the <a href="https://github.com/liballeg/allegro5/pull/1318" target="_blank" rel="noopener noreferrer">PR</a></li>
</ol>
<p>The resulting program is hosted <a href="https://tedious-porter.surge.sh/ex_palette.html" target="_blank" rel="noopener noreferrer">here</a>.</p>
<blockquote>
<p>It turned out that none of this knowledge of how to do palette swapping in Allegro 5 would be necessary for upgrading Zelda Classic's Allegro, although
initially I thought it might. Still, it was a nice introduction to the library.</p>
</blockquote>
<p>Next I wanted to write a simple <code>CMakeLists.txt</code> that I could wrap my head around, one that builds Allegro from source and also supports building with Emscripten.</p>
<blockquote>
<p>Emscripten supports building projects configured with CMake via <a href="https://github.com/Emscripten-core/Emscripten/blob/main/emcmake.py" target="_blank" rel="noopener noreferrer"><code>emcmake</code></a>, which is a small program that configures an Emscripten CMake <a href="https://github.com/Emscripten-core/Emscripten/blob/main/cmake/Modules/Platform/Emscripten.cmake" target="_blank" rel="noopener noreferrer">toolchain</a>. Essentially, running <code>emcmake cmake <path/to/source></code> configures the build to use <code>emcc</code> as the compiler.</p>
</blockquote>
<p>I spent some time reading many tutorials on CMake, going through real-world <code>CMakeLists.txt</code> and trying to understand it all line-by-line. The CMake <a href="https://cmake.org/cmake/help/latest/" target="_blank" rel="noopener noreferrer">documentation</a> was excellent during this process. Eventually, I ended up with this:</p>
<p><a href="https://github.com/connorjclark/allegro-project/blob/main/CMakeLists.txt" target="_blank" rel="noopener noreferrer"><code>https://github.com/connorjclark/allegro-project/blob/main/CMakeLists.txt</code></a></p>
<pre class="language-cmake"><code class="language-cmake"><span class="token keyword">cmake_minimum_required</span><span class="token punctuation">(</span><span class="token property">VERSION</span> <span class="token number">3.5</span><span class="token punctuation">)</span><br /><span class="token keyword">project</span> <span class="token punctuation">(</span>AllegroProject<span class="token punctuation">)</span><br /><span class="token keyword">include</span><span class="token punctuation">(</span>FetchContent<span class="token punctuation">)</span><br /><br /><span class="token function">FetchContent_Declare</span><span class="token punctuation">(</span><br /> allegro5<br /> GIT_REPOSITORY https://github.com/liballeg/allegro5.git<br /> GIT_TAG <span class="token number">5.2.7.0</span><br /><span class="token punctuation">)</span><br /><span class="token function">FetchContent_GetProperties</span><span class="token punctuation">(</span>allegro5<span class="token punctuation">)</span><br /><span class="token keyword">if</span><span class="token punctuation">(</span><span class="token operator">NOT</span> allegro5_POPULATED<span class="token punctuation">)</span><br /> <span class="token function">FetchContent_Populate</span><span class="token punctuation">(</span>allegro5<span class="token punctuation">)</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token variable">MSVC</span><span class="token punctuation">)</span><br /> <span class="token keyword">set</span><span class="token punctuation">(</span><span class="token namespace">SHARED</span> <span class="token boolean">ON</span><span class="token punctuation">)</span><br /> <span class="token keyword">else</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br /> <span class="token keyword">set</span><span class="token punctuation">(</span><span class="token namespace">SHARED</span> <span class="token boolean">OFF</span><span class="token punctuation">)</span><br /> <span class="token keyword">endif</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br /> <span class="token keyword">set</span><span class="token punctuation">(</span>WANT_TESTS <span class="token boolean">OFF</span><span class="token punctuation">)</span><br /> <span class="token keyword">set</span><span class="token punctuation">(</span>WANT_EXAMPLES <span class="token boolean">OFF</span><span class="token punctuation">)</span><br /> <span class="token keyword">set</span><span class="token punctuation">(</span>WANT_DEMO <span class="token boolean">OFF</span><span class="token punctuation">)</span><br /> <span class="token keyword">add_subdirectory</span><span class="token punctuation">(</span><span class="token punctuation">${</span><span class="token variable">allegro5_SOURCE_DIR</span><span class="token punctuation">}</span> <span class="token punctuation">${</span><span class="token variable">allegro5_BINARY_DIR</span><span class="token punctuation">}</span> <span class="token property">EXCLUDE_FROM_ALL</span><span class="token punctuation">)</span><br /><span class="token keyword">endif</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br /><br /><span class="token keyword">add_executable</span><span class="token punctuation">(</span>al_example src/main.c<span class="token punctuation">)</span><br /><span class="token keyword">target_include_directories</span><span class="token punctuation">(</span>al_example <span class="token namespace">PUBLIC</span> <span class="token punctuation">${</span><span class="token variable">allegro5_SOURCE_DIR</span><span class="token punctuation">}</span>/include<span class="token punctuation">)</span><br /><span class="token keyword">target_include_directories</span><span class="token punctuation">(</span>al_example <span class="token namespace">PUBLIC</span> <span class="token punctuation">${</span><span class="token variable">allegro5_BINARY_DIR</span><span class="token punctuation">}</span>/include<span class="token punctuation">)</span><br /><span class="token keyword">target_link_libraries</span><span class="token punctuation">(</span>al_example LINK_PUBLIC allegro allegro_main allegro_font allegro_primitives<span class="token punctuation">)</span><br /><br /><span class="token comment"># These include files are typically copied into the correct places via allegro's install</span><br /><span class="token comment"># target, but we do it manually.</span><br /><span class="token keyword">file</span><span class="token punctuation">(</span>COPY <span class="token punctuation">${</span><span class="token variable">allegro5_SOURCE_DIR</span><span class="token punctuation">}</span>/addons/font/allegro5/allegro_font.h<br /> DESTINATION <span class="token punctuation">${</span><span class="token variable">allegro5_SOURCE_DIR</span><span class="token punctuation">}</span>/include/allegro5<br /><span class="token punctuation">)</span><br /><span class="token keyword">file</span><span class="token punctuation">(</span>COPY <span class="token punctuation">${</span><span class="token variable">allegro5_SOURCE_DIR</span><span class="token punctuation">}</span>/addons/primitives/allegro5/allegro_primitives.h<br /> DESTINATION <span class="token punctuation">${</span><span class="token variable">allegro5_SOURCE_DIR</span><span class="token punctuation">}</span>/include/allegro5<br /><span class="token punctuation">)</span></code></pre>
<blockquote>
<p>This could have been simpler, but Allegro's <code>CMakeLists.txt</code> requires a <a href="https://github.com/liballeg/allegro5/issues/1328" target="_blank" rel="noopener noreferrer">few modifications</a> for it to be easily consumed as a dependency.</p>
</blockquote>
<p>Initally I tried using CMake's <a href="https://cmake.org/cmake/help/latest/module/ExternalProject.html" target="_blank" rel="noopener noreferrer"><code>ExternalProject</code></a> instead of <a href="https://cmake.org/cmake/help/latest/module/FetchContent.html" target="_blank" rel="noopener noreferrer"><code>FetchContent</code></a>, but the former was problematic with Emscripten because it runs <code>cmake</code> under the hood, and it seemed like it was not aware of the toolchain that <code>emcmake</code> provides. I don't know why I couldn't get it to work, but I know <code>FetchContent</code> is the newer of the two and I had better luck with it.</p>
<h3 id="allegro-legacy" tabindex="-1">Allegro Legacy</h3>
<p>Allegro 4 and 5 can be considered entirely different libraries:</p>
<ul>
<li>literally every API was rewritten, and not in a 1:1 way</li>
<li>A4 uses polling for events while A5 uses event queues / loops</li>
<li>A4 only supports software rendering, and directly supports palettes (which ZC makes heavy use of); while A5 supports shaders / GPU-accelerated rendering (but dropped palette manipulation)</li>
<li>And most importantly for my concerns, only A5 can be compiled with Emscripten (trivially, because of its SDL support)</li>
</ul>
<p>Replacing calls to A4's API with A5 essentially means a rewrite, and given the size of Zelda Classic that was not an option. Fortunately, this is where <a href="https://github.com/NewCreature/Allegro-Legacy" target="_blank" rel="noopener noreferrer">Allegro Legacy</a> steps in.</p>
<p>To support multiple platforms, Allegro abstracts anything OS-specific to a "system driver". There is one for each supported platform that implements low-level operations like filesystem access, window management, etc. Allegro Legacy bridges the gap between A4 and A5 by creating a system driver <em>that uses A5</em> to implement A4's system interfaces. In other words, Allegro Legacy is just A4 with A5 as its driver. All the files in <a href="https://github.com/NewCreature/Allegro-Legacy/tree/master/src" target="_blank" rel="noopener noreferrer"><code>src</code></a> are just A4 (with a few modifications), except for the <code>a5</code> folder which provides the A5 implementation.</p>
<p>This is the entire architecture of running Zelda Classic in a browser:</p>
<div class="captioned-image">
<img style="max-width: min(700px, 100%)" src="https://hoten.cc/images/zc/ascii-arch.png" alt="ASCII diagram of Zelda Classic running on the web" />
<span>🐢 I've fixed/worked-around bugs in every layer of this.</span>
</div>
<p>I used my newly wrangled working knowledge of CMake to configure Zelda Classic's <code>CMakeLists.txt</code> to build Allegro 5 & Allegro Legacy from source. Allegro Legacy was very nearly a drop-in replacement. I struggled initially with an "unresolved symbol" linker error, for a function I was certain was being included in the compilation, but this turned out to be a simple <a href="https://github.com/NewCreature/Allegro-Legacy/pull/23" target="_blank" rel="noopener noreferrer">oversight</a> in a header file. Not really being a C/C++ guy this took me <em>way</em> too long to debug!</p>
<p>Once things actually linked and compilation was successful, Allegro Legacy <em>just worked</em>, although I fixed some minor bugs related to <a href="https://github.com/NewCreature/Allegro-Legacy/pull/21" target="_blank" rel="noopener noreferrer">sticky mouse input</a> and <a href="https://github.com/NewCreature/Allegro-Legacy/pull/24" target="_blank" rel="noopener noreferrer">file paths</a>.</p>
<blockquote>
<p>I sent a <a href="https://github.com/ArmageddonGames/ZeldaClassic/pull/774" target="_blank" rel="noopener noreferrer">PR for upgrading to Allegro 5</a> to the Zelda Classic repro, but I expect it will remain unmerged until a future major release.</p>
</blockquote>
<h3 id="starting-to-build-zelda-classic-with-emscripten" tabindex="-1">Starting to build Zelda Classic with Emscripten</h3>
<p>Even though Zelda Classic was now on A5 and building it from source, there were still a few pre-built libraries being used for music. I didn't want to deal with this yet, so to start I stubbed out the music layer with dummy functions so everything would still link with Emscripten.</p>
<p><code>zcmusic_fake.cpp</code></p>
<pre class="language-cpp"><code class="language-cpp"><span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">include</span> <span class="token string"><stddef.h></span></span><br /><span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">include</span> <span class="token string">"zcmusic.h"</span></span><br /><br /><span class="token keyword">int32_t</span> zcmusic_bufsz <span class="token operator">=</span> <span class="token number">64</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">bool</span> <span class="token function">zcmusic_init</span><span class="token punctuation">(</span><span class="token keyword">int32_t</span> flags<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><br /><span class="token keyword">bool</span> <span class="token function">zcmusic_poll</span><span class="token punctuation">(</span><span class="token keyword">int32_t</span> flags<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><br /><span class="token keyword">void</span> <span class="token function">zcmusic_exit</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><br /><br />ZCMUSIC <span class="token keyword">const</span> <span class="token operator">*</span><span class="token function">zcmusic_load_file</span><span class="token punctuation">(</span><span class="token keyword">char</span> <span class="token operator">*</span>filename<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token constant">NULL</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><br />ZCMUSIC <span class="token keyword">const</span> <span class="token operator">*</span><span class="token function">zcmusic_load_file_ex</span><span class="token punctuation">(</span><span class="token keyword">char</span> <span class="token operator">*</span>filename<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token constant">NULL</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><br /><span class="token keyword">bool</span> <span class="token function">zcmusic_play</span><span class="token punctuation">(</span>ZCMUSIC <span class="token operator">*</span>zcm<span class="token punctuation">,</span> <span class="token keyword">int32_t</span> vol<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><br /><span class="token keyword">bool</span> <span class="token function">zcmusic_pause</span><span class="token punctuation">(</span>ZCMUSIC <span class="token operator">*</span>zcm<span class="token punctuation">,</span> <span class="token keyword">int32_t</span> pause<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><br /><span class="token keyword">bool</span> <span class="token function">zcmusic_stop</span><span class="token punctuation">(</span>ZCMUSIC <span class="token operator">*</span>zcm<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><br /><span class="token keyword">void</span> <span class="token function">zcmusic_unload_file</span><span class="token punctuation">(</span>ZCMUSIC <span class="token operator">*</span><span class="token operator">&</span>zcm<span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><br /><span class="token keyword">int32_t</span> <span class="token function">zcmusic_get_tracks</span><span class="token punctuation">(</span>ZCMUSIC <span class="token operator">*</span>zcm<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token number">0</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><br /><span class="token keyword">int32_t</span> <span class="token function">zcmusic_change_track</span><span class="token punctuation">(</span>ZCMUSIC <span class="token operator">*</span>zcm<span class="token punctuation">,</span> <span class="token keyword">int32_t</span> tracknum<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token number">0</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><br /><span class="token keyword">int32_t</span> <span class="token function">zcmusic_get_curpos</span><span class="token punctuation">(</span>ZCMUSIC <span class="token operator">*</span>zcm<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token number">0</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><br /><span class="token keyword">void</span> <span class="token function">zcmusic_set_curpos</span><span class="token punctuation">(</span>ZCMUSIC <span class="token operator">*</span>zcm<span class="token punctuation">,</span> <span class="token keyword">int32_t</span> value<span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><br /><span class="token keyword">void</span> <span class="token function">zcmusic_set_speed</span><span class="token punctuation">(</span>ZCMUSIC <span class="token operator">*</span>zcm<span class="token punctuation">,</span> <span class="token keyword">int32_t</span> value<span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token punctuation">}</span></code></pre>
<p>Zelda Classic reads various configuration files from disk, including data files containing large things like MIDIs. Emscripten can package such data alongside Wasm deployments via the <a href="https://emscripten.org/docs/porting/files/packaging_files.html" target="_blank" rel="noopener noreferrer"><code>--preload-data</code></a> flag. These files can be pretty large (<code>zc.data</code> is ~9 MB), so a long-term caching strategy is best: <code>--use-preload-cache</code> is a nice Emscripten feature that will cache this file in IndexedDB. However, the key it uses is unique to every build, so any deployment invalidates the cache of all users. That's no good, but there's a quick hack to make the hash content-based instead:</p>
<pre class="language-bash"><code class="language-bash"><span class="token comment"># See https://github.com/emscripten-core/emscripten/issues/11952</span><br /><span class="token assign-left variable">HASH</span><span class="token operator">=</span><span class="token variable"><span class="token variable">$(</span>shasum -a <span class="token number">256</span> module.data <span class="token operator">|</span> <span class="token function">awk</span> <span class="token string">'{print $1}'</span><span class="token variable">)</span></span><br /><span class="token function">sed</span> -i -e <span class="token string">"s/<span class="token entity" title="\"">\"</span>package_uuid<span class="token entity" title="\"">\"</span>: <span class="token entity" title="\"">\"</span>[^<span class="token entity" title="\"">\"</span>]*<span class="token entity" title="\"">\"</span>/<span class="token entity" title="\"">\"</span>package_uuid<span class="token entity" title="\"">\"</span>:<span class="token entity" title="\"">\"</span><span class="token variable">$HASH</span><span class="token entity" title="\"">\"</span>/"</span> module.data.js<br /><span class="token keyword">if</span> <span class="token operator">!</span> <span class="token function">grep</span> -q <span class="token string">"<span class="token variable">$HASH</span>"</span> module.data.js<br /><span class="token keyword">then</span><br /> <span class="token builtin class-name">echo</span> <span class="token string">"failed to replace data hash"</span><br /> <span class="token builtin class-name">exit</span> <span class="token number">1</span><br /><span class="token keyword">fi</span></code></pre>
<blockquote>
<p>I also sent a <a href="https://github.com/emscripten-core/emscripten/pull/16807" target="_blank" rel="noopener noreferrer">PR to Emscripten</a> to fix the above</p>
</blockquote>
<h3 id="let-there-be-threads" tabindex="-1">Let there be threads</h3>
<p>As soon as I got Zelda Classic building with Emscripten and running in a browser, I'm faced with a page that does nothing but busy-hangs the main thread. Pausing in DevTools shows the problem:</p>
<pre class="language-cpp"><code class="language-cpp"><span class="token keyword">static</span> BITMAP <span class="token operator">*</span> <span class="token function">a5_display_init</span><span class="token punctuation">(</span><span class="token keyword">int</span> w<span class="token punctuation">,</span> <span class="token keyword">int</span> h<span class="token punctuation">,</span> <span class="token keyword">int</span> vw<span class="token punctuation">,</span> <span class="token keyword">int</span> vh<span class="token punctuation">,</span> <span class="token keyword">int</span> color_depth<span class="token punctuation">)</span><br /><span class="token punctuation">{</span><br /> BITMAP <span class="token operator">*</span> bp<span class="token punctuation">;</span><br /> ALLEGRO_STATE old_state<span class="token punctuation">;</span><br /> <span class="token keyword">int</span> pixel_format<span class="token punctuation">;</span><br /><br /> _a5_new_display_flags <span class="token operator">=</span> <span class="token function">al_get_new_display_flags</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> _a5_new_bitmap_flags <span class="token operator">=</span> <span class="token function">al_get_new_bitmap_flags</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">al_identity_transform</span><span class="token punctuation">(</span><span class="token operator">&</span>_a5_transform<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> bp <span class="token operator">=</span> <span class="token function">create_bitmap</span><span class="token punctuation">(</span>w<span class="token punctuation">,</span> h<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span><span class="token punctuation">(</span>bp<span class="token punctuation">)</span><br /> <span class="token punctuation">{</span><br /> <span class="token keyword">if</span><span class="token punctuation">(</span><span class="token operator">!</span>_a5_disable_threaded_display<span class="token punctuation">)</span><br /> <span class="token punctuation">{</span><br /> _a5_display_creation_done <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span><br /> _a5_display_width <span class="token operator">=</span> w<span class="token punctuation">;</span><br /> _a5_display_height <span class="token operator">=</span> h<span class="token punctuation">;</span><br /> _a5_screen_thread <span class="token operator">=</span> <span class="token function">al_create_thread</span><span class="token punctuation">(</span>_a5_display_thread<span class="token punctuation">,</span> <span class="token constant">NULL</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">al_start_thread</span><span class="token punctuation">(</span>_a5_screen_thread<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">while</span><span class="token punctuation">(</span><span class="token operator">!</span>_a5_display_creation_done<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// <<<<<<<<<<<<<<<<<< Hanging here!</span><br /> <span class="token punctuation">}</span><br /> <span class="token keyword">else</span><br /> <span class="token punctuation">{</span><br /> <span class="token keyword">if</span><span class="token punctuation">(</span><span class="token operator">!</span><span class="token function">_a5_setup_screen</span><span class="token punctuation">(</span>w<span class="token punctuation">,</span> h<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token constant">NULL</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /> gfx_driver<span class="token operator">-></span>w <span class="token operator">=</span> bp<span class="token operator">-></span>w<span class="token punctuation">;</span><br /> gfx_driver<span class="token operator">-></span>h <span class="token operator">=</span> bp<span class="token operator">-></span>h<span class="token punctuation">;</span><br /> <span class="token keyword">return</span> bp<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token keyword">return</span> <span class="token constant">NULL</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<blockquote>
<p>The busy-wait while loop pattern is problematic because it spins the CPU and wastes cycles. However, in this case it's actually pretty OK because the initialization code is expected to finish quickly. In general, a <a href="https://www.ibm.com/docs/en/aix/7.1?topic=programming-using-condition-variables" target="_blank" rel="noopener noreferrer">condition variable</a> is preferred to allow the thread to sleep until the state it cares about changes.</p>
</blockquote>
<p>Emscripten can build multithreaded applications that work on the web by using Web Workers and <code>SharedArrayBuffer</code>, but by default it will not build with thread support, so everything happens on the main thread.</p>
<blockquote>
<p>For a deep dive on threads in Wasm, read <a href="https://web.dev/webassembly-threads/" target="_blank" rel="noopener noreferrer">this</a></p>
</blockquote>
<blockquote>
<p><code>SharedArrayBuffer</code> requires special response headers to be set, even for localhost. The simplest way to do this is to use Paul Irish's <a href="https://github.com/paulirish/statikk" target="_blank" rel="noopener noreferrer"><code>stattik</code></a>: just run <code>npx statikk --port 8000 --coi</code></p>
</blockquote>
<p>In the above case, a thread is created which is expected to instantly set <code>_a5_display_creation_done</code>, but due to the lack of threads that never happens so the main thread is left hanging forever.</p>
<p>Clearly, I needed to enable <a href="https://emscripten.org/docs/porting/pthreads.html" target="_blank" rel="noopener noreferrer"><code>pthread</code></a> support.</p>
<p>I figured it'd be best to also enable <code>PROXY_TO_PTHREAD</code>, which moves the main application thread into a pthread AKA web worker (instead of the main browser thread), but that was a dead-end due to <a href="https://github.com/emscripten-core/emscripten/issues/16492" target="_blank" rel="noopener noreferrer">various</a> <a href="https://github.com/libsdl-org/SDL/issues/5260" target="_blank" rel="noopener noreferrer">unexpected</a> issues with SDL which means it does not support this setting.</p>
<blockquote>
<p>I got <a href="https://github.com/emscripten-core/emscripten/issues/6009#issuecomment-1096131889" target="_blank" rel="noopener noreferrer">close</a> to getting <code>PROXY_TO_PTHREAD</code> to work, but not close enough.</p>
</blockquote>
<p>In lieu of this, I had to add <code>rest(0)</code> to many places where Zelda Classic busy waits on the main application thread, otherwise Emscripten's <a href="https://web.dev/asyncify/" target="_blank" rel="noopener noreferrer"><code>ASYNCIFY</code></a> feature has no opportunity to yield the main thread back to the browser, resulting in the page hanging. For example, this code is problematic to run on the main thread:</p>
<pre class="language-cpp"><code class="language-cpp"><span class="token keyword">do</span><br /><span class="token punctuation">{</span><br /><span class="token punctuation">}</span><br /><span class="token keyword">while</span><span class="token punctuation">(</span><span class="token function">gui_mouse_b</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>because mouse input can only be registered when the main browser thread is in control. Hence, a <code>rest(0)</code> fixes the hang by yielding back to the browser via <code>ASYNCIFY</code>:</p>
<pre class="language-cpp"><code class="language-cpp"><span class="token keyword">do</span><br /><span class="token punctuation">{</span><br /> <span class="token comment">// ASYNCIFY will save the stack, yield to the browser</span><br /> <span class="token comment">// (processing any user input or rendering), then restore</span><br /> <span class="token comment">// the stack and continue on.</span><br /> <span class="token function">rest</span><span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><br /><span class="token keyword">while</span><span class="token punctuation">(</span><span class="token function">gui_mouse_b</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<h3 id="of-mutexes-and-deadlocks" tabindex="-1">Of mutexes and deadlocks</h3>
<p>The most difficult problem I ran into during this entire project was debugging a deadlock. It took a few days of getting nowhere, logging when a lock was acquired/released and by what thread (big waste of time!)</p>
<p>Eventually I realized I should stop trying to debug the large mess of a program in front of me and try to build up a reproduction of the issue from scratch.</p>
<p>SDL provides an interface for mutexes that, on Unix, uses <code>pthread</code>. Apparently, some platforms do not support recursive mutexes - that is, allowing a thread to lock the same mutex multiple times, only releasing the lock when it matches with an equal number of unlocks. To support platforms without this functionality, SDL fakes it.</p>
<p><a href="https://github.com/libsdl-org/SDL/blob/c36bd78474c962119db2f5161be6b0d4f07d535e/src/thread/pthread/SDL_sysmutex.c#L91" target="_blank" rel="noopener noreferrer"><code>SDL_sysmutex.c</code></a></p>
<pre class="language-cpp"><code class="language-cpp"><span class="token comment">/* Lock the mutex */</span><br /><span class="token keyword">int</span><br /><span class="token function">SDL_LockMutex</span><span class="token punctuation">(</span>SDL_mutex <span class="token operator">*</span> mutex<span class="token punctuation">)</span><br /><span class="token punctuation">{</span><br /><span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">if</span> <span class="token expression">FAKE_RECURSIVE_MUTEX</span></span><br /> pthread_t this_thread<span class="token punctuation">;</span><br /><span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">endif</span></span><br /><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>mutex <span class="token operator">==</span> <span class="token constant">NULL</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token function">SDL_InvalidParamError</span><span class="token punctuation">(</span><span class="token string">"mutex"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /><span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">if</span> <span class="token expression">FAKE_RECURSIVE_MUTEX</span></span><br /> this_thread <span class="token operator">=</span> <span class="token function">pthread_self</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>mutex<span class="token operator">-></span>owner <span class="token operator">==</span> this_thread<span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token operator">++</span>mutex<span class="token operator">-></span>recursive<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span><br /> <span class="token comment">/* The order of operations is important.<br /> We set the locking thread id after we obtain the lock<br /> so unlocks from other threads will fail.<br /> */</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">pthread_mutex_lock</span><span class="token punctuation">(</span><span class="token operator">&</span>mutex<span class="token operator">-></span>id<span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token number">0</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> mutex<span class="token operator">-></span>owner <span class="token operator">=</span> this_thread<span class="token punctuation">;</span><br /> mutex<span class="token operator">-></span>recursive <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token function">SDL_SetError</span><span class="token punctuation">(</span><span class="token string">"pthread_mutex_lock() failed"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /><span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">else</span></span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">pthread_mutex_lock</span><span class="token punctuation">(</span><span class="token operator">&</span>mutex<span class="token operator">-></span>id<span class="token punctuation">)</span> <span class="token operator">!=</span> <span class="token number">0</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token function">SDL_SetError</span><span class="token punctuation">(</span><span class="token string">"pthread_mutex_lock() failed"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">endif</span></span><br /> <span class="token keyword">return</span> <span class="token number">0</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Once I realized that the deadlock did not happen when condition variables were not used, I was able to create a small reproduction that resulted in a deadlock via Emscripten
but not when building for Mac. I reported the <a href="https://github.com/libsdl-org/SDL/issues/5428" target="_blank" rel="noopener noreferrer">bug</a> to SDL, and even proposed a <a href="https://github.com/libsdl-org/SDL/pull/5479" target="_blank" rel="noopener noreferrer">patch</a> to improve the fake recursive mutex code,
(at least, it fixed my deadlock) but it turns out that mixing condtion variables and recursive mutexes is a <a href="https://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_mutexattr_settype.html#:~:text=It%20is%20advised%20that%20an%20application%20should%20not%20use" target="_blank" rel="noopener noreferrer">very bad idea</a>, and in general is <a href="https://github.com/libsdl-org/SDL/pull/5479#issuecomment-1090221325" target="_blank" rel="noopener noreferrer">impossible</a> to get right.</p>
<p>Eventually I realized it was odd that Emscripten doesn't support recursive mutexes. And sure enough, after writing a quick sample program, I determined <a href="https://github.com/libsdl-org/SDL/pull/5479#issuecomment-1089790046" target="_blank" rel="noopener noreferrer">it actually does support them</a>. Turns out the problem was in SDL's header configuration for Emscripten <a href="https://github.com/libsdl-org/SDL/pull/5496" target="_blank" rel="noopener noreferrer">not specifiying</a> that recursive mutexes are supported.</p>
<h2 id="getting-it-fully-functional" tabindex="-1">Getting it fully functional</h2>
<h3 id="playing-midi-with-timidity" tabindex="-1">Playing MIDI with Timidity</h3>
<p>Zelda Classic <code>.qst</code> files contain MIDIs, but browsers can't directly play MIDI files. In order to synthesize audio from a MIDI file you need:</p>
<ul>
<li>a sound sample database</li>
<li>code to interpret the various MIDI commands, such as turning notes on and off</li>
</ul>
<p>Emscripten supports various audio formats with <a href="https://emscripten.org/docs/getting_started/FAQ.html#what-are-my-options-for-audio-playback" target="_blank" rel="noopener noreferrer"><code>SDL_mixer</code></a>, configured via <a href="https://emsettings.surma.technology/#SDL2_MIXER_FORMATS" target="_blank" rel="noopener noreferrer"><code>SDL2_MIXER_FORMATS</code></a>. However, there was no support for MIDI. Luckily <code>SDL_mixer</code> already supports MIDI playback (it uses <a href="https://github.com/SDL-mirror/SDL_mixer/tree/master/src/codecs/timidity#readme" target="_blank" rel="noopener noreferrer">Timidity</a>). It was <a href="https://github.com/emscripten-core/emscripten/pull/16556" target="_blank" rel="noopener noreferrer">straightfoward to configure the Emscripten port system</a> to include Timidity support when requested.</p>
<p>As for the sound samples, I just grabbed some free ones called <a href="https://www.npmjs.com/package/freepats" target="_blank" rel="noopener noreferrer">freepats</a>. Initially I added them to the Wasm preload datafile, but it's actually pretty large at 30+ MB so a better solution is to load the individual samples from the network as requested. I knew of a Timidity <a href="https://github.com/feross/timidity" target="_blank" rel="noopener noreferrer">fork</a> that did just that, so I studied how it worked there. When a MIDI file loads, that fork checks all the instruments a song will use and <a href="https://github.com/feross/timidity/commit/d1790eef24ff3b4067c536e45aa88c0863ad9676#diff-6ff6417493baaa56336d5c73f273ea180db9c16c2f4a37adf4f5abc380ffc6ccR207" target="_blank" rel="noopener noreferrer">logs which ones are missing</a>. Then the JS code checks that log, fetches the missing ones, and reloads the data. I basically did the same, but all within Timidity/<code>EM_JS</code>.</p>
<p>These fetches freeze the game (but not the browser main thread!) until they complete, which isn't too bad when starting a quest but can be especially jarring when reaching a new area that plays a song with new MIDI instruments. To make this a bit more bearable, I wrote a <a href="https://gist.github.com/connorjclark/6afb9fb588331a23a2d8fa57cfefe8f5" target="_blank" rel="noopener noreferrer"><code>fetchWithProgress</code></a> function to display a progress bar in the page header.</p>
<blockquote>
<p>While freepats is nice (free, small, and good quality), it is <a href="https://freepats.zenvoid.org/SoundSets/general-midi.html#FreePatsGM" target="_blank" rel="noopener noreferrer">missing many instruments</a>. I found some <a href="https://www.doomworld.com/idgames/music/dgguspat" target="_blank" rel="noopener noreferrer">90s-era GUS sound files</a> on a DOOM modding site to fill the gaps. There's a comment on that page suggesting the PPL160 is even better quality, so I <a href="https://github.com/chocolate-doom/chocolate-doom/issues/878" target="_blank" rel="noopener noreferrer">located</a> those too. I'm not too happy with the result of <a href="https://github.com/connorjclark/ZeldaClassic/blob/wasm-web/timidity/make-cfg.js" target="_blank" rel="noopener noreferrer">meshing</a> these various instruments together. I'm sure this could be improved, but at least no MIDI files will have missing instruments.</p>
</blockquote>
<h3 id="music-working-but-no-sfx" tabindex="-1">Music working, but no SFX?</h3>
<p>Zelda Classic uses different output channels for music and SFX, which is pretty common in games. Especially because you may wish to sample the two at different rates, which means they can't use the same output channel. Music is typically sampled at a higher rate for quality purposes, which takes more processing time but that's OK because it is ok to buffer–latency isn't such a big deal, unless you're syncing music to video or something. SFX is typically sampled at a lower rate, because there is more urgency to play a sound effect in reaction to gameplay.</p>
<p>With MIDI support included, music was now playing on the title screen, but no SFX was playing. I compiled the Allegro sound example <a href="https://allegro5.org/examples/examples/ex_saw.html" target="_blank" rel="noopener noreferrer"><code>ex_saw</code></a>, which I knew already worked with Emscripten because the hosted example Wasm worked. However, building locally nothing would play, so I had another bug in Allegro to fix.</p>
<p>I added some printf'ing to <code>SDL_SetError</code> and noticed that when Allegro called <code>SDL_Init(SDL_INIT_EVERYTHING)</code>, it would error with <code>"SDL not built with haptic support"</code> ... and then SDL would proceed to tear everything down! SDL failed to setup the haptic subsystem because it does not provide an Emscripten implementation for it. And since Allegro initialized SDL by requesting everything, SDL could not comply. That doesn't explain why it was working before but isn't today–to explain that, I <code>git blame</code>'d the <code>SDL_Init</code> function and saw that a change was made recently to shutdown everything if any subsystem errors. Mystery solved, and I sent a <a href="https://github.com/liballeg/allegro5/pull/1322" target="_blank" rel="noopener noreferrer">PR</a> to Allegro to fix it.</p>
<pre class="language-diff-cpp"><code class="language-diff-cpp">diff --git a/src/sdl/sdl_system.c b/src/sdl/sdl_system.c<br /><span class="token coord">--- a/src/sdl/sdl_system.c</span><br /><span class="token coord">+++ b/src/sdl/sdl_system.c</span><br /><span class="token unchanged language-cpp"><span class="token prefix unchanged"> </span><span class="token keyword">static</span> ALLEGRO_SYSTEM <span class="token operator">*</span><span class="token function">sdl_initialize</span><span class="token punctuation">(</span><span class="token keyword">int</span> flags<span class="token punctuation">)</span><br /></span><span class="token deleted-sign deleted language-cpp"><span class="token prefix deleted">-</span> <span class="token function">SDL_Init</span><span class="token punctuation">(</span>SDL_INIT_EVERYTHING<span class="token punctuation">)</span><span class="token punctuation">;</span><br /></span><span class="token inserted-sign inserted language-cpp"><span class="token prefix inserted">+</span> <span class="token keyword">unsigned</span> <span class="token keyword">int</span> sdl_flags <span class="token operator">=</span> SDL_INIT_EVERYTHING<span class="token punctuation">;</span><br /><span class="token prefix inserted">+</span><span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">ifdef</span> <span class="token expression">__EMSCRIPTEN__</span></span><br /><span class="token prefix inserted">+</span> <span class="token comment">// SDL currently does not support haptic feedback for emscripten.</span><br /><span class="token prefix inserted">+</span> sdl_flags <span class="token operator">&=</span> <span class="token operator">~</span>SDL_INIT_HAPTIC<span class="token punctuation">;</span><br /><span class="token prefix inserted">+</span><span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">endif</span></span><br /><span class="token prefix inserted">+</span> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">SDL_Init</span><span class="token punctuation">(</span>sdl_flags<span class="token punctuation">)</span> <span class="token operator"><</span> <span class="token number">0</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /><span class="token prefix inserted">+</span> <span class="token function">ALLEGRO_ERROR</span><span class="token punctuation">(</span><span class="token string">"SDL_Init failed: %s"</span><span class="token punctuation">,</span> <span class="token function">SDL_GetError</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token prefix inserted">+</span> <span class="token keyword">return</span> <span class="token constant">NULL</span><span class="token punctuation">;</span><br /><span class="token prefix inserted">+</span> <span class="token punctuation">}</span></span></code></pre>
<blockquote>
<p>The Web does have a <a href="https://developer.mozilla.org/en-US/docs/Web/API/Vibration_API" target="_blank" rel="noopener noreferrer">Vibration API</a> (for vibrating a mobile device), and experimental support for <a href="https://developer.mozilla.org/en-US/docs/Web/API/GamepadHapticActuator" target="_blank" rel="noopener noreferrer">Gamepad haptic feedback</a>, so it's certainly possible for SDL to support.</p>
</blockquote>
<p>Now the <code>ex_saw</code> example worked when built locally, but SFX still didn't play in Zelda Classic. After some more printf'ing, I noticed that SDL was failing to open a second audio channel for SFX. Weird... I opened up SDL's <a href="https://github.com/libsdl-org/SDL/blob/55a4e1d336db0dd0af70bf22df8ec3ae0b38644a/src/audio/emscripten/SDL_emscriptenaudio.c#L347" target="_blank" rel="noopener noreferrer">audio implementation for Emscripten</a>, and a variable named <code>OnlyHasDefaultOutputDevice</code> grabbed my attention:</p>
<pre class="language-cpp"><code class="language-cpp"><span class="token keyword">static</span> SDL_bool<br /><span class="token function">EMSCRIPTENAUDIO_Init</span><span class="token punctuation">(</span>SDL_AudioDriverImpl <span class="token operator">*</span> impl<span class="token punctuation">)</span><br /><span class="token punctuation">{</span><br /> SDL_bool available<span class="token punctuation">,</span> capture_available<span class="token punctuation">;</span><br /><br /> <span class="token comment">/* Set the function pointers */</span><br /> impl<span class="token operator">-></span>OpenDevice <span class="token operator">=</span> EMSCRIPTENAUDIO_OpenDevice<span class="token punctuation">;</span><br /> impl<span class="token operator">-></span>CloseDevice <span class="token operator">=</span> EMSCRIPTENAUDIO_CloseDevice<span class="token punctuation">;</span><br /><br /> impl<span class="token operator">-></span>OnlyHasDefaultOutputDevice <span class="token operator">=</span> SDL_TRUE<span class="token punctuation">;</span><br /> <span class="token comment">// ...</span></code></pre>
<p>Thinking "no way this will work", I set that to <code>SDL_FALSE</code> and ... it worked! I reported this as a bug <a href="https://github.com/libsdl-org/SDL/issues/5485" target="_blank" rel="noopener noreferrer">here</a>. It's not so obvious that this is the proper way to fix this, so this won't be actually resolved in SDL for a bit. Which leads me to the next topic...</p>
<h3 id="build-script-hacking" tabindex="-1">Build script hacking</h3>
<p>When you fix a bug in a dependency, there is typically a waiting period before a new version of that dependency can be used normally. This is not a problem because there are other ways to use a non-official version of a dependency:</p>
<ul>
<li>package managers can be configured to point to a specific fork or commit</li>
<li>you could vendor the dependency, making it part of your project's source control</li>
<li>you could maintain a set of diff patches to apply on top of the official release</li>
</ul>
<blockquote>
<p>Playwright has <a href="https://github.com/microsoft/playwright/tree/main/browser_patches" target="_blank" rel="noopener noreferrer">nice infrastructure</a> for building browsers with custom patches that wouldn't be (pick one: appropriate, quick, or easy) to merge upstream. Brave <a href="https://github.com/brave/brave-core/tree/master/patches" target="_blank" rel="noopener noreferrer">does something similar</a>.</p>
</blockquote>
<p>Then there's the lazy way: At first, I reached for the expediency of <code>sed</code> commands. I'd find a bug in a dependency, figure out how to use <code>sed</code> to fix it locally, plop it into my build script, and make a note to upstream the bug fix some time later.</p>
<pre class="language-bash"><code class="language-bash"><span class="token comment"># Temporary workarounds until various things are fixed upstream.</span><br /><br /><span class="token keyword">if</span> <span class="token punctuation">[</span> <span class="token operator">!</span> -d <span class="token string">"<span class="token variable">$EMCC_CACHE_DIR</span>/ports/sdl2"</span> <span class="token punctuation">]</span><br /><span class="token keyword">then</span><br /> <span class="token comment"># Ensure that the SDL source code has been downloaded.</span><br /> embuilder build sdl2<br /><span class="token keyword">fi</span><br /><span class="token comment"># Must manually delete the SDL library to force Emscripten to rebuild it.</span><br /><span class="token function">rm</span> -rf <span class="token string">"<span class="token variable">$EMCC_CACHE_LIB_DIR</span>"</span>/libSDL2.a <span class="token string">"<span class="token variable">$EMCC_CACHE_LIB_DIR</span>"</span>/libSDL2-mt.a<br /><br /><span class="token comment"># See https://github.com/libsdl-org/SDL/pull/5496</span><br /><span class="token keyword">if</span> <span class="token operator">!</span> <span class="token function">grep</span> -q SDL_THREAD_PTHREAD_RECURSIVE_MUTEX <span class="token string">"<span class="token variable">$EMCC_CACHE_DIR</span>/ports/sdl2/SDL-release-2.0.20/include/SDL_config_emscripten.h"</span><span class="token punctuation">;</span> <span class="token keyword">then</span><br /> <span class="token builtin class-name">echo</span> <span class="token string">"#define SDL_THREAD_PTHREAD_RECURSIVE_MUTEX 1"</span> <span class="token operator">>></span> <span class="token string">"<span class="token variable">$EMCC_CACHE_DIR</span>/ports/sdl2/SDL-release-2.0.20/include/SDL_config_emscripten.h"</span><br /><span class="token keyword">fi</span><br /><br /><span class="token comment"># SDL's emscripten audio specifies only one default audio output device, but turns out</span><br /><span class="token comment"># that can be ignored and things will just work. Without this, only SFX will play and MIDIs</span><br /><span class="token comment"># will error on opening a handle to the audio device.</span><br /><span class="token comment"># See https://github.com/libsdl-org/SDL/issues/5485</span><br /><span class="token function">sed</span> -i -e <span class="token string">'s/impl->OnlyHasDefaultOutputDevice = 1/impl->OnlyHasDefaultOutputDevice = 0/'</span> <span class="token string">"<span class="token variable">$EMCC_CACHE_DIR</span>/ports/sdl2/SDL-release-2.0.20/src/audio/emscripten/SDL_emscriptenaudio.c"</span></code></pre>
<p>And those are just the changes to SDL. I had more for Allegro...</p>
<p>Keeping it simple early helped keep things moving, but once the changes became larger than tweaking a line or two this process became unfeasible. Eventually I setup a simple system: a pretty straightforward application of <code>git diff</code> and <code>patch</code>. There's some annoying cache clearing that needs to be done in order for patches to Emscripten's ports to take effect, but it wasn't too bad. Here's the entirety of it:</p>
<pre class="language-bash"><code class="language-bash"><span class="token shebang important">#!/bin/bash</span><br /><br /><span class="token comment"># A very basic patching system. Only supports a single patch per directory.</span><br /><span class="token comment"># To update a patch:</span><br /><span class="token comment"># 1) cd to the directory</span><br /><span class="token comment"># 2) make your changes</span><br /><span class="token comment"># 3) git add .</span><br /><span class="token comment"># 4) git diff --staged | pbcopy</span><br /><span class="token comment"># 5) overwrite existing patch file with new one</span><br /><br /><span class="token builtin class-name">set</span> -e<br /><br /><span class="token assign-left variable">SCRIPT_DIR</span><span class="token operator">=</span><span class="token variable"><span class="token variable">`</span> <span class="token builtin class-name">cd</span> <span class="token string">"<span class="token variable"><span class="token variable">$(</span> <span class="token function">dirname</span> <span class="token string">"<span class="token variable">${<span class="token environment constant">BASH_SOURCE</span><span class="token punctuation">[</span>0<span class="token punctuation">]</span>}</span>"</span> <span class="token variable">)</span></span>"</span> <span class="token operator">&&</span> <span class="token builtin class-name">pwd</span> <span class="token variable">`</span></span><br /><span class="token assign-left variable">EMCC_DIR</span><span class="token operator">=</span><span class="token string">"<span class="token variable"><span class="token variable">$(</span><span class="token function">dirname</span> <span class="token punctuation">$(</span>which emcc<span class="token punctuation">)</span><span class="token variable">)</span></span>"</span><br /><span class="token assign-left variable">EMCC_CACHE_DIR</span><span class="token operator">=</span><span class="token string">"<span class="token variable">$EMCC_DIR</span>/cache"</span><br /><br /><span class="token assign-left variable">NO_GIT_CLEAN</span><span class="token operator">=</span>false<br /><span class="token assign-left variable">GIT_CLEAN</span><span class="token operator">=</span>true<br /><br /><span class="token comment"># folder patch</span><br /><span class="token keyword">function</span> <span class="token function-name function">apply_patch</span> <span class="token punctuation">{</span><br /> <span class="token builtin class-name">cd</span> <span class="token string">"<span class="token variable">$1</span>"</span><br /> <span class="token builtin class-name">echo</span> <span class="token string">"Applying patch: <span class="token variable">$2</span>"</span><br /><br /> <span class="token keyword">if</span> <span class="token punctuation">[</span> -d .git <span class="token punctuation">]</span><span class="token punctuation">;</span> <span class="token keyword">then</span><br /> <span class="token function">git</span> restore --staged <span class="token builtin class-name">.</span><br /> <span class="token comment"># Cleaning is the sensible thing to do, unless there are build-time generated</span><br /> <span class="token comment"># files (ex: allegro will create some header configuration files based on the environment).</span><br /> <span class="token keyword">if</span> <span class="token variable">$3</span> <span class="token punctuation">;</span> <span class="token keyword">then</span><br /> <span class="token function">git</span> clean -fdq<br /> <span class="token keyword">fi</span><br /> <span class="token function">git</span> checkout -- <span class="token builtin class-name">.</span><br /> <span class="token keyword">else</span><br /> <span class="token function">git</span> init <span class="token operator">></span> /dev/null<br /> <span class="token function">git</span> <span class="token function">add</span> <span class="token builtin class-name">.</span><br /> <span class="token function">git</span> commit -m init<br /> <span class="token keyword">fi</span><br /><br /> patch -s -p1 <span class="token operator"><</span> <span class="token string">"<span class="token variable">$2</span>"</span><br /> <span class="token builtin class-name">cd</span> - <span class="token operator">></span> /dev/null<br /><span class="token punctuation">}</span><br /><br /><span class="token builtin class-name">echo</span> <span class="token string">"Applying patches ..."</span><br /><br />apply_patch <span class="token string">"<span class="token variable">$EMCC_DIR</span>"</span> <span class="token string">"<span class="token variable">$SCRIPT_DIR</span>/emscripten.patch"</span> <span class="token variable">$GIT_CLEAN</span><br /><br /><span class="token comment"># Ensure that the SDL source code has been downloaded,</span><br /><span class="token comment"># otherwise the patches can't be applied.</span><br /><span class="token keyword">if</span> <span class="token punctuation">[</span> <span class="token operator">!</span> -d <span class="token string">"<span class="token variable">$EMCC_CACHE_DIR</span>/ports/sdl2"</span> <span class="token punctuation">]</span><br /><span class="token keyword">then</span><br /> embuilder build sdl2<br /><span class="token keyword">fi</span><br /><span class="token keyword">if</span> <span class="token punctuation">[</span> <span class="token operator">!</span> -d <span class="token string">"<span class="token variable">$EMCC_CACHE_DIR</span>/ports/sdl2_mixer/SDL_mixer-release-2.0.4"</span> <span class="token punctuation">]</span><br /><span class="token keyword">then</span><br /> <span class="token function">rm</span> -rf <span class="token string">"<span class="token variable">$EMCC_CACHE_DIR</span>/ports/sdl2_mixer"</span><br /> embuilder build sdl2_mixer<br /><span class="token keyword">fi</span><br /><br /><span class="token comment"># Manually delete libraries from Emscripten cache to force a rebuild.</span><br /><span class="token function">rm</span> -rf <span class="token string">"<span class="token variable">$EMCC_CACHE_LIB_DIR</span>"</span>/libSDL2-mt.a<br /><span class="token function">rm</span> -rf <span class="token string">"<span class="token variable">$EMCC_CACHE_LIB_DIR</span>"</span>/libSDL2_mixer_gme_mid-mod-mp3-ogg.a<br /><br />apply_patch <span class="token string">"<span class="token variable">$EMCC_CACHE_DIR</span>/ports/sdl2/SDL-4b8d69a41687e5f6f4b05f7fd9804dd9fcac0347"</span> <span class="token string">"<span class="token variable">$SCRIPT_DIR</span>/sdl2.patch"</span> <span class="token variable">$GIT_CLEAN</span><br />apply_patch <span class="token string">"<span class="token variable">$EMCC_CACHE_DIR</span>/ports/sdl2_mixer/SDL_mixer-release-2.0.4"</span> <span class="token string">"<span class="token variable">$SCRIPT_DIR</span>/sdl2_mixer.patch"</span> <span class="token variable">$GIT_CLEAN</span><br />apply_patch _deps/allegro5-src <span class="token string">"<span class="token variable">$SCRIPT_DIR</span>/allegro5.patch"</span> <span class="token variable">$NO_GIT_CLEAN</span><br /><br /><span class="token builtin class-name">echo</span> <span class="token string">"Done applying patches!"</span></code></pre>
<h2 id="making-it-awesome" tabindex="-1">Making it awesome</h2>
<h3 id="quest-list" tabindex="-1">Quest List</h3>
<p>Up until now only the built-in original Zelda was playable. Now that I had sound working, I wanted to be able to play custom quests. From my previous work on Quest Maker, I had scraped 600+ quests and their metadata from PureZC.com. Quests are just single <code>.qst</code> files, and I needed a way to get Zelda Classic their data. Adding them to the <code>--preload-data</code> is not an option, because in total they are about 2 GB! No, each file needs to be loaded only upon request.</p>
<blockquote>
<p><a href="https://hoten.cc/quest-maker/play/" target="_blank" rel="noopener noreferrer">Quest Maker</a> was my attempt at remaking Zelda Classic. Eventually I realized it would take 20 years to recreate a 20 year-old game engine, so I gave up.</p>
</blockquote>
<p>When creating a new save file, you can select the quest file to use from a file selector dialog. In order to support that on the web, I needed to populate an empty file for every quest file so that the user could at least select it from this dialog. To do that, I used a metadata file of the <a href="https://hoten.cc/quest-maker/play/quest-manifest.json" target="_blank" rel="noopener noreferrer">entire quest corpus</a> to seed the filesystem with empty files.</p>
<pre class="language-js"><code class="language-js"><span class="token comment">// This function is called early on in main() setup.</span><br /><span class="token constant">EM_ASYNC_JS</span><span class="token punctuation">(</span><span class="token keyword">void</span><span class="token punctuation">,</span> em_init_fs_<span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">{</span><br /> <span class="token comment">// Initialize the filesystem with 0-byte files for every quest.</span><br /> <span class="token keyword">const</span> quests <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token constant">ZC</span><span class="token punctuation">.</span><span class="token function">fetch</span><span class="token punctuation">(</span><span class="token string">"https://hoten.cc/quest-maker/play/quest-manifest.json"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token constant">FS</span><span class="token punctuation">.</span><span class="token function">mkdir</span><span class="token punctuation">(</span><span class="token string">'/_quests'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">function</span> <span class="token function">writeFakeFile</span><span class="token punctuation">(</span><span class="token parameter">path<span class="token punctuation">,</span> url</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token constant">FS</span><span class="token punctuation">.</span><span class="token function">writeFile</span><span class="token punctuation">(</span>path<span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> window<span class="token punctuation">.</span><span class="token constant">ZC</span><span class="token punctuation">.</span>pathToUrl<span class="token punctuation">[</span>path<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token string">'https://hoten.cc/quest-maker/play/'</span> <span class="token operator">+</span> url<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">let</span> i <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> i <span class="token operator"><</span> quests<span class="token punctuation">.</span>length<span class="token punctuation">;</span> i<span class="token operator">++</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> quest <span class="token operator">=</span> quests<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span>quest<span class="token punctuation">.</span>urls<span class="token punctuation">.</span>length<span class="token punctuation">)</span> <span class="token keyword">continue</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">const</span> url <span class="token operator">=</span> quest<span class="token punctuation">.</span>urls<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">;</span><br /> <span class="token keyword">const</span> path <span class="token operator">=</span> window<span class="token punctuation">.</span><span class="token constant">ZC</span><span class="token punctuation">.</span><span class="token function">createPathFromUrl</span><span class="token punctuation">(</span>url<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">writeFakeFile</span><span class="token punctuation">(</span>path<span class="token punctuation">,</span> url<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<div class="captioned-image">
<img style="max-width: min(700px, 100%)" src="https://hoten.cc/images/zc/filepicker.png" alt="" />
<span>The in-game file selector dialog. Quests are stored in their own folders such as: <code>_quests/1/OcarinaOfPower.qst</code>, requiring knowledge of where the quest you want is and multiple clicks to navigate to it</span>
</div>
<p>Just before Zelda Classic actually opens a file, <code>em_fetch_file_</code> is called and the data will be fetched and written to the filesystem.</p>
<pre class="language-js"><code class="language-js"><span class="token constant">EM_ASYNC_JS</span><span class="token punctuation">(</span><span class="token keyword">void</span><span class="token punctuation">,</span> em_fetch_file_<span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token keyword">const</span> char <span class="token operator">*</span>path<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">{</span><br /> <span class="token keyword">try</span> <span class="token punctuation">{</span><br /> path <span class="token operator">=</span> <span class="token function">UTF8ToString</span><span class="token punctuation">(</span>path<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token constant">FS</span><span class="token punctuation">.</span><span class="token function">stat</span><span class="token punctuation">(</span>path<span class="token punctuation">)</span><span class="token punctuation">.</span>size<span class="token punctuation">)</span> <span class="token keyword">return</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">const</span> url <span class="token operator">=</span> window<span class="token punctuation">.</span><span class="token constant">ZC</span><span class="token punctuation">.</span>pathToUrl<span class="token punctuation">[</span>path<span class="token punctuation">]</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span>url<span class="token punctuation">)</span> <span class="token keyword">return</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">const</span> data <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token constant">ZC</span><span class="token punctuation">.</span><span class="token function">fetch</span><span class="token punctuation">(</span>url<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token constant">FS</span><span class="token punctuation">.</span><span class="token function">writeFile</span><span class="token punctuation">(</span>path<span class="token punctuation">,</span> data<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span> <span class="token keyword">catch</span> <span class="token punctuation">(</span>e<span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token comment">// Fetch failed (could be offline) or path did not exist.</span><br /> console<span class="token punctuation">.</span><span class="token function">error</span><span class="token punctuation">(</span><span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">error loading </span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>path<span class="token interpolation-punctuation punctuation">}</span></span><span class="token template-punctuation string">`</span></span><span class="token punctuation">,</span> e<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>There are also a few quests that come with external music files (mp3, ogg). They can be added to this "lazy" filesystem too:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">const</span> extraResourceUrl <span class="token keyword">of</span> quest<span class="token punctuation">.</span>extraResources <span class="token operator">||</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token function">writeFakeFile</span><span class="token punctuation">(</span>window<span class="token punctuation">.</span><span class="token constant">ZC</span><span class="token punctuation">.</span><span class="token function">createPathFromUrl</span><span class="token punctuation">(</span>extraResourceUrl<span class="token punctuation">)</span><span class="token punctuation">,</span> extraResourceUrl<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>But this file selector dialog is <em>really awkward</em> to use. Let's leverage one of the web's superpowers here: the URL. I created a "Quest List" directory that presents a <code>Play!</code> link:</p>
<p><a href="https://web.zquestclassic.com/play/?open=purezc/quests/731/" target="_blank" rel="noopener noreferrer">https://web.zquestclassic.com/play/?open=purezc/quests/731/</a></p>
<p>and in Zelda Classic I grabbed that query parameter and hacked away at the title screen code to either 1) start a new save file with the quest or 2) load an existing save file of that quest. This makes it simpler than ever to jump into a Zelda Classic quest.</p>
<p>Zelda Classic's editor has a testing feature that allows a quest editor to jump into the game at the current screen they are editing. Natively, that's done with command line arguments, but to do the same for the web we have our friend the URL. Click this URL and you'll find yourself at the end of the game!</p>
<p><a href="https://web.zquestclassic.com/play/?test=/quests/purezc/373/r01/Quest.qst&dmap=16&screen=65" target="_blank" rel="noopener noreferrer">https://web.zquestclassic.com/play/?test=/quests/purezc/373/r01/Quest.qst&dmap=16&screen=65</a></p>
<p>You can also get a deep link to open a specific screen in the editor:</p>
<p><a href="https://web.zquestclassic.com/create/?open=quests/purezc/373/r01/Quest.qst&map=3&screen=65" target="_blank" rel="noopener noreferrer">https://web.zquestclassic.com/create/?open=quests/purezc/373/r01/Quest.qst&map=3&screen=65</a></p>
<h3 id="mp3s-and-oggs-and-retro-music" tabindex="-1">MP3s, and OGGs and retro music</h3>
<p>OK, so remember when I did all the fake <code>zcmusic</code> stuff, just to get things building and punt the prebuilt sound library stuff? Well, eventually I realized that SDL_mixer supports OGG and MP3 so it should be straight-forward to implement <code>zcmusic</code> using SDL_mixer. SDL_mixer and Emscripten have the know-how to synthesize these various audio formats, so I don't need to work out how to compile these audio libraries myself.</p>
<p>I should mention, Zelda Classic has two separate code paths for music: one is for MIDI, which I've already discussed, and the other is <code>"zcmusic"</code> which is just a wrapper around various audio libraries to support OGG, MP3, and various retro video-game specific formats:</p>
<ul>
<li>gbs (GameBoy Sound)</li>
<li>nsf (NES Sound Format)</li>
<li>spc (SNES Sound)</li>
<li>vgm (<a href="https://en.wikipedia.org/wiki/VGM_(file_format)" target="_blank" rel="noopener noreferrer">Video Game Music</a>, a grab-bag of multiple game systems)</li>
</ul>
<p>So Emscripten + SDL_mixer handles everything but these retro formats. For that, Zelda Classic uses the <a href="https://bitbucket.org/mpyne/game-music-emu/src/master/" target="_blank" rel="noopener noreferrer">Game Music Emulator</a> (GME) library. Luckily I found a fork of SDL_mixer called <a href="https://github.com/WohlSoft/SDL-Mixer-X" target="_blank" rel="noopener noreferrer">SDL_mixer X</a> which integrates GME into SDL. It was pretty straightforward to grab that and merge the changes into the port that Emscriten uses. I also needed to add GME to Emscripten's port system, which was pretty <a href="https://gist.github.com/connorjclark/b9e0986c518d0193031c71181c8e2fd3" target="_blank" rel="noopener noreferrer">straightforward</a>.</p>
<blockquote>
<p>I sent SDL_mixer a <a href="https://github.com/libsdl-org/SDL_mixer/pull/378" target="_blank" rel="noopener noreferrer">PR</a> for adding GME. If that gets merged, I'll also add a <code>gme</code> option to Emscripten. But for now, I'm just fine with my patching workflow.</p>
</blockquote>
<p>As for <code>zcmusic</code>, I just had to implement the small API surface using SDL_mixer directly. The native version of the library brings in format-specific audio handling libraries, so it's actually much simpler now because SDL_mixer handles all that format-specific logic.</p>
<h3 id="persisting-data" tabindex="-1">Persisting data</h3>
<p>By default, all data written to Emscripten's filesystem is only held in memory, and is lost when refreshing the page. Emscripten provides a simple interface to <a href="https://emscripten.org/docs/api_reference/Filesystem-API.html#filesystem-api-idbfs" target="_blank" rel="noopener noreferrer">mount a folder backed by IndexedDB</a>, which solves the problem of persistence, but many other issues still exist:</p>
<ol>
<li>Players of Zelda Classic have existing save files they may want to transfer into the browser</li>
<li>Players will want access to these files (either to make backups or share them), but browsers don't expose IndexedDB to non-technical users</li>
<li>Browsers avoid clearing data in IndexedDB if <code>navigator.storage.persist()</code> is called, but still: losing data such as save files (and especially a quest author's <code>.qst</code> file) is catastrophic, and I don't trust anything to live inside a browser forever</li>
</ol>
<p>Using the real filesystem would avoid all these issues. Luckily, there's been a lot of progress on this front in the last year: The <a href="https://web.dev/file-system-access/" target="_blank" rel="noopener noreferrer">Filesystem Access API</a> provides a way to prompt a user to share a folder with a page, even allowing the page to write back to it. Given <code>window.showDirectoryPicker()</code>, the browser opens a folder dialog prompt and the user's selection is given as a <code>FileSystemDirectoryHandle</code>.</p>
<blockquote>
<p>The only annoyance is that permission doesn't persist across multiple sessions, and even opening the permission prompt is (understandably) gated behind user interaction, so every subsequent visit I must show the user a permission flow. At the very least, the <code>FileSystemDirectoryHandle</code> can be cached in IndexedDB so the user doesn't need to specify which folder to use every time.</p>
</blockquote>
<p>Unfortunately only Chromium browsers have implemented <code>window.showDirectoryPicker()</code>; Firefox has no plans to implement, and Safari currently only supports a <a href="https://webkit.org/blog/12257/the-file-system-access-api-with-origin-private-file-system/" target="_blank" rel="noopener noreferrer">limited part of the API</a> called Origin Private Filesystem, which is not backed by real files on disk.</p>
<blockquote>
<p><a href="https://wicg.github.io/file-system-access/#sandboxed-filesystem" target="_blank" rel="noopener noreferrer">The Origin Private Filesystem</a> provides an origin-unique directory handle via <code>navigator.storage.getDirectory()</code>. The spec defines this folder to not necessarily map to real files on disk, so this is not viable for Zelda Classic</p>
</blockquote>
<p>Emscripten did not provide an interface for mounting a <code>FileSystemDirectoryHandle</code> to its own filesystem, so I wrote one myself. The existing IndexedDB interface is very similar to what I needed, and handles the logic of syncing deltas both ways rather nicely, so I based my interface on that. This seems like it'd be really useful to others, so I sent a <a href="https://github.com/emscripten-core/emscripten/pull/16804" target="_blank" rel="noopener noreferrer">patch to Emscripten</a>.</p>
<p>While I'm happy I can provide an ideal persistence story in Chromium, I still had to do something for other browsers. IndexedDB + <code>navigator.storage.persist()</code> isn't the worst thing in the world, but I needed to solve issues 1 and 2 above. To that end, a user can:</p>
<ol>
<li>download any individual file backed by IndexedDB</li>
<li>perform a one-way upload of a file or an entire folder into the browser (<a href="https://web.dev/browser-fs-access/" target="_blank" rel="noopener noreferrer"><code>browser-fs-access</code></a> helped here)</li>
</ol>
<h3 id="gamepads" tabindex="-1">Gamepads</h3>
<p>Zelda Classic is certainly playable with the keyboard, but it also supports gamepads. And so does the web and Emscripten! I was hopeful that things would "just work" here. I bought myself a nice Xbox controller to test things out and... nada. I noticed that the gamepad would only connect if I actively twiddled with its inputs while the page loaded. The bug could have been anywhere: Emscripten, my controller, SDL, Allegro, Allegro Legacy... so the first task was to narrow down a repro.</p>
<p>I wrote a quick <a href="https://gist.github.com/connorjclark/0b7268acd6bfa324c4db38dde7928110" target="_blank" rel="noopener noreferrer">SDL program</a> that prints when a joystick connects and disconnects. I compiled with Emscripten, loaded the page and it worked. So that just left Allegro/Allegro Legacy as the culprits. I did notice a difference between running when compiled for Mac vs for the web: On Mac, SDL detects a joystick immediately, but in the browser detection only happens after the first input on the controller. This is by design–the purpose is to <a href="https://developer.mozilla.org/en-US/docs/Web/API/Gamepad_API/Using_the_Gamepad_API#:~:text=In%20Firefox%2C%20gamepads-,are%20only%20exposed,-to%20a%20page" target="_blank" rel="noopener noreferrer">avoid a potential vector for fingerprinting</a>.</p>
<p>So that was a big clue–Allegro works only when twiddling the input at start up because it must be mishandling joysticks that are connected post-initialization. Pulling up <a href="https://github.com/liballeg/allegro5/blob/668a0a35afd4132dfeb86325d8f3e3c10628b529/src/sdl/sdl_joystick.c" target="_blank" rel="noopener noreferrer">Allegro's SDL interface for joysticks</a>, a variable <code>count</code> jumps out:</p>
<pre class="language-cpp"><code class="language-cpp"><span class="token keyword">void</span> <span class="token function">_al_sdl_joystick_event</span><span class="token punctuation">(</span>SDL_Event <span class="token operator">*</span>e<span class="token punctuation">)</span><br /><span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>count <span class="token operator"><=</span> <span class="token number">0</span><span class="token punctuation">)</span><br /> <span class="token keyword">return</span><span class="token punctuation">;</span><br /><br /> <span class="token comment">// ...</span><br /><span class="token punctuation">}</span><br /><br /><span class="token keyword">static</span> <span class="token keyword">bool</span> <span class="token function">sdl_init_joystick</span><span class="token punctuation">(</span><span class="token keyword">void</span><span class="token punctuation">)</span><br /><span class="token punctuation">{</span><br /> count <span class="token operator">=</span> <span class="token function">SDL_NumJoysticks</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// <<<<<<<<<<<< Only ever set once!</span><br /> joysticks <span class="token operator">=</span> <span class="token function">calloc</span><span class="token punctuation">(</span>count<span class="token punctuation">,</span> <span class="token keyword">sizeof</span> <span class="token operator">*</span> joysticks<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token comment">// ...</span><br /><span class="token punctuation">}</span></code></pre>
<p>For some unknown reason... all joystick events are ignored if there are no currently connected joysticks. The expectation in Allegro programs is to call <code>al_reconfigure_joysticks</code> (which would call <code>sdl_init_joystick</code> again) when a joystick is added or removed to recreate the internal data structures, but a program never gets a chance to do so because Allegro's SDL joystick driver never forwards <code>SDL_JOYDEVICEADDED</code> events when no joysticks are present. <a href="https://github.com/liballeg/allegro5/pull/1326" target="_blank" rel="noopener noreferrer">The fix</a> was straightforward: remove that unnecessary <code>count</code> guard, and fix a use-after-free bug from very unexpected behavior (to me, a web developer) of <a href="https://en.cppreference.com/w/c/memory/calloc" target="_blank" rel="noopener noreferrer"><code>calloc</code></a> when <code>0</code> is given as input.</p>
<blockquote>
<p>I found an unfortunate bug in Firefox where my Xbox controller is <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1763931" target="_blank" rel="noopener noreferrer">improperly mapped</a>.</p>
</blockquote>
<p>After all that, connecting a gamepad was working. The default joystick button mappings happened to be OK too, but I wanted to improve the existing Zelda Classic settings menu for configuring the gamepad controls: currently it gave no indication of what button an action is mapped to, only providing a button number (not a name). I found that Allegro does support a joystick button name api, so I used it but that didn't help so much:</p>
<div class="captioned-image">
<img style="max-width: min(700px, 100%)" src="https://hoten.cc/images/zc/buttons.png" alt="" />
<span>button button button button, button button, button 🦬</span>
</div>
<p>The problem was that Allegro's SDL joystick interface didn't know about SDL's API for getting a button name. <a href="https://github.com/liballeg/allegro5/pull/1327" target="_blank" rel="noopener noreferrer">The fix</a> was simple:</p>
<pre class="language-diff-cpp"><code class="language-diff-cpp">diff --git a/src/sdl/sdl_joystick.c b/src/sdl/sdl_joystick.c<br /><span class="token coord">--- a/src/sdl/sdl_joystick.c</span><br /><span class="token coord">+++ b/src/sdl/sdl_joystick.c</span><br /><span class="token unchanged language-cpp"><span class="token prefix unchanged"> </span> <span class="token keyword">static</span> <span class="token keyword">bool</span> <span class="token function">sdl_init_joystick</span><span class="token punctuation">(</span><span class="token keyword">void</span><span class="token punctuation">)</span><br /><span class="token prefix unchanged"> </span> info<span class="token operator">-></span>num_buttons <span class="token operator">=</span> bn<span class="token punctuation">;</span><br /><span class="token prefix unchanged"> </span> <span class="token keyword">int</span> b<span class="token punctuation">;</span><br /><span class="token prefix unchanged"> </span> <span class="token keyword">for</span> <span class="token punctuation">(</span>b <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> b <span class="token operator"><</span> bn<span class="token punctuation">;</span> b<span class="token operator">++</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /></span><span class="token deleted-sign deleted language-cpp"><span class="token prefix deleted">-</span> info<span class="token operator">-></span>button<span class="token punctuation">[</span>b<span class="token punctuation">]</span><span class="token punctuation">.</span>name <span class="token operator">=</span> <span class="token string">"button"</span><span class="token punctuation">;</span><br /></span><span class="token inserted-sign inserted language-cpp"><span class="token prefix inserted">+</span> info<span class="token operator">-></span>button<span class="token punctuation">[</span>b<span class="token punctuation">]</span><span class="token punctuation">.</span>name <span class="token operator">=</span> <span class="token function">SDL_IsGameController</span><span class="token punctuation">(</span>i<span class="token punctuation">)</span> <span class="token operator">?</span><br /><span class="token prefix inserted">+</span> <span class="token function">SDL_GameControllerGetStringForButton</span><span class="token punctuation">(</span>b<span class="token punctuation">)</span> <span class="token operator">:</span> <span class="token string">"button"</span><span class="token punctuation">;</span><br /></span><span class="token unchanged language-cpp"><span class="token prefix unchanged"> </span> <span class="token punctuation">}</span><br /><span class="token prefix unchanged"> </span> <span class="token punctuation">}</span><br /><span class="token prefix unchanged"> </span> <span class="token function">SDL_JoystickEventState</span><span class="token punctuation">(</span>SDL_ENABLE<span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<blockquote>
<p>I was curious how SDL could determine what the button names are, given that the Gamepad Web API has nothing for "give me the name of this button". Turns out, SDL uses the gamepad's device id (which the Web API does expose) to map known gamepads to a "standard" button layout. One such database can be found <a href="https://github.com/gabomdq/SDL_GameControllerDB/blob/master/gamecontrollerdb.txt" target="_blank" rel="noopener noreferrer">here</a> (but I think SDL ships with a much smaller set). These configurations are meant for standardizing rando gamepads to a sensible layout (such that the "right-side bottom button" has the same value to SDL independent of the gamepad hardware), but it also doubles as a button name store.</p>
</blockquote>
<h3 id="mobile-support" tabindex="-1">Mobile support</h3>
<div class="captioned-image">
<img src="https://hoten.cc/images/zc/mobile.png" alt="" width="50%" />
<span>Very basic touch controls</span>
</div>
<p>I thought it'd be cool to support mobile, but I didn't want to spend a lot of time on making touch controls feel good so the end result is a pretty subpar. The most tedious part was getting the browser <code>touch</code> events to work just-right. Actually fowarding them as events to Allegro was just a matter of exposing a C function to JavaScript that emitted a fake Allegro user event:</p>
<pre class="language-cpp"><code class="language-cpp"><span class="token keyword">bool</span> has_init_fake_key_events <span class="token operator">=</span> <span class="token boolean">false</span><span class="token punctuation">;</span><br />ALLEGRO_EVENT_SOURCE fake_src<span class="token punctuation">;</span><br /><span class="token keyword">extern</span> <span class="token string">"C"</span> <span class="token keyword">void</span> <span class="token function">create_synthetic_key_event</span><span class="token punctuation">(</span>ALLEGRO_EVENT_TYPE type<span class="token punctuation">,</span> <span class="token keyword">int</span> keycode<span class="token punctuation">)</span><br /><span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span>has_init_fake_key_events<span class="token punctuation">)</span><br /> <span class="token punctuation">{</span><br /> <span class="token function">al_init_user_event_source</span><span class="token punctuation">(</span><span class="token operator">&</span>fake_src<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">a5_keyboard_queue_register_event_source</span><span class="token punctuation">(</span><span class="token operator">&</span>fake_src<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> has_init_fake_key_events <span class="token operator">=</span> <span class="token boolean">true</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> ALLEGRO_EVENT event<span class="token punctuation">;</span><br /> event<span class="token punctuation">.</span>any<span class="token punctuation">.</span>type <span class="token operator">=</span> type<span class="token punctuation">;</span><br /> event<span class="token punctuation">.</span>keyboard<span class="token punctuation">.</span>keycode <span class="token operator">=</span> keycode<span class="token punctuation">;</span><br /> <span class="token function">al_emit_user_event</span><span class="token punctuation">(</span><span class="token operator">&</span>fake_src<span class="token punctuation">,</span> <span class="token operator">&</span>event<span class="token punctuation">,</span> <span class="token constant">NULL</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Luckily Gamepads work just fine on mobile devices. Here's me playing with a wireless Xbox controller on my phone:</p>
<div class="captioned-image">
<img src="https://hoten.cc/images/zc/gamepad.jpg" alt="" width="50%" />
<span>🥔📷 (had to use my webcam)</span>
</div>
<h3 id="pwa" tabindex="-1">PWA</h3>
<p>I used the following <a href="https://developers.google.com/web/tools/workbox" target="_blank" rel="noopener noreferrer">Workbox</a> config to generate a service worker:</p>
<pre class="language-js"><code class="language-js">module<span class="token punctuation">.</span>exports <span class="token operator">=</span> <span class="token punctuation">{</span><br /> <span class="token literal-property property">runtimeCaching</span><span class="token operator">:</span> <span class="token punctuation">[</span><br /> <span class="token punctuation">{</span><br /> <span class="token literal-property property">urlPattern</span><span class="token operator">:</span> <span class="token regex"><span class="token regex-delimiter">/</span><span class="token regex-source language-regex">png|jpg|jpeg|svg|gif</span><span class="token regex-delimiter">/</span></span><span class="token punctuation">,</span><br /> <span class="token literal-property property">handler</span><span class="token operator">:</span> <span class="token string">'CacheFirst'</span><span class="token punctuation">,</span><br /> <span class="token punctuation">}</span><span class="token punctuation">,</span><br /> <span class="token punctuation">{</span><br /> <span class="token comment">// Match everything except the wasm data file, which is cached in</span><br /> <span class="token comment">// IndexedDB by emscripten.</span><br /> <span class="token function-variable function">urlPattern</span><span class="token operator">:</span> <span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> url <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token operator">!</span>url<span class="token punctuation">.</span>pathname<span class="token punctuation">.</span><span class="token function">endsWith</span><span class="token punctuation">(</span><span class="token string">'.data'</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">handler</span><span class="token operator">:</span> <span class="token string">'NetworkFirst'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">options</span><span class="token operator">:</span> <span class="token punctuation">{</span><br /> <span class="token literal-property property">matchOptions</span><span class="token operator">:</span> <span class="token punctuation">{</span><br /> <span class="token comment">// Otherwise the html page won't be cached (it can have query parameters).</span><br /> <span class="token literal-property property">ignoreSearch</span><span class="token operator">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span><br /> <span class="token punctuation">}</span><span class="token punctuation">,</span><br /> <span class="token punctuation">}</span><span class="token punctuation">,</span><br /> <span class="token punctuation">}</span><span class="token punctuation">,</span><br /> <span class="token punctuation">]</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">swDest</span><span class="token operator">:</span> <span class="token string">'sw.js'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">skipWaiting</span><span class="token operator">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">clientsClaim</span><span class="token operator">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">offlineGoogleAnalytics</span><span class="token operator">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre>
<p>This gets me offline support, although notably there is no precaching: I chose to avoid precaching because there's ~6GB of quest data which is fetched only when needed, so the user will need to load a particular quest while online at least once for it to work offline. So I didn't see the point in precaching any part of the webapp.</p>
<p>With a service worker, and a <a href="https://web.zquestclassic.com/manifest.json" target="_blank" rel="noopener noreferrer">manifest.json</a>, the webapp can be installed as a PWA. I listen for the <code>beforeinstallprompt</code> event to display my own install prompt:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">const</span> installEl <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">createElement</span><span class="token punctuation">(</span><span class="token string">'button'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br />installEl<span class="token punctuation">.</span>textContent <span class="token operator">=</span> <span class="token string">'Install as App'</span><span class="token punctuation">;</span><br />installEl<span class="token punctuation">.</span>classList<span class="token punctuation">.</span><span class="token function">add</span><span class="token punctuation">(</span><span class="token string">'panel-button'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br />installEl<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span><span class="token string">'click'</span><span class="token punctuation">,</span> <span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span>deferredPrompt<span class="token punctuation">)</span> <span class="token keyword">return</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">const</span> <span class="token punctuation">{</span> outcome <span class="token punctuation">}</span> <span class="token operator">=</span> <span class="token keyword">await</span> deferredPrompt<span class="token punctuation">.</span><span class="token function">prompt</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>outcome <span class="token operator">===</span> <span class="token string">'accepted'</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> deferredPrompt <span class="token operator">=</span> <span class="token keyword">undefined</span><span class="token punctuation">;</span><br /> installEl<span class="token punctuation">.</span>textContent <span class="token operator">=</span> <span class="token string">'Installed! Open from home screen for better experience'</span><span class="token punctuation">;</span><br /> <span class="token function">setTimeout</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> installEl<span class="token punctuation">.</span><span class="token function">remove</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token number">1000</span> <span class="token operator">*</span> <span class="token number">5</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">let</span> deferredPrompt<span class="token punctuation">;</span><br />window<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span><span class="token string">'beforeinstallprompt'</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token parameter">e</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> e<span class="token punctuation">.</span><span class="token function">preventDefault</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> deferredPrompt <span class="token operator">=</span> e<span class="token punctuation">;</span><br /><br /> document<span class="token punctuation">.</span><span class="token function">querySelector</span><span class="token punctuation">(</span><span class="token string">'.panel-buttons'</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">append</span><span class="token punctuation">(</span>installEl<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>In Chrome, when a PWA is installed the view transitions to the fullscreen, standalone version of the app. Unfortunately on Android, when installed there is no such transition, which makes for an awkward flow (the user can choose to close the browser tab and hunt down the newly installed app, or they can continue in the current browser tab and on subsequent visits use the app entry).</p>
<blockquote>
<p>Chrome 102 just landed, which introduces <a href="https://blog.chromium.org/2022/04/chrome-102-window-controls-overlay-host.html#:~:text=File%20Handlers%20Web%20App%20Manifest%20Member" target="_blank" rel="noopener noreferrer"><code>file_handlers</code></a>. Definitely something I'll eventually add to handle opening <code>.qst</code> files from the OS!</p>
</blockquote>
<h2 id="takeaways" tabindex="-1">Takeaways</h2>
<ul>
<li>As soon as you run into what seems like an intractable bug, stop trying to debug it from the context of your application and try to make a minimial reproduction. It will become easier to reason about the problem, and if the bug belongs to a dependency you will have a ready-made repro to provide in bug report.</li>
<li>File bug reports and upstream bug fixes when possible! But also, have <em>some way</em> to tweak your dependencies, be it with hard forks or a patching system. You can't allow a bug in a dependency that you know how to resolve stall progress.</li>
<li>Break down problems with unknown solutions. For example, my first attempt at porting Zelda Classic was over a year ago, and that failed miserably because I jumped right into the end task of "actually port it", without first taking the time to really learn the tools involved, which resulted in me spinning my wheels. This time around, I avoided that by making my first task to fully understand how to port the simplest Allegro program.</li>
</ul>