Wayne Conrad2017-01-18T18:01:16+00:00http://wconrad.github.comWayne ConradJörg W Mittag: How functional languages handle side-effects2017-01-18T07:00:00+00:00http://wconrad.github.com/20170118/jorg-w-mittag-on-functional-languages<p>On the <a href="https://chat.stackoverflow.com/rooms/44914/ruby-sometimes-on-rails">stackoverflow Ruby
chat</a>,
conversation about the difference between statements and expressions
soon turned to a
<a href="https://chat.stackoverflow.com/transcript/44914">discourse</a> about how
functional languages handle side-effects such as printing. Parts of
that conversation are reproduced here with the permission of
participants <a href="https://stackoverflow.com/users/2988/j%C3%B6rg-w-mittag">Jörg W
Mittag</a> and
<a href="https://stackoverflow.com/users/2115680/marc-andre">Marc-Andre</a>.</p>
<p>Jörg W Mittag:</p>
<blockquote>
<p>An expression evaluates to a value. A statement has no value, only a
side-effect. Ruby has only expressions, there are no statements. Even
a method definition, a module/class definition, or a loop return
values. (Even if that value is just nil.)</p>
</blockquote>
<blockquote>
<p>The thing is that this distinction is somewhat arbitrary. In fact,
languages which distinguish between both usually have an “expression
statement”, which is a statement that consists only of a single
expression and throws away its value. They also sometimes have
“statement expressions” which are expressions that consist of a
statement and evaluate to some bogus value (e.g. NULL).</p>
</blockquote>
<blockquote>
<p>Functional languages typically have no statements. After all, they
are all about values and something that has no value has no value in
FP ;-)</p>
</blockquote>
<blockquote>
<p>What impure functional languages have instead is a “Unit Type”. The
unit type is a type that is only inhabited by one value. Typically,
that value (and the type) is written as (), i.e. the empty
tuple. (There can only be one tuple that has no element.) “Statements”
are then expressions of type (), and “void procedures” are functions
that return (). E.g. in Scala, println returns Unit and assignment
evaluates to ().</p>
</blockquote>
<p>Marc-Andre:</p>
<blockquote>
<p>I think I get it, but it still a bit blurry, but it’s a bit better
now. Since some statement don’t return value per se, they use a
generic value to return something?</p>
</blockquote>
<p>Jörg W Mittag:</p>
<blockquote>
<p>In a pure functional language, the very idea of a statement doesn’t
make sense: there are no side-effects, so something that doesn’t
return a value just doesn’t do anything.</p>
</blockquote>
<p>Marc-Andre:</p>
<blockquote>
<p>In a pure functional language what would a println function return
or would it even be define like a “normal” println ?</p>
</blockquote>
<p>Jörg W Mittag:</p>
<blockquote>
<p>However, in an impure language, there can be side-effects. One way
of handling this is to separate things that have values (expressions,
functions) from things that have side-effects (statements,
procedures). But that complicates the language and the syntax. So,
what we do instead is to define a value that carries zero information
(like the empty tuple) of a type that has this value as its only
instance, and define this “information-less” value as the return value
of something that has no meaningful value. Kind-of like puts in Ruby
returns nil, because, well, there is no meaningful thing it could
return. @Marc-Andre In a pure functional language, println would take
two arguments: a string and the state of the world, and return a new
state of the world in which the string is printed to the screen. (At
least that’s one way to interpret it.)</p>
</blockquote>
<blockquote>
<p>However, we have a problem here: the caller could have held onto the
old world value! Now our caller has two worlds at its disposal: one in
which the string is printed and one in which it isn’t. What do we do
if it calls println again with the old world value as input? We can’t
“unprint” what we printed (especially if we printed to an actual
printer instead the screen.) We need to make sure that “Worlds” don’t
get re-used. There are some type system tricks we can use: there is a
concept called linear types, which are types that can only be used
once. Clean works this way, all IO functions take and return World
types that are based on linear types.</p>
</blockquote>
<blockquote>
<p>Haskell goes a different route: it uses a concept called
monads. Monads allow you to “enrich” a computation with additional
structure but hide that structure away from the computation. So, the
“world values” never actually get exposed, they are always hidden (and
since they are never exposed, they don’t even really exist in the
runtime at all). In both cases, the result is the same: the pure
functional program returns a purely functional value that is basically
a description of the side-effects that the program would like to
perform. This description is then interpreted by the impure language
runtime. This allows the programming language semantics themselves to
remain pure. Someone once said that Haskell has better support for
side-effects than C, since in Haskell, they are first-class, can be
passed around as arguments, can be returned as values, can be stored
in variables, can be composed. In C, they just happen.</p>
</blockquote>
<p>Marc-Andre:</p>
<blockquote>
<p>Really interesting! It really amazing how “simple” things like
printing to a screen, can be such an important task and need a lot of
thought to it!</p>
</blockquote>
<p>Jörg W Mittag:</p>
<blockquote>
<p>Purely functional programming would be pretty boring otherwise. You
need side-effects, otherwise all you do is make the CPU hot. Which,
some might argue, is also a side-effect.</p>
</blockquote>
A Virtualbox Networking Recipe2016-03-22T07:00:00+00:00http://wconrad.github.com/20160322/a-virtualbox-networking-recipe<p>This is a recipe for setting up a virtual network when using Oracle’s
VirtualBox software.</p>
<h1 id="the-requirements">The requirements</h1>
<ul>
<li>
<p>There are multiple virtual machines on a virtual network</p>
</li>
<li>
<p>Each virtual machine needs to be able to talk to any other virtual
machine on the same virtual network.</p>
</li>
<li>
<p>The host also needs to be able to talk to any of those virtual
machines.</p>
</li>
<li>
<p>Each virtual machine has internet access.</p>
</li>
</ul>
<h1 id="conditions">Conditions</h1>
<p>I tested this recipe with:</p>
<ul>
<li>
<p>Host OS: Debian 8 (Jessie), 64-bit, amd64</p>
</li>
<li>
<p>Guest OS: “</p>
</li>
<li>
<p>VirtualBox 5.0.16 r105871</p>
</li>
<li>
<p>The host is on the class C network 192.168.0.0/24</p>
</li>
</ul>
<h1 id="the-recipe">The recipe</h1>
<p>In this recipe, each VM is dual-homed.The first network interfaces
gives the VM access to the internet through the NatNetwork, and also
gives the host access to the VMs. The second network interface
connects the host and VMs together through a “host-only network.”</p>
<h1 id="configuring-the-host-only-network">Configuring the Host-only network</h1>
<ul>
<li>
<p>In VirtualBox, select File/Preferences/Network</p>
</li>
<li>
<p>Select the “Host-only Networks” tab</p>
</li>
<li>
<p>Configure a host-only network. By default, there’s one called
vboxnet0. You can use it, or create another. The settings:</p>
<ul>
<li>
<p>IPv4 Address: 192.168.51.1</p>
</li>
<li>
<p>IPv4 Network Mask: 255.255.255.0</p>
</li>
<li>
<p>IPv6 Address: (blank)</p>
</li>
<li>
<p>IPv5 Netowrk Mask Length: 0</p>
</li>
</ul>
</li>
<li>
<p>DHCP Server:</p>
<ul>
<li>Enable Server: not checked</li>
</ul>
</li>
</ul>
<h1 id="configuring-each-vms-virtualbox-network">Configuring each VM’s VirtualBox Network</h1>
<ul>
<li>
<p>In VirtualBox, select the VM’s Settings</p>
</li>
<li>
<p>Select the Network tab.</p>
</li>
<li>
<p>Leave Adapter 1 set to “NAT”. This is the adapter that gives the VM
access to the internet.</p>
</li>
<li>
<p>Enable Adapter 2.</p>
</li>
<li>
<p>Attach adapter 2 to “host only adapter”</p>
</li>
<li>
<p>Select the virtual network you configured above (probably
“vboxnet0”)</p>
</li>
</ul>
<h1 id="configuring-each-vm-oss-network">Configuring each VM OS’s network.</h1>
<p>We now configure each VM’s OS to be dual-homed, with one DHCP
interface for internet, and one static interface for communicating
with other VMs and the host.</p>
<p>Edit /etc/apt/interfaces. You should see something like this:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>...
allow-hotplug eth0
iface eht0 inet dhcp
</code></pre>
</div>
<p>This is the configuration for the NAT interface, the one that connects
the VM to the internet. Add the configuration for the “host-only
network” interface, the one that connects the VM to other VMs and to
the host:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>auto eth1
iface eth1 inet static
address 192.168.51.2 # Unique for each VM. Do not use .1.
netmask 255.255.255.0
</code></pre>
</div>
<p>Now bring up the interface using <code class="highlighter-rouge">ifup eth1</code></p>
<p>Do this for each VM.</p>
<h1 id="testing">Testing</h1>
<p>From each VM, you should be able to get to the internet:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>ping google.com
</code></pre>
</div>
<p>You should be able to get to another vm:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>ping 192.168.51.2
</code></pre>
</div>
<p>From the host, you should be able to get to each vm:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>ping 192.168.51.2
ping 192.168.51.3
</code></pre>
</div>
Simh/Nova - Mounting a second hard drive2015-12-10T07:00:00+00:00http://wconrad.github.com/20151210/nova-second-drive<p><em>This post is part of the series on <a href="/20151207/simh-toc.html">Learning the Data General Nova
with simh</a>.</em></p>
<p>Let’s mount a second hard drive. First, two ways that don’t work, and
then the way that does.</p>
<h1 id="let-simh-make-a-new-file-doesnt-work">Let simh make a new file (doesn’t work)</h1>
<p>From the simh documentation, it seems that you should be able to name
a non-existent file when using simh’s <em>attach</em> command, so let’s try
that. First, boot the simulator normally. Then use ^e (control-e) to
enter the simh prompt:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>R
^e
simh>
</code></pre>
</div>
<p>The “show” command will show us the disk devices:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sim> show dkp
DKP, 4 units
DKP0, 1247KW, attached to rdos_d31.dsk, write enabled, 4047 (Diablo 31)
DKP1, 1247KW, not attached, write enabled, autosize
DKP2, 1247KW, not attached, write enabled, autosize
DKP3, 1247KW, not attached, write enabled, autosize
</code></pre>
</div>
<p>The “attach” command will attach the file to the DKP1:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sim> att dkp1 foo.dsk
sim> show dkp
DKP, 4 units
DKP0, 1247KW, attached to rdos_d31.dsk, write enabled, 4047 (Diablo 31)
DKP1, 1247KW, attached to foo.dsk, write enabled, 4047 (Diablo 31)
DKP2, 1247KW, not attached, write enabled, autosize
DKP3, 1247KW, not attached, write enabled, autosize
</code></pre>
</div>
<p>This created a zero-size file:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sim> ! ls -l foo.dsk
-rw-r--r-- 1 wayne wayne 0 Dec 7 15:56 foo.dsk
</code></pre>
</div>
<p>(<code class="highlighter-rouge">!</code> is the simh command to run a shell command. <code class="highlighter-rouge">ls -l foo.dsk</code> is
the Linux command to list the file and its attributes.)</p>
<p>Go back to RDOS, and tell it to initialize the drive. The <code class="highlighter-rouge">/f</code> switch tells RDOS
to format the drive.</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sim> cont
R
init dp1/f
DISK FORMAT ERROR: DP1
R
</code></pre>
</div>
<p>Hmm, that didn’t work.</p>
<p>It also doesn’t work to make an all-zero file that is the same size as
rdos_d31.dsk. I tried it. RDOS, or simh, or both, care about some
data in the file.</p>
<h1 id="use-a-copy-of-the-rdos-disk">Use a copy of the RDOS disk.</h1>
<p>What does work is to make a copy of the RDOS disk and “init” it with
the switch to format the disk. Escape to simh:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>R
^e
Simulation stopped, PC: 41737 (STA 3,13)
</code></pre>
</div>
<p>If there’s a disk attached to dkp1, detach it:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sim> det dkp1
</code></pre>
</div>
<p>Use a shell command to copy the RDOS disk to the new disk, attach it,
and return to RDOS:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sim> ! cp rdos_d31.dsk wayne.dsk
sim> att dkp1 wayne.dsk
sim> cont
</code></pre>
</div>
<p>Now initialize the drive with the /f (format) switch:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>R
init/f dp1
CONFIRM? YES
</code></pre>
</div>
<p>The DIR command makes that drive the current drive:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>R
dir dp1
</code></pre>
</div>
<p>And the list command shows that it is empty:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>R
list
</code></pre>
</div>
<h1 id="references">References</h1>
<p><a href="http://bitsavers.trailing-edge.com/pdf/dg/software/rdos/093-000109-01_RDOS_Command_Line_Interpreter.pdf">RDOS/DOS Command Line Interpreter User’s
Manual</a></p>
<ul>
<li>p. 3-74 - <code class="highlighter-rouge">DIR</code> command</li>
<li>p. 3-77 - <code class="highlighter-rouge">INIT</code> command</li>
</ul>
Simh/Nova - Scripting Startup2015-12-09T07:00:00+00:00http://wconrad.github.com/20151209/simh-scripting-startup<p><em>This post is part of the series on <a href="/20151207/simh-toc.html">Learning the Data General Nova
with simh</a>.</em></p>
<h1 id="starting-simh-the-hard-way">Starting simh the hard way</h1>
<p>In order to start simh with RDOS, we need to enter the same series of
commands every time:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>$ dgnova
NOVA simulator V3.8-1
sim> att dkp0 rdos_d31.dsk
sim> set tti dasher
sim> boot dkp0
NOVA simulator V3.8-1
</code></pre>
</div>
<h1 id="starting-simh-with-a-script">Starting simh with a script</h1>
<p>We can put those commands into a file so that we don’t have to type
them. Create a file boot.simh with the simh commands:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>att dkp0 rdos_d31.dsk
set tti dasher
boot dkp0
</code></pre>
</div>
<p>If we pass the name of that file to simh, it will run the commands
from that file:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>$ dgnova boot.simh
NOVA simulator V3.8-1
</code></pre>
</div>
<p>But I don’t want to type even that much. Let’s create the file
boot.sh:</p>
<div class="highlighter-rouge"><pre class="highlight"><code><span class="c">#!/bin/sh</span>
dgnova boot.simh
</code></pre>
</div>
<p>Mark it as executable:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>chmod +x boot.sh
</code></pre>
</div>
<p>And now start the sim with it:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>$ ./boot.sh
NOVA simulator V3.8-1
</code></pre>
</div>
Simh/Nova - Recovering from a Bad Shutdown2015-12-08T07:00:00+00:00http://wconrad.github.com/20151208/simh-nova-bad-shutdown<p><em>This post is part of the series on <a href="/20151207/simh-toc.html">Learning the Data General Nova
with simh</a>.</em></p>
<p>RDOS keeps an open count for each file. That’s not unusual. What is
unusual is that it keeps that count on disk. I’m not sure why. Can
two Nova CPUs can access the same physical disk concurrently? That
would be one reason to keep use counts on disk.</p>
<h1 id="use-counts-after-a-normal-start">Use counts after a normal start</h1>
<p>Let’s start the simulation as above, and then use the LIST command to
see some files:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>R
list/e
...
BACKUP. 978 D 05/30/85 21:05 05/31/85 [005011] 0
CLI.S0 0 D 12/07/115 11:45 12/07/115 [005015] 1
CLI.SV 11264 SD 05/02/85 20:05 05/31/85 [000206] 0
OVLDR.SV 7168 SD 05/02/85 21:33 05/31/85 [001316] 0
CLI.ER 3584 D 05/02/85 20:05 12/07/115 [000377] 1
ASM.SV 10240 SD 05/01/85 18:59 05/31/85 [001647] 0
ENPAT.SV 3584 SD 05/02/85 21:35 05/31/85 [002053] 0
NSID.SR 793 D 10/20/83 20:44 05/31/85 [002165] 0
CLI.OL 50176 C 05/02/85 20:05 12/07/115 [000235] 1
MCABOOT.SV 5120 SD 05/08/85 10:28 05/31/85 [001127] 0
...
</code></pre>
</div>
<p>I gave the command as <code class="highlighter-rouge">list/e</code>, but it could have been given as <code class="highlighter-rouge">LIST/E</code>
just as well. RDOS is case insensitive.</p>
<p>The <code class="highlighter-rouge">/E</code> switch tells RDOS to list the dates and use counts for each
file. The last number on each line is the file’s use count.</p>
<h1 id="shut-down-improperly">Shut down improperly</h1>
<p>Get the simh prompt using ^e, and then exit simh. This will
simulation pulling the plug on the Nova:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>R
^e
Simulation stopped, PC: 41741 (MOVZL# 1,1,SNC)
sim> quit
Goodbye
</code></pre>
</div>
<p>Now start simh/nova. During boot, RDOS will give you a warning
indicating that there are non-zero use counts:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>NOVA simulator V3.8-1
Filename?
Partition in use - Type C to continue
continue
NOVA RDOS Rev 7.50
Date (m/d/y) ? 12/7/2015
Time (h:m:s) ? 12:24:24
R
</code></pre>
</div>
<p>Let’s see the counts</p>
<div class="highlighter-rouge"><pre class="highlight"><code>R
list/e
...
OVLDR.SV 7168 SD 05/02/85 21:33 05/31/85 [001316] 0
CLI.ER 3584 D 05/02/85 20:05 12/07/115 [000377] 2
ASM.SV 10240 SD 05/01/85 18:59 05/31/85 [001647] 0
ENPAT.SV 3584 SD 05/02/85 21:35 05/31/85 [002053] 0
NSID.SR 793 D 10/20/83 20:44 05/31/85 [002165] 0
CLI.OL 50176 C 05/02/85 20:05 12/07/115 [000235] 2
MCABOOT.SV 5120 SD 05/08/85 10:28 05/31/85 [001127] 0
...
</code></pre>
</div>
<p>No use count should be greater than 1 right now, so things are not
right.</p>
<h1 id="clearning-use-counts">Clearning use counts</h1>
<p>Let’s use the <code class="highlighter-rouge">CLEAR</code> command to clear the use counts:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>R
clear/a/d/v
CLEARED $TTI.
CLEARED $PTP.
CLEARED $PTR.
CLEARED $TTO.
CLEARED $TTP.
CLEARED $TTR.
CLEARED $LPT.
CLEARED SYS.DR
</code></pre>
</div>
<p><code class="highlighter-rouge">clear/a/d/v</code> clears most files, but not all of them. Most notably:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>R
list/e cli.-
CLI.S0 0 D 12/07/115 12:25 12/07/115 [005015] 1
CLI.SV 11264 SD 05/02/85 20:05 05/31/85 [000206] 0
CLI.ER 3584 D 05/02/85 20:05 12/07/115 [000377] 2
CLI.OL 50176 C 05/02/85 20:05 12/07/115 [000235] 2
CLI.CM 131 D 12/14/95 16:20 12/14/95 [005055] 0
</code></pre>
</div>
<p>We have to name these files explicitly to clear their use counts:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>R
CLEAR/V CLI.S0 CLI.ER CLI.OL
CLEARED CLI.S0
CLEARED CLI.ER
CLEARED CLI.OL
R
list/e cli.-
CLI.S0 0 D 12/07/115 12:25 12/07/115 [005015] 0
CLI.SV 11264 SD 05/02/85 20:05 05/31/85 [000206] 0
CLI.ER 3584 D 05/02/85 20:05 12/07/115 [000377] 0
CLI.OL 50176 C 05/02/85 20:05 12/07/115 [000235] 0
CLI.CM 131 D 12/14/95 16:20 12/14/95 [005055] 0
</code></pre>
</div>
<p>The use counts have been reset, but they are now wrong: Some of the
files we’ve cleared the use count on are actually in use (certain of
the CLI.- files). The use count is used to prevent modification or
deletion of a file that is in use, so as long as we leave those alone,
it seems to me that there should be no harm in leaving their use
counts incorrect set to 0. I don’t know for sure.</p>
<h1 id="normal-shutdown-and-restart">Normal shutdown and restart</h1>
<p>A normal shutdown and reboot would return the counts to their normal
values, so let’s try it:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>release dp0
MASTER DEVICE RELEASED
</code></pre>
</div>
<p>We then exit simh and reboot (not shown), and see:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>NOVA simulator V3.8-1
Filename?
NOVA RDOS Rev 7.50
Date (m/d/y) ? 12/7/2015
Time (h:m:s) ? 12:47:40
R
</code></pre>
</div>
<p>There was no “PARTITION” warning, which is good. Let’s check those
use counts:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>R
LIST CLI.ER CLI
...
OVLDR.SV 7168 SD 05/02/85 21:33 05/31/85 [001316] 0
CLI.ER 3584 D 05/02/85 20:05 12/07/115 [000377] 1
ASM.SV 10240 SD 05/01/85 18:59 05/31/85 [001647] 0
ENPAT.SV 3584 SD 05/02/85 21:35 05/31/85 [002053] 0
NSID.SR 793 D 10/20/83 20:44 05/31/85 [002165] 0
CLI.OL 50176 C 05/02/85 20:05 12/07/115 [000235] 1
MCABOOT.SV 5120 SD 05/08/85 10:28 05/31/85 [001127] 0
...
</code></pre>
</div>
<p><em>Note</em>: The clear command will need to be issued on every partition
and directory that was “initialized” (mounted) at the time the system
was improperly shut down. We didn’t have any directories initialized,
so didn’t need to do that.</p>
<h1 id="references">References</h1>
<p><a href="http://bitsavers.trailing-edge.com/pdf/dg/software/rdos/093-000109-01_RDOS_Command_Line_Interpreter.pdf">RDOS/DOS Command Line Interpreter User’s
Manual</a></p>
<ul>
<li>p. 4-17 - <code class="highlighter-rouge">CLEAR</code> command</li>
<li>p. 4-42 - <code class="highlighter-rouge">LIST</code> command</li>
</ul>
Simh/Nova - Hello World2015-12-07T19:01:00+00:00http://wconrad.github.com/20151207/simh-nova-hello-world<p><em>This post is part of the series on <a href="/20151207/simh-toc.html">Learning the Data General Nova
with simh</a>.</em></p>
<p><a href="https://github.com/simh/simh">simh</a> is a fantastic program that simulates more than a dozen
computers. Its documentation, although complete, is unfriendly to
someone who has never used it. What it lacks is something I consider
essential to documentation: The “Hello, world” examples. A “Hello,
world” example is something that gets you started even if you know
nothing at all about a program. Without that, you’re left with a lot
of details about switches and functions, but no idea how to use them
to do anything. It’s like having a dictionary but not knowing how to
put the words together into a simple sentence.</p>
<p>I’m exploring the <a href="https://en.wikipedia.org/wiki/Data_General_Nova">Data General Nova</a> using simh. This is my
journal of that experience. I’ll document what I learn, including how
to use simh to simulate the Nova. At least at first, I’ll try to keep
the documentation explicit enough and detailed enough that you could
retrace my steps.</p>
<h1 id="my-environment">My environment</h1>
<p>I run Linux, so any shell commands I provide will need to be changed to work with Windows.</p>
<h1 id="install-simh">Install simh</h1>
<p>My OS, Debian Linux 8.2 (aka “jessie”), has a package for simh, so I
installed it:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">sudo apt-get install simh</code></pre></figure>
<p>This installed simh version 3.8.1.</p>
<h1 id="download-the-operating-system">Download the operating system</h1>
<p>On the <a href="http://simh.trailing-edge.com/software.html">“Software Kits” page</a> of the simh web site, you will find a
file titled “RDOS V7.5 for the Nova” and named <code class="highlighter-rouge">rdosswre.tar.Z</code>.
Download it:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">wget http://simh.trailing-edge.com/kits/rdosswre.tar.Z</code></pre></figure>
<p>And unpack it:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">tar xzf rdosswre.tar.Z</code></pre></figure>
<p>This created the directories and files:</p>
<ul>
<li>Disks/</li>
<li>Disks/rdos_d31.dsk</li>
<li>Licenses/</li>
<li>Licenses/rdos_license.txt</li>
<li>Licenses/README.txt</li>
</ul>
<p>Move the disk image and get rid of the rest:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">mv Disks/rdos_d31.dsk .
rm -rf Disks Licenses</code></pre></figure>
<h1 id="starting-simh-and-rdos">Starting simh and RDOS</h1>
<div class="highlighter-rouge"><pre class="highlight"><code>$ dgnova
NOVA simulator V3.8-1
sim> att dkp0 rdos_d31.dsk
sim> set tti dasher
sim> boot dkp0
Filename?
NOVA RDOS Rev 7.50
Date (m/d/y) ? 12/7/2015
Time (h:m:s) ? 5:23:20
R
</code></pre>
</div>
<p>The <code class="highlighter-rouge">R</code> is RDOS’s prompt. The Nova is booted, runing RDOS, and
awaiting our command.</p>
<p>Yes, you’ll need to enter the date and time each time you boot. The
Nova had no battery powered clock that kept the time while the
computer was off.</p>
<p>simh is quite scriptable, so maybe later I’ll see if I can script the
entering of the date and time.</p>
<h2 id="simh-devices">Simh devices</h2>
<p>Simh calls each of the computer’s elements a “device,” including the
CPU. Let’s use some simh commands to examine devices. Although our
simulator is running, we can interrupted it and get back to the simh
command prompt by typing ^e (control-e):</p>
<div class="highlighter-rouge"><pre class="highlight"><code>R
^e
Simulation stopped, PC: 41741 (MOVZL# 1,1,SNC)
sim>
</code></pre>
</div>
<p>Note: The ^e shown above won’t show on your screen.</p>
<p>The simulation is paused and we’re at the simh prompt. Let’s list all
of the devices:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sim> show dev
NOVA simulator configuration
CPU
PTR
PTP
TTI
TTO
TTI1
TTO1
RTC, 60Hz
PLT
LPT
DSK
DKP, 4 units
MTA, 8 units
QTY, disabled
ALM, lines=64
</code></pre>
</div>
<p>DKP is the controller for the moving head disk controller, which can
control up to four disk drives. These disk drives do the same thing
as a modern disk drive, although they are far slower, far smaller (in
capacity), far larger (in physical size), and much more expensive.
Oh, and they are removable–the media is in a “pack” that can be
removed. Otherwise, they are exactly the same as a modern hard drive.
Let’s look at that controller:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sim> show DKP
DKP, 4 units
DKP0, 1247KW, attached to rdos_d31.dsk, write enabled, 4047 (Diablo 31)
DKP1, 1247KW, not attached, write enabled, autosize
DKP2, 1247KW, not attached, write enabled, autosize
DKP3, 1247KW, not attached, write enabled, autosize
</code></pre>
</div>
<p>To get out of the simh prompt and back to the simulation:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sim> cont
</code></pre>
</div>
<h1 id="displaying-the-date-and-time">Displaying the date and time</h1>
<p>The <code class="highlighter-rouge">GTOD</code> command displays the date and time:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>R
GTOD
12/07/115 05:43:58
</code></pre>
</div>
<p>115 is how it display the year “2015”. It’s interesting that the OS
handles the year 2015 at all. After all, RDOS is from the 70’s.
Either it’s just a lucky accident that the year 2015 works, or it only
appears to work, or someone was thinking 25 years ahead when they
created RDOS. I prefer to think someone was thinking 25 years ahead.</p>
<h1 id="stopping-rdos-and-existing-simh">Stopping RDOS and existing simh</h1>
<p>RDOS should be shut down by unmounting the device that RDOS is mounted
on:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>R
RELEASE DP0
MASTER DEVICE RELEASED
HALT instruction, PC: 00001 (INC 2,2)
sim>
</code></pre>
</div>
<p>To exit simh, use the <code class="highlighter-rouge">quit</code> command:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sim> quit
$
</code></pre>
</div>
<h1 id="references">References</h1>
<p><a href="http://bitsavers.trailing-edge.com/pdf/dg/software/rdos/093-000109-01_RDOS_Command_Line_Interpreter.pdf">RDOS/DOS Command Line Interpreter User’s
Manual</a></p>
<ul>
<li>p. 4-37 - GTOD command</li>
<li>p. 4-59 - RELEASE command</li>
</ul>
Simh/Nova - Table of Contents2015-12-07T19:00:00+00:00http://wconrad.github.com/20151207/simh-toc<p>Posts in the series “Learning the Data General Nova with the simh
emulator”:</p>
<ul>
<li><a href="/20151206/memories-of-minicomputers.html">Memories of Minicomputers</a></li>
<li><a href="/20151207/simh-nova-hello-world.html">Hello World</a></li>
<li><a href="/20151208/simh-nova-bad-shutdown.html">Recovering from a Bad Shutdown</a></li>
<li><a href="/20151209/simh-scripting-startup.html">Scripting simh startup</a></li>
<li>Mounting a second hard drive</li>
</ul>
Memories of minicomputers: The Modcomp II, and the Data General Nova2015-12-06T19:00:00+00:00http://wconrad.github.com/20151206/memories-of-minicomputers<p><em>This post is part of the series on <a href="/20151207/simh-toc.html">Learning the Data General Nova
with simh</a>.</em></p>
<p>Once upon a time, I worked in a small computer lab attached to an
Exxon Nuclear research facility. There were two 16-bit minicomputers:
a <a href="https://en.wikipedia.org/wiki/MODCOMP">Modcomp</a> II that my boss and I programmed, and a <a href="https://en.wikipedia.org/wiki/Data_General_Nova">Data General
Nova</a> that another small team programmed. The computers were
hooked up to a laser lab where humongous no-kidding
burn-holes-in-things lasers were being used to refine uranium. I
can’t tell you how powerful the lasers were, because that was
(probably still is) classified, but I’ve forgotten anyhow. State
secrets are safe with someone whose memory is as bad as mine.</p>
<p>The Modcomp II minicomputer that I programmed was hooked up to sensors
that the computer used to collect data during the experiments, storing
the data to disk files. My boss wrote the software that collected the
data, and I wrote the FORTRAN IV programs that charted the data. We
had this fancy Techtronics graphing terminal that my code would draw
on, and then a thermal printer that could produce a monochome picture
of what was on the screen. After an experiment, I ran my program,
took screen prints of the graphs, made copies, and carried the results
around to the scientists and engineers. It was fun work, and it was
satisfying to have an important part to play.</p>
<p>The Modcomp II was just used for data acquisition–it had no way to
influence the world. The Data General Nova was (I believe–this was a
while ago) used for real-time control of certain parts of the
experiment. Like the Modcomp II, it was hooked up to various sensors,
but it also had outputs that it could use to drive things. If I
remembered what it could control, I probably couldn’t say, but again,
I’ve forgotten so those secrets are safe.</p>
<p>I enjoyed programming that Modcomp. By today’s standards, it was
minimal (64K x 16-bits of RAM), but the processor was pretty fast and
it had a lot of I/O bandwidth. The instruction set was friendly to
write assembly language in, and the OS and compilers were capable.
But what it wasn’t was sexy. It came in two 19-inch rack-mount
cabinets, full of huge wire-wrapped planes covered with ICs on one
side and a dense mat of wires on the other. The OS was reminiscent of
mainframes of old. There’s just nothing exciting about that.</p>
<p>The Data General Nova was, in my mind, sexier than the Modcomp II.
Its panel looked more modern to me. It was soldered instead of
wire-wrapped, which seemed better (it was more reliable, anyhow). Its
OS had a more modern feel to it, more like those of the microcomputers
that were bursting on the scene. I remember looking over at the guys
programming it and feeling a little bit jealous. Even though I
enjoyed the Modcomp, the Nova looked like more fun. It might have
been a case of “the grass is greener” syndrome.</p>
<p>I never did get a chance to program that Nova, but it turns out that
the <a href="https://github.com/simh/simh">simh</a> project can simulate it, <em>and</em> they have a licensed copy
of the RDOS operating system. Finally, I get a chance to play with
the Nova and learn what it’s about. In future blog entries, I will
write about my discoveries as I explore the simulated Data General
Nova. In the next blog entry, I’ll show how to get simh simulating
the Nova even if you know nothing about either. It’ll be a sort of
simh/nova “hello world.”</p>
Configuring SqlServer MAXDOP (Maximum Degree of Parallelism)2015-12-04T07:00:00+00:00http://wconrad.github.com/20151204/sqlserver-maxdop<p>Things I want to remember about SqlServer MAXDOP</p>
<h1 id="spaskbrent">sp_AskBrent</h1>
<p><a href="http://www.brentozar.com/askbrent/">sp_AskBrent</a> is the bees knees. Microsoft gives you plenty of
statistics–data–about how SqlServer is running. This tool turns the
data into information, telling you which things are more likely to be
problems that need to be fixed.</p>
<p>sp_askBrent may report that Wait Stats/CXPacket is high. If it does,
<a href="http://www.brentozar.com/sql/wait-stats/#CXPACKET">this page</a> explains what to do about it.</p>
<h1 id="how-to-adjust-maxdop">How to adjust MAXDOP</h1>
<p><a href="support.microsoft.com/en-us/kb/2806535">This page</a> explains what values you should set MAXDOP to.</p>
<p><a href="https://technet.microsoft.com/en-us/library/ms178144(v=sql.105).aspx">This page</a> has a query you can use to learn how many NUMA nodes
the server has. The query is:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>SELECT DISTINCT memory_node_id FROM sys.dm_os_memory_clerks
</code></pre>
</div>
<p><a href="http://shaunjstuart.com/archive/2012/07/changing-sql-servers-maxdop-setting/">This page</a> shows how to change the MAXDOP option. The commands
are:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>EXEC dbo.sp_configure 'show advanced options', 1;
GO
RECONFIGURE;
GO
</code></pre>
</div>
Agile flowchart2015-10-09T07:00:00+00:00http://wconrad.github.com/20151009/agile-flowchart<p>Some of my teammates weren’t understanding just how critical
communication is to an agile process, so I drew them this flowchart:</p>
<p><img src="http://wconrad.github.io/assets/agile_flowchart.png" alt="Agile Flowchart" /></p>
When git goes wrong--Removing files from a commit2015-09-01T07:00:00+00:00http://wconrad.github.com/20150901/when-git-goes-wrong-2<h1 id="i-added-too-many-files-to-my-commit">I added too many files to my commit</h1>
<p>Suppose you have committed some work:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git add -A .
<span class="gp">$ </span>git commit -m<span class="s1">'Fix recursion bug'</span></code></pre></figure>
<p>But later you look at the commit and discover you have committed
<em>sekrit</em>, a file that should not have been committed.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git log --name-status
commit 790af0ed7712c5e47949c1e267793bca54080aeb
Author: Wayne Conrad <wconrad@hsmove.com>
Date: Wed Sep 2 11:31:04 2015 -0700
Fix recursion bug
A somefile.rb
A sekrit</code></pre></figure>
<p><em>As long as this commit has not been pushed to origin</em>, you can remove
the file from that commit. I’ll show two different ways to remote the
file. Which way you pick depends upon whether you want to delete the
file from disk ask well as from the commit, or keep it on disk and
just remove it from the commit.</p>
<h1 id="to-delete-the-file-from-disk-and-from-the-commit">To delete the file from disk and from the commit</h1>
<p>If you want the file deleted from disk as well as from the commit,
just delete the file and then stage it:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>rm sekrit
<span class="gp">$ </span>git add sekrit</code></pre></figure>
<p>Or you could use this git shortcut for “delete a file and stage the
deletion”:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git rm sekrit</code></pre></figure>
<p>Either way, <code class="highlighter-rouge">git status</code> will show that the file is staged for
deletion:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git status
On branch master
Changes to be committed:
<span class="o">(</span>use <span class="s2">"git reset HEAD <file>..."</span> to unstage<span class="o">)</span>
deleted: sekrit</code></pre></figure>
<p>Now amend the previous comment:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git commit --amend</code></pre></figure>
<p>The commit will no longer have that file:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git log --name-status -1
commit 55da4e9b07b1337fb8161b9491f864845aad2abb
Author: Wayne Conrad <wconrad@hsmove.com>
Date: Wed Sep 2 11:31:04 2015 -0700
Fix recursion bug
A somefile.rb</code></pre></figure>
<p>And it is gone from disk:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>ls sekrit
ls: cannot access sekrit: No such file or directory</code></pre></figure>
<h1 id="to-delete-the-file-from-the-commit-but-not-from-disk">To delete the file from the commit but not from disk</h1>
<p>To remove that file from the commit but keep it on disk, we’ll tell
git to undo the previous commit, but leave all of that commit’s
changes on disk. But first, remember the sha1 of the commit:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git log -1 --oneline
78bcad9 Fix recursion bug</code></pre></figure>
<p>Having that sha1 will keep us from having to type the commit message
again. Now, undo the previous commit:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git reset HEAD~
<span class="gp">$ </span>git status
On branch master
Untracked files:
<span class="o">(</span>use <span class="s2">"git add <file>..."</span> to include <span class="k">in </span>what will be committed<span class="o">)</span>
sekrit
somefile.rb
nothing added to commit but untracked files present <span class="o">(</span>use <span class="s2">"git add"</span> to track<span class="o">)</span></code></pre></figure>
<p>Now stage the commit again, but this time only add the files that
should be in the commit:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git add somefile.rb</code></pre></figure>
<p>And finally, commit. The <code class="highlighter-rouge">-C</code> switch tells git to use the same commit
message, timestamp, and so on. It’s not necessary to use the -C
switch – you’d just have to type the commit message again.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git commit -C 78bcad9
<span class="o">[</span>master b36109c] Fix recursion bug
Date: Wed Sep 2 11:31:04 2015 -0700
1 file changed, 0 insertions<span class="o">(</span>+<span class="o">)</span>, 0 deletions<span class="o">(</span>-<span class="o">)</span>
create mode 100644 somefile.rb</code></pre></figure>
<p>The commit no longer has file <em>sekrit</em> in it:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git log --name-status -1
commit b36109cf76020f365050a33d374dc15b386050d0
Author: Wayne Conrad <wconrad@hsmove.com>
Date: Wed Sep 2 11:31:04 2015 -0700
Fix recursion bug
A somefile.rb</code></pre></figure>
<p>But the file is on disk:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git status
On branch master
Untracked files:
<span class="o">(</span>use <span class="s2">"git add <file>..."</span> to include <span class="k">in </span>what will be committed<span class="o">)</span>
sekrit
nothing added to commit but untracked files present <span class="o">(</span>use <span class="s2">"git add"</span> to track<span class="o">)</span></code></pre></figure>
<h1 id="how-to-keep-git-from-committing-that-file-ever-again">How to keep git from committing that file ever again</h1>
<p>If the file should never be committed to git, add it to the
<code class="highlighter-rouge">.gitignore</code> file:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span><span class="nb">echo</span> <span class="s1">'sekrit'</span> >>.gitignore
<span class="gp">$ </span>git add .gitignore
<span class="gp">$ </span>git commit -m<span class="s1">'Ignore sekrit'</span>
<span class="o">[</span>master 15336c9] Ignore sekrit
1 file changed, 1 insertion<span class="o">(</span>+<span class="o">)</span></code></pre></figure>
<p>Git will no longer consider the file untracked:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git status
On branch master
nothing to commit, working directory clean</code></pre></figure>
When git goes wrong--Removing a file from a commit2015-08-30T08:00:00+00:00http://wconrad.github.com/20150830/when-git-goes-wrong-2<p><a href="/20150828/when-git-goes-wrong-1.html">When Git Goes Wrong</a>, continued.</p>
<h1 id="i-added-too-many-files-to-a-commit">I added too many files to a commit</h1>
<p>Suppose you have made changes to a few files:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git status
On branch master
Changes not staged <span class="k">for </span>commit:
<span class="o">(</span>use <span class="s2">"git add <file>..."</span> to update what will be committed<span class="o">)</span>
<span class="o">(</span>use <span class="s2">"git checkout -- <file>..."</span> to discard changes <span class="k">in </span>working directory<span class="o">)</span>
modified: bar
modified: foo
no changes added to commit <span class="o">(</span>use <span class="s2">"git add"</span> and/or <span class="s2">"git commit -a"</span><span class="o">)</span></code></pre></figure>
<p>And you create a commit with one of the files, accidentally leaving
the other file out:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git add bar
<span class="gp">$ </span>git commit -m<span class="s1">'Fixed issue #1'</span>
<span class="o">[</span>master 12bf5d9] Fixed issue <span class="c">#1</span>
1 file changed, 1 insertion<span class="o">(</span>+<span class="o">)</span></code></pre></figure>
<p>Then, when doing a <code class="highlighter-rouge">git status</code>, you notice that a file was not
committed:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git status
On branch master
Changes not staged <span class="k">for </span>commit:
<span class="o">(</span>use <span class="s2">"git add <file>..."</span> to update what will be committed<span class="o">)</span>
<span class="o">(</span>use <span class="s2">"git checkout -- <file>..."</span> to discard changes <span class="k">in </span>working directory<span class="o">)</span>
modified: foo
no changes added to commit <span class="o">(</span>use <span class="s2">"git add"</span> and/or <span class="s2">"git commit -a"</span><span class="o">)</span></code></pre></figure>
<p>You confirm that by using <code class="highlighter-rouge">git log</code> to look at the most recent commit:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git log --name-status -1
commit 12bf5d90ac19826320efabb20972e48ce4207844
Author: Wayne Conrad <wconrad@hsmove.com>
Date: Fri Aug 28 11:36:15 2015 -0700
Fixed issue <span class="c">#1</span>
M bar</code></pre></figure>
<h1 id="the-wrong-way-to-fix-it">The wrong way to fix it</h1>
<p>You could just make another commit with the missing files:</p>
<div class="highlighter-rouge"><pre class="highlight"><code># Please don't do this
$ git add bar
$ git commit -m'Adding missing file from fix for issue #1'
</code></pre>
</div>
<p>And, if you have already pushed the previous commit to origin, this is
what you’ll have to do. But when you have a choice, please don’t do
this. You want the commit’s <em>idea</em> to be represented in a single
commit, not in two. Also, it’s unlikely that the first of the two
commits would have passing tests.</p>
<h1 id="so-lets-fix-it">So let’s fix it</h1>
<p>To the file you left out to the commit, you can first stage the file
using <code class="highlighter-rouge">git add</code>:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git add foo</code></pre></figure>
<p>and then add the staged files to the <em>previous</em> commit by using the
<code class="highlighter-rouge">git commit</code> with the <code class="highlighter-rouge">--amend</code> switch:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git add foo
<span class="gp">$ </span>git commit --amend</code></pre></figure>
<p>This will bring up an editor window which you can use to edit the
commit message, if desired:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>Fixed issue #1
# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
#
# Date: Fri Aug 28 11:36:15 2015 -0700
#
# On branch master
# Changes to be committed:
# modified: bar
# modified: foo
#
</code></pre>
</div>
<p>After you save the file, git will ammend the commit, adding the file
that you accidentally left out.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git log -1 --name-status
commit 75b7e96d876e5fb4909469d26019d766b7189ae2
Author: Wayne Conrad <wconrad@hsmove.com>
Date: Fri Aug 28 11:36:15 2015 -0700
Fixed issue <span class="c">#1</span>
M bar
M foo</code></pre></figure>
<p>Git means never having to say you’re sorry.</p>
When git goes wrong--Adding files to a commit2015-08-28T19:00:00+00:00http://wconrad.github.com/20150828/when-git-goes-wrong-1<p><a href="https://git-scm.com/">git</a> is more than a backup mechanism. It’s a way to communicate
with other programmers and with your future self. In order for this
communication to be clear, it’s important to keep your git commits
clean. A commit is cleanest when it does one thing, has passing
tests, and has a good description.</p>
<p>We’re going to explore how to create good git commits when you start
out with bad ones. It’s pretty common to accidentally commit the
wrong thing, or forget to commit a file, but git makes it easy to fix
a previous commit. But first, this important warning:</p>
<p><strong>IMPORTANT WARNING</strong></p>
<p>Once you have pushed a commit to a remote server that other
programmers can pull from, you shouldn’t change it. You can change
it–it’s not difficult, but you can really mess up life for those
other programmers if you don’t know what you’re doing. Until then,
don’t attempt to rewrite a commit that’s already been pushed to a
remote server.</p>
<p>However, <em>before</em> you’ve pushed any commits upstream to origin, you
can change them around all you want. Let’s learn how.</p>
<h1 id="i-forgot-to-add-some-files-to-my-commit">I forgot to add some files to my commit</h1>
<p>Suppose you have made changes to a few files:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git status
On branch master
Changes not staged <span class="k">for </span>commit:
<span class="o">(</span>use <span class="s2">"git add <file>..."</span> to update what will be committed<span class="o">)</span>
<span class="o">(</span>use <span class="s2">"git checkout -- <file>..."</span> to discard changes <span class="k">in </span>working directory<span class="o">)</span>
modified: bar
modified: foo
no changes added to commit <span class="o">(</span>use <span class="s2">"git add"</span> and/or <span class="s2">"git commit -a"</span><span class="o">)</span></code></pre></figure>
<p>And you create a commit with one of the files, accidentally leaving
the other file out:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git add bar
<span class="gp">$ </span>git commit -m<span class="s1">'Fixed issue #1'</span>
<span class="o">[</span>master 12bf5d9] Fixed issue <span class="c">#1</span>
1 file changed, 1 insertion<span class="o">(</span>+<span class="o">)</span></code></pre></figure>
<p>Then, when doing a <code class="highlighter-rouge">git status</code>, you notice that a file was not
committed:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git status
On branch master
Changes not staged <span class="k">for </span>commit:
<span class="o">(</span>use <span class="s2">"git add <file>..."</span> to update what will be committed<span class="o">)</span>
<span class="o">(</span>use <span class="s2">"git checkout -- <file>..."</span> to discard changes <span class="k">in </span>working directory<span class="o">)</span>
modified: foo
no changes added to commit <span class="o">(</span>use <span class="s2">"git add"</span> and/or <span class="s2">"git commit -a"</span><span class="o">)</span></code></pre></figure>
<p>You confirm that by using <code class="highlighter-rouge">git log</code> to look at the most recent commit:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git log --name-status -1
commit 12bf5d90ac19826320efabb20972e48ce4207844
Author: Wayne Conrad <wconrad@hsmove.com>
Date: Fri Aug 28 11:36:15 2015 -0700
Fixed issue <span class="c">#1</span>
M bar</code></pre></figure>
<h1 id="the-wrong-way-to-fix-it">The wrong way to fix it</h1>
<p>You could just make another commit with the missing files:</p>
<div class="highlighter-rouge"><pre class="highlight"><code># Please don't do this
$ git add bar
$ git commit -m'Adding missing file from fix for issue #1'
</code></pre>
</div>
<p>And, if you have already pushed the previous commit to origin, this is
what you’ll have to do. But when you have a choice, please don’t do
this. You want the commit’s <em>idea</em> to be represented in a single
commit, not in two. Also, it’s unlikely that the first of the two
commits would have passing tests.</p>
<h1 id="so-lets-fix-it">So let’s fix it</h1>
<p>To the file you left out to the commit, you can first stage the file
using <code class="highlighter-rouge">git add</code>:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git add foo</code></pre></figure>
<p>and then add the staged files to the <em>previous</em> commit by using the
<code class="highlighter-rouge">git commit</code> with the <code class="highlighter-rouge">--amend</code> switch:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git add foo
<span class="gp">$ </span>git commit --amend</code></pre></figure>
<p>This will bring up an editor window which you can use to edit the
commit message, if desired:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>Fixed issue #1
# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
#
# Date: Fri Aug 28 11:36:15 2015 -0700
#
# On branch master
# Changes to be committed:
# modified: bar
# modified: foo
#
</code></pre>
</div>
<p>After you save the file, git will ammend the commit, adding the file
that you accidentally left out.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git log -1 --name-status
commit 75b7e96d876e5fb4909469d26019d766b7189ae2
Author: Wayne Conrad <wconrad@hsmove.com>
Date: Fri Aug 28 11:36:15 2015 -0700
Fixed issue <span class="c">#1</span>
M bar
M foo</code></pre></figure>
<p>Git means never having to say you’re sorry.</p>
A trick to make backup2l ignore certain directories2015-08-28T07:00:00+00:00http://wconrad.github.com/20150828/backup2l-ignore-directories<p>My main drive’s RAID configuration protects me against drive failures,
but it doesn’t protect me against accidentally deleting a file. For
that, I use the <a href="http://backup2l.sourceforge.net/">backup2l</a> program to periodically backup my files
to a third hard drive.</p>
<p>backup2l has a fairly easy way to ignore a file or directory: If the
name of its path includes “.nobackup” anywhere in it, then that file
or directory won’t be backed up. However, it’s ugly to have to look
at “.nobackup” in directory names. Also, some directories that you
don’t want to back up can’t be renamed. For example, your virtual
machine manager may keep virtual images, which are very large, in a
directory that it expects to have a certain name. You can’t just
rename the directory to end in “.nobackup”, or the virtual machine
manager won’t be able to find the images.</p>
<p>The workaround is simple. Go ahead and rename the directory to
include “.nobackup”. Also, start it with a “.” so it will be hidden:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>mv my_big_directory .my_big_directory.nobackup
</code></pre>
</div>
<p>Now create a symlink to the directory so it can be referenced with its
original name:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>ln -s .my_big_directory.nobackup my_big_directory
</code></pre>
</div>
<p>You will see the symlink when you use “ls -l”:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>lrwxrwxrwx 1 wayne wayne 24 Aug 28 10:02 my_big_directory -> .my_big_directory.nobackup
</code></pre>
</div>
<p>but otherwise the directory will function the same as it did
before–except that it won’t be backed up by backup2l.</p>
A few tips for using bunny library with RabbitMQ2015-08-05T19:00:00+00:00http://wconrad.github.com/20150805/bunny-tricks<p>Some things I want to remember about using the bunny library to
publish and read messages from a RabbitMQ server.</p>
<h1 id="no-blocking-read">No blocking read</h1>
<p>The bunny library does not have a blocking read, so you’ll have to
synthesize it yourself:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="c1"># Returns delivery_info, properties, body</span>
<span class="k">def</span> <span class="nf">synchronous_get</span>
<span class="kp">loop</span> <span class="k">do</span>
<span class="n">a</span> <span class="o">=</span> <span class="vi">@channel</span><span class="p">.</span><span class="nf">basic_get</span><span class="p">(.</span><span class="nf">.</span><span class="o">.</span><span class="p">)</span>
<span class="k">return</span> <span class="n">a</span> <span class="k">unless</span> <span class="n">a</span><span class="p">.</span><span class="nf">first</span><span class="p">.</span><span class="nf">nil?</span>
<span class="nb">sleep</span><span class="p">(</span><span class="mi">4</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<h1 id="publishing-messages-reliably">Publishing messages reliably</h1>
<p>If you need to be sure that a message was accepted, you’ll have to
jump through some hoops:</p>
<p>Set the channel into “confirm select” mode:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="vi">@channel</span><span class="p">.</span><span class="nf">confirm_select</span></code></pre></figure>
<p>Register a callback that will be
called if the message is returned:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="vi">@exchange</span><span class="p">.</span><span class="nf">on_return</span> <span class="k">do</span>
<span class="vi">@returned</span> <span class="o">=</span> <span class="kp">true</span>
<span class="k">end</span></code></pre></figure>
<p>Now, to publish a message:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="vi">@returned</span> <span class="o">=</span> <span class="kp">false</span>
<span class="vi">@exchange</span><span class="p">.</span><span class="nf">publish</span><span class="p">(.</span><span class="nf">.</span><span class="p">.</span><span class="nf">,</span> <span class="ss">persistent: </span><span class="kp">true</span><span class="p">)</span>
<span class="vi">@channel</span><span class="p">.</span><span class="nf">wait_for_confirms</span>
<span class="k">if</span> <span class="vi">@returned</span>
<span class="k">raise</span> <span class="no">RoutingError</span><span class="p">,</span> <span class="s2">"Message could not be routed"</span>
<span class="k">end</span></code></pre></figure>
Keep String#% in mind2015-07-23T07:00:00+00:00http://wconrad.github.com/20150723/keep-sprintf-in-mind<p>When a Ruby program needs to embed the value of an expression in a
string, <a href="https://en.wikibooks.org/wiki/Ruby_Programming/Syntax/Literals#Interpolation">string interpolation</a> is the go-to tool:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="nb">name</span> <span class="o">=</span> <span class="s2">"Fred"</span>
<span class="nb">puts</span> <span class="s2">"Hello, </span><span class="si">#{</span><span class="nb">name</span><span class="si">}</span><span class="s2">"</span>
<span class="c1">#=> "Hello, Fred"</span></code></pre></figure>
<p>But sometimes the string can get a little long:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="nb">puts</span> <span class="s2">"</span><span class="si">#{</span><span class="n">blamee</span><span class="p">(</span><span class="n">longest_line</span><span class="p">.</span><span class="nf">location</span><span class="p">)</span><span class="si">}</span><span class="s2">: </span><span class="si">#{</span><span class="n">longest_line</span><span class="p">.</span><span class="nf">location</span><span class="si">}</span><span class="s2"> is </span><span class="si">#{</span><span class="n">longest_line</span><span class="p">.</span><span class="nf">length</span><span class="si">}</span><span class="s2"> bytes long"</span></code></pre></figure>
<p>This can be fixed by continuing the string on another line:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="nb">puts</span> <span class="s2">"</span><span class="si">#{</span><span class="n">blamee</span><span class="p">(</span><span class="n">longest_line</span><span class="p">.</span><span class="nf">location</span><span class="p">)</span><span class="si">}</span><span class="s2">: </span><span class="si">#{</span><span class="n">longest_line</span><span class="p">.</span><span class="nf">location</span><span class="si">}</span><span class="s2"> "</span><span class="p">\</span>
<span class="s2">"is </span><span class="si">#{</span><span class="n">longest_line</span><span class="p">.</span><span class="nf">length</span><span class="si">}</span><span class="s2"> bytes long"</span></code></pre></figure>
<p>But that’s ugly. Much prettier is to use the <a href="http://ruby-doc.org/core-2.2.2/String.html#method-i-25">String#%</a> method:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="nb">puts</span> <span class="s1">'%s: %s is %d bytes long'</span> <span class="o">%</span> <span class="p">[</span>
<span class="n">blamee</span><span class="p">(</span><span class="n">longest_line</span><span class="p">.</span><span class="nf">location</span><span class="p">),</span>
<span class="n">longest_line</span><span class="p">.</span><span class="nf">location</span><span class="p">,</span>
<span class="n">longest_line</span><span class="p">.</span><span class="nf">length</span><span class="p">,</span>
<span class="p">]</span></code></pre></figure>
<p>This takes care of the long line. It also puts each expression on a
line by itself, making it easier to read and edit.</p>
<p>String#% is a wrapper for <a href="http://ruby-doc.org/core-2.2.2/Kernel.html#method-i-sprintf">Kernel#sprintf</a>, which has many useful
ways to format things. This print statement left-justifies the
version string with a width of 9, and right-justifies the use count
with a width of 7:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="nb">puts</span> <span class="s2">"version %-9s used %7d times from %s through %s"</span> <span class="o">%</span> <span class="p">[</span>
<span class="n">api_version</span><span class="p">.</span><span class="nf">version</span><span class="p">,</span>
<span class="n">api_version</span><span class="p">.</span><span class="nf">use_count</span><span class="p">,</span>
<span class="n">format_time</span><span class="p">(</span><span class="n">api_version</span><span class="p">.</span><span class="nf">min_time</span><span class="p">),</span>
<span class="n">format_time</span><span class="p">(</span><span class="n">api_version</span><span class="p">.</span><span class="nf">max_time</span><span class="p">),</span>
<span class="p">]</span>
<span class="c1"># => version v3.1.0 used 2453 times from 2015-07-20 through 2015-07-22</span>
<span class="c1"># => version v3.0.0 used 138893 times from 2015-07-19 through 2015-07-23</span></code></pre></figure>
<p>And it does so while looking good.</p>
<p>So the next time you’re interpolating and the string gets too long or
hard to read, consider the virtues of String#%.</p>
Tiny DSL using throw/catch2015-07-14T19:00:00+00:00http://wconrad.github.com/20150714/tiny-dsl-using-throw-catch<p>Someone asked me whether I had used Ruby’s <a href="http://rubylearning.com/blog/2011/07/12/throw-catch-raise-rescue--im-so-confused/">throw/catch</a> mechanism
in production. I thought I had, but couldn’t remember a specific
instance. Today, I ran across some of my production code that uses
throw/catch.</p>
<p>The product has a form that the end user fills out in order to pay a
bill. For the user’s convenience, the form is prefilled from any of
several sources. For example, if the user has made a payment before,
we’ll use the address they filled in last time. If they have not made
a payment but the system knows the address that their statement was
mailed to, it will use that. There is also a test mode that can
provide prefills, and defaults to use when no prefill value is found.</p>
<p>The class that knows the prefill rules has a constructor that takes
the different objects that prefill information can come from:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">BillPayPrefill</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">document</span><span class="p">,</span> <span class="n">demo</span><span class="p">,</span> <span class="n">merchant</span><span class="p">,</span> <span class="n">bill_pay_request</span><span class="p">)</span>
<span class="vi">@document</span> <span class="o">=</span> <span class="n">document</span>
<span class="vi">@demo</span> <span class="o">=</span> <span class="n">demo</span>
<span class="vi">@merchant</span> <span class="o">=</span> <span class="n">merchant</span>
<span class="vi">@bill_pay_request</span> <span class="o">=</span> <span class="n">bill_pay_request</span>
<span class="k">end</span>
<span class="p">.</span><span class="nf">.</span><span class="p">.</span>
<span class="nf">end</span></code></pre></figure>
<p>There are about 20 fields, and so 20 methods to encapsulate the
business rules for how prefill works with each field. At first, these
methods looked like this:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">first_name</span>
<span class="k">if</span> <span class="vi">@bill_pay_request</span>
<span class="vi">@bill_pay_request</span><span class="p">.</span><span class="nf">billing_first_name</span>
<span class="k">elsif</span> <span class="vi">@document</span>
<span class="vi">@document</span><span class="p">.</span><span class="nf">first_name</span>
<span class="k">elsif</span> <span class="n">test_mode?</span>
<span class="s1">'BARNEY'</span>
<span class="k">else</span>
<span class="kp">nil</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>Just one or two methods like that is no big deal. But with 20 of
them, we wanted a way to make the business rules stand out better. We
ended up with this instead:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">first_name</span>
<span class="n">get_value</span> <span class="k">do</span>
<span class="n">from_request</span> <span class="ss">:billing_first_name</span>
<span class="n">from_document</span> <span class="ss">:first_name</span>
<span class="n">when_test_mode</span> <span class="s1">'BARNEY'</span>
<span class="n">default</span> <span class="kp">nil</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>The DSL that makes this work is implemented in just a few private
methods. This is the first of them. All it does is to catch a symbol
and then yield to the passed block:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">get_value</span>
<span class="kp">catch</span><span class="p">(</span><span class="ss">:value</span><span class="p">)</span> <span class="k">do</span>
<span class="k">yield</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>The other methods throw a value, if it exists. If the value does not
exist, they just return so that the next method can be tried:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">from_request</span><span class="p">(</span><span class="n">request_attribute</span><span class="p">)</span>
<span class="k">if</span> <span class="vi">@bill_pay_request</span>
<span class="kp">throw</span> <span class="ss">:value</span><span class="p">,</span> <span class="vi">@bill_pay_request</span><span class="p">.</span><span class="nf">send</span><span class="p">(</span><span class="n">request_attribute</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">from_document</span><span class="p">(</span><span class="n">document_attribute</span><span class="p">)</span>
<span class="k">if</span> <span class="vi">@document</span>
<span class="kp">throw</span> <span class="ss">:value</span><span class="p">,</span> <span class="vi">@document</span><span class="p">.</span><span class="nf">send</span><span class="p">(</span><span class="n">document_attribute</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">when_test_mode</span><span class="p">(</span><span class="n">test_value</span><span class="p">)</span>
<span class="k">if</span> <span class="n">test_mode?</span>
<span class="kp">throw</span> <span class="ss">:value</span><span class="p">,</span> <span class="n">test_value</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">default</span><span class="p">(</span><span class="n">default_value</span><span class="p">)</span>
<span class="kp">throw</span> <span class="ss">:value</span><span class="p">,</span> <span class="n">default_value</span>
<span class="k">end</span></code></pre></figure>
<p>The result of the top-level <code class="highlighter-rouge">catch</code> is the result of the first <code class="highlighter-rouge">throw</code>
that gets executed. Using throw/catch gives the DSL a nice way to
stop looking when the prefill value is found.</p>
Avoid Tabular Alignment in Code2015-06-27T23:00:00+00:00http://wconrad.github.com/20150627/tabular-alignment-antipattern<p>Sometimes you’ll see (or write) code like this:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="n">let</span><span class="p">(</span><span class="ss">:resource</span><span class="p">)</span> <span class="p">{</span> <span class="no">FactoryGirl</span><span class="p">.</span><span class="nf">create</span> <span class="ss">:device</span> <span class="p">}</span>
<span class="n">let</span><span class="p">(</span><span class="ss">:type</span><span class="p">)</span> <span class="p">{</span> <span class="no">Type</span><span class="p">.</span><span class="nf">find</span> <span class="n">resource</span><span class="p">.</span><span class="nf">type_id</span> <span class="p">}</span></code></pre></figure>
<p>or this:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="vi">@cached_metadata</span> <span class="o">=</span> <span class="n">options</span><span class="p">[</span><span class="ss">:metadata</span><span class="p">]</span>
<span class="vi">@logger</span> <span class="o">=</span> <span class="n">options</span><span class="p">[</span><span class="ss">:logger</span><span class="p">]</span></code></pre></figure>
<p>This show what I call “Tabular Alignment,” the insertion of horizontal
white space to make certain elements of the code line up. It makes
the code nicer to read, but it is costly to maintain.</p>
<h1 id="makes-editing-a-chore">Makes Editing a Chore</h1>
<p>Editing code that is tabular aligned can be a chore. Suppose we are
starting with this:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="vi">@cached_metadata</span> <span class="o">=</span> <span class="n">options</span><span class="p">[</span><span class="ss">:metadata</span><span class="p">]</span>
<span class="vi">@logger</span> <span class="o">=</span> <span class="n">options</span><span class="p">[</span><span class="ss">:logger</span><span class="p">]</span></code></pre></figure>
<p>and need to add this:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="vi">@cached_capabilities</span> <span class="o">=</span> <span class="n">options</span><span class="p">[</span><span class="ss">:capabilities</span><span class="p">]</span></code></pre></figure>
<p>Since the width of the left column has changed, we need to edit every
line in the block in order to maintain the tabular alignment:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="vi">@cached_metadata</span> <span class="o">=</span> <span class="n">options</span><span class="p">[</span><span class="ss">:metadata</span><span class="p">]</span> <span class="c1"># edited</span>
<span class="vi">@logger</span> <span class="o">=</span> <span class="n">options</span><span class="p">[</span><span class="ss">:logger</span><span class="p">]</span> <span class="c1"># edited</span>
<span class="vi">@cached_capabilities</span> <span class="o">=</span> <span class="n">options</span><span class="p">[</span><span class="ss">:capabilities</span><span class="p">]</span> <span class="c1"># added</span></code></pre></figure>
<p>Who wants to work that hard?</p>
<h1 id="is-destroyed-by-searchreplace">Is Destroyed by Search/Replace</h1>
<p>Search/Replace destroys tabular alignment. Given this:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="vi">@cached_metadata</span> <span class="o">=</span> <span class="n">options</span><span class="p">[</span><span class="ss">:metadata</span><span class="p">]</span>
<span class="vi">@logger</span> <span class="o">=</span> <span class="n">options</span><span class="p">[</span><span class="ss">:logger</span><span class="p">]</span></code></pre></figure>
<p>Suppose we decide to rename <code class="highlighter-rouge">@cached_metadata</code> to <code class="highlighter-rouge">@metadata</code>, so we
use a search/replace operation to do it. Now, we get this:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="vi">@metadata</span> <span class="o">=</span> <span class="n">options</span><span class="p">[</span><span class="ss">:metadata</span><span class="p">]</span>
<span class="vi">@logger</span> <span class="o">=</span> <span class="n">options</span><span class="p">[</span><span class="ss">:logger</span><span class="p">]</span></code></pre></figure>
<p>In order to preserve tabular alignment, we need to examine every line
of code where a replacement was done to ensure that tabular alignment
is maintained.</p>
<p>If the search/replace takes place in a mode that doesn’t have you
review every replacement, you won’t even know that you’ve messed up
some alignment. The usual result is that misaligned code.</p>
<p>Now that’s not pretty, is it?</p>
<h1 id="makes-a-mess-of-version-control-history">Makes a Mess of Version Control History</h1>
<p>When a block of tabularly alined code needs to be realigned, the
version control system will treat each of those lines as having been
modified:</p>
<figure class="highlight"><pre><code class="language-diff" data-lang="diff"><span class="gh">diff --git a/foo.rb b/foo.rb
index 40f7833..694d8fe 100644
</span><span class="gd">--- a/foo.rb
</span><span class="gi">+++ b/foo.rb
</span><span class="gu">@@ -1,8 +1,8 @@
</span> class Foo
def initialize(options)
<span class="gd">- @cached_metadata = options[:metadata]
- @logger = options[:logger]
</span><span class="gi">+ @metadata = options[:metadata]
+ @logger = options[:logger]
</span> end
end</code></pre></figure>
<p>Now suppose that in the mean time, in another branch, a programmer has
added a new variable:</p>
<figure class="highlight"><pre><code class="language-diff" data-lang="diff"><span class="gh">diff --git a/foo.rb b/foo.rb
index 40f7833..86648cb 100644
</span><span class="gd">--- a/foo.rb
</span><span class="gi">+++ b/foo.rb
</span><span class="gu">@@ -3,6 +3,7 @@ class Foo
</span> def initialize(options)
@cached_metadata = options[:metadata]
@logger = options[:logger]
<span class="gi">+ @kittens = options[:kittens]
</span> end
end</code></pre></figure>
<p>Merging that branch will fail:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>wayne@mercury:/tmp/foo$ git merge add_kittens
Auto-merging foo.rb
CONFLICT (content): Merge conflict in foo.rb
Automatic merge failed; fix conflicts and then commit the result.
</code></pre>
</div>
<p>Without tabular alignment, this merge would have succeeded
automatically.</p>
<p>Why make more work for ourselves?</p>
More ConcatenatingEnumerator2015-03-10T21:45:00+00:00http://wconrad.github.com/20150310/more-concatenating-enumerator<p>In a previous article I introduced a <a href="/20140603/concatenating-enumerator.html">Concatenating Enumerator</a>.
Today I found a use for it in production code. It could be that I
just <a href="http://en.wiktionary.org/wiki/if_all_you_have_is_a_hammer,_everything_looks_like_a_nail">had a hammer and was looking for a nail</a>, but I think the
code came out really clean.</p>
<h1 id="concatenatingenumerator">ConcatenatingEnumerator</h1>
<p>Here’s the concatenating enumerator. It lets you glue any number of
enumerable things together and treat them as a single enumerator:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">ConcatenatingEnumerator</span> <span class="o"><</span> <span class="no">Enumerator</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">enumerators</span> <span class="o">=</span> <span class="p">[])</span>
<span class="k">super</span><span class="p">()</span> <span class="k">do</span> <span class="o">|</span><span class="n">yielder</span><span class="o">|</span>
<span class="n">enumerators</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">enumerator</span><span class="o">|</span>
<span class="n">enumerator</span><span class="p">.</span><span class="nf">to_enum</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">item</span><span class="o">|</span>
<span class="n">yielder</span><span class="p">.</span><span class="nf">yield</span> <span class="n">item</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>It can be used like this:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="n">enum1</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span>
<span class="n">enum2</span> <span class="o">=</span> <span class="p">[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">]</span>
<span class="n">enum</span> <span class="o">=</span> <span class="no">ConcatenatingEnumerator</span><span class="p">.</span><span class="nf">new</span><span class="p">([</span><span class="n">enum1</span><span class="p">,</span> <span class="n">enum2</span><span class="p">])</span>
<span class="nb">p</span> <span class="n">enum</span><span class="p">.</span><span class="nf">to_a</span> <span class="c1">#=> [1, 2, 3, 4, 5]</span></code></pre></figure>
<p>This example uses #to_a, but any of the usual enumeration methods will
work on a ConcatenatingEnumerator: #each, #first, #map, and so on.</p>
<p>Since ConcatenatingEnumerator calls <code class="highlighter-rouge">#to_enum</code> on its arguments, it
will take either enumerators, or anything that can be treated as an
enumerator (like Array). (Calling #to_enum is a change from the
previous version of ConctatenatingEnumerator).</p>
<h1 id="using-concatenatingenumerator-to-read-log-files">Using ConcatenatingEnumerator to read log files</h1>
<h2 id="the-main-method">The main method</h2>
<p>The application parses one or more log files and prints some
statistics. Here’s the program’s main method:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="n">logs</span> <span class="o">=</span> <span class="no">Logs</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="vi">@args</span><span class="p">.</span><span class="nf">paths</span><span class="p">)</span>
<span class="n">stats</span> <span class="o">=</span> <span class="no">Stats</span><span class="p">.</span><span class="nf">new</span>
<span class="n">parser</span> <span class="o">=</span> <span class="no">Parser</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">stats</span><span class="p">)</span>
<span class="n">logs</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">line</span><span class="o">|</span>
<span class="n">parser</span><span class="p">.</span><span class="nf">parse</span><span class="p">(</span><span class="n">line</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">StatsPrinter</span><span class="p">.</span><span class="nf">print</span><span class="p">(</span><span class="n">stats</span><span class="p">)</span></code></pre></figure>
<p>Even though this program reads multiple log files, the main method
isn’t concerned with that. The <code class="highlighter-rouge">Logs</code> class treats all the log files
as a single enumeration. This makes the main method’s life easy, and
its logic is easy to follow. Only the highest level of abstraction is
visible here.</p>
<h2 id="logs">Logs</h2>
<p><code class="highlighter-rouge">Logs</code> is perfectly simple:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">Logs</span> <span class="o"><</span> <span class="no">ConcatenatingEnumerator</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">paths</span><span class="p">)</span>
<span class="n">logs</span> <span class="o">=</span> <span class="n">paths</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">path</span><span class="o">|</span> <span class="no">Log</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">path</span><span class="p">)</span> <span class="p">}</span>
<span class="k">super</span><span class="p">(</span><span class="n">logs</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>We turn the log paths into instances of <code class="highlighter-rouge">Log</code>. A <code class="highlighter-rouge">Log</code> is enumerable,
so ConcatenatingEnumerator can glue them together into a single
enumeration.</p>
<h2 id="log">Log</h2>
<p>A <code class="highlighter-rouge">Log</code> is more interesting, because we want to open each file as
needed, and close it as soon as possible. That way, the program only
needs to have one file open at a time.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">Log</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">path</span><span class="p">)</span>
<span class="vi">@path</span> <span class="o">=</span> <span class="n">path</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">to_enum</span>
<span class="no">Enumerator</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span> <span class="o">|</span><span class="n">yielder</span><span class="o">|</span>
<span class="no">File</span><span class="p">.</span><span class="nf">open</span><span class="p">(</span><span class="vi">@path</span><span class="p">,</span> <span class="s2">"r"</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">file</span><span class="o">|</span>
<span class="n">file</span><span class="p">.</span><span class="nf">each_line</span> <span class="k">do</span> <span class="o">|</span><span class="n">line</span><span class="o">|</span>
<span class="n">yielder</span><span class="p">.</span><span class="nf">yield</span><span class="p">(</span><span class="n">line</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p><code class="highlighter-rouge">#to_enum</code> is what lets the ConcatenatingEnumerator use this object.
The method returns an enumerator. When that enumerator is used (and
not before), the file is opened, and each line is yielded in turn.
Once all of the lines are yielded, the file is closed.</p>
<h1 id="the-real-program-does-more">The real program does more</h1>
<p>This code teeters on the edge of too much abstraction. There’s a lot
of mechanism being used to simply read some log files one after the
other. The main method could be more like this, eliminating the
<code class="highlighter-rouge">Logs</code> class and the use of <code class="highlighter-rouge">ConcatenatingEnumerator</code>:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="n">stats</span> <span class="o">=</span> <span class="no">Stats</span><span class="p">.</span><span class="nf">new</span>
<span class="n">parser</span> <span class="o">=</span> <span class="no">Parser</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">stats</span><span class="p">)</span>
<span class="vi">@args</span><span class="p">.</span><span class="nf">paths</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">path</span><span class="o">|</span>
<span class="no">Log</span><span class="p">.</span><span class="nf">open</span><span class="p">(</span><span class="n">path</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">log</span><span class="o">|</span>
<span class="n">log</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">line</span><span class="o">|</span>
<span class="n">parser</span><span class="p">.</span><span class="nf">parse</span><span class="p">(</span><span class="n">line</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">StatsPrinter</span><span class="p">.</span><span class="nf">print</span><span class="p">(</span><span class="n">stats</span><span class="p">)</span></code></pre></figure>
<p>But I prefer the more minimal main method that ConcatenatingEnumerator
makes possible. You may reasonably disagree.</p>
Debugging ActiveRecord Issues2015-02-09T04:16:00+00:00http://wconrad.github.com/20150209/debugging-activerecord-issues<p>Things I learned when exploring <a href="https://github.com/composite-primary-keys/composite_primary_keys/issues/287">an apparent issue with the
<code class="highlighter-rouge">composite_primary_keys</code> gem</a>:</p>
<ul>
<li>
<p>The <a href="https://github.com/rroblak/seed_dump">seed_dump</a> gem is a great way to capture the contents of a
production database so that you can debug locally.</p>
</li>
<li>
<p>Similarly, <code class="highlighter-rouge">rake db:dump</code> is good for capturing the schema of a
production database.</p>
</li>
<li>
<p><code class="highlighter-rouge">rails g task</code> generates a skeleton rake task which has a database
connection and access to the project’s models.</p>
</li>
<li>
<p>The <a href="https://github.com/rails/rails">rails</a> project has great <a href="https://github.com/rails/rails/tree/master/guides/bug_report_templates">bug report templates</a> for
creating stand-alone programs that reproduce an active-record issue.</p>
</li>
</ul>
Why to check in a library's Gemfile.lock2015-01-23T17:30:00+00:00http://wconrad.github.com/20150123/why-to-check-in-gemfile-dot-lock<p>It is common advice to not check your Gemfile.lock into version
control for a Ruby gem that is a library. I think that advice is
wrong, at least for the way I work with gem libraries.</p>
<h1 id="the-advice-and-why-i-disagree-with-it">The advice, and why I disagree with it</h1>
<p>Here’s the reason as <a href="http://yehudakatz.com/2010/12/16/clarifying-the-roles-of-the-gemspec-and-gemfile/">stated by Yehuda Katz</a> (<em>emphasis mine</em>):</p>
<blockquote>
<p>“This is also why we recommend that people do not check in their
Gemfile.lock files in gem development. That file is emitted by
bundler, and guarantees that all gems, including dependencies of
dependencies, remain the same. However, when doing gem development,
<em>you want to know immediately when some change in the overall
ecosystem breaks your setup</em>. While you may want to insist on using
a particular gem from its git location, you do not want to hardcode
your development to a very specific set of gems, only to find out
later that a gem released after you ran bundle install, but
compatible with your version range, doesn’t work with your code.”</p>
</blockquote>
<p>Whether or not I check in the Gemfile.lock, <em>it exists on my drive</em>,
left over from the last time I did a “bundle install” in that project.
Because of that, I will <em>never</em> know immediately that some later
version of a dependency has broken my gem. I only get to find out
after I do a <code class="highlighter-rouge">bundle update</code>.</p>
<p>Not checking in the Gemfile.lock doesn’t change that.</p>
<h1 id="why-using-a-particular-gem-from-its-git-location-does-not-matter">Why “using a particular gem from its git location” does not matter.</h1>
<p>One of the arguments against checking in Gemfile.lock is that it will
capture “diversions” made in the Gemfile to temporarily use a gem from
some path on your hard drive or from a particular git location. I
don’t understand this objection: When I add something like this to a
Gemfile:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>gem "foo", path: "/path/to/my/local/foo"
</code></pre>
</div>
<p>It’s a temporary change that never gets checked into version control.
When I’m done making whatever changes that “foo” needs in order to
work with the new version of <em>this</em> gem, and tests for both “foo” and
this gem are passing, I:</p>
<ul>
<li>publish foo</li>
<li>Update this gem’s gemspec to require the new version of foo</li>
<li>Remove the diversion from this project’s Gemfile</li>
<li>run “bundle install” to remove the record of that diversion from the
Gemfile.lock</li>
<li>run tests</li>
</ul>
<p>Only then do I check in my changes. The diversion never gets
published, either to version control or to rubygems.org. Since this
workflow never checks in “use this gem over there instead” diversions
made to Gemfile/Gemfile.lock, this reason is not a compelling reason
to keep Gemfile.lock out of version control.</p>
<h1 id="the-harm-of-not-checking-in-gemfilelock">The harm of not checking in gemfile.lock</h1>
<p>Here’s the problem with not checking in Gemfile.lock. Let’s say I
decide to find out if any changes to the ecosystem <em>have</em> broken my
gem, so I do a <code class="highlighter-rouge">bundle update</code>. Since my Gemfile properly contains
the line “gemspec”, the <code class="highlighter-rouge">bundle update</code> will get the latest versions
of the dependencies that are consistent with the version constraints
in the gemspec. All is fine, until I run tests and find that some of
them are now broken. Now, what information I have about what versions
changed?</p>
<p>Absolutely nothing. Well, <code class="highlighter-rouge">bundler update</code> did tell me what version
of each gem it is using <em>now</em>, so I can scroll back and see, but I
have no information about what version was being used the last time I
ran the tests and had them pass. That information is in the version
of the Gemfile.lock that just got clobbered. However, if Gemfile.lock
were checked in, the version control system could tell I what version
I was running before. I could even revert to the previous
Gemfile.lock and start over. But if Gemfile.lock is not checked in,
the best I can do is to get the prior version of the Gemfile.lock from
my backups, if I have them.</p>
<h1 id="a-reason-that-checking-in-gemfilelock-might-actually-be-bad">A reason that checking in Gemfile.lock might actually be bad.</h1>
<p>There may be a good reason, not mentioned by Mrk. Katz, to keep
Gemfile.lock out of version control: So that continuous integration
tests can find out when changes in the ecosystem have broken your gem.
If you check in a Gemfile.lock, then when the CI system does a <code class="highlighter-rouge">bundle
install</code> and then <code class="highlighter-rouge">bundle exec rake test</code> (or whatever), it will run
with exactly the same gem versions you developed with. That’s fine
for showing that you didn’t check in any test failures, but it won’t
show you if the ecosystem has changed in a way that breaks your gem.
If you don’t check in Gemfile.lock, then the “bundle install” done by
CI will get the latest version of every gem that is consistent with
the versions constraints in the gemspec. In that case, the CI system
will be able to warn you that the ecosystem has broken your gem.</p>
<p>That’s a plus, but not an overwhelming plus for me. In any case, the
CI system could be made to do a “bundle update” before running tests;
that way the Gemfile.lock could be checked in and CI could still let
you know when the ecosystem has broken your gem.</p>
REST API Versioning2014-08-01T18:00:00+00:00http://wconrad.github.com/20140801/rest-api-versioning<p>These are notes summarizing my research into how to version a RESTful
API. These notes do not represent actual experience. They’re just me
remembering links I found useful.</p>
<h1 id="restful-api">Restful API</h1>
<p>Cribbed from <a href="http://www.troyhunt.com/2014/02/your-api-versioning-is-wrong-which-is.html">Your API versioning is wrong…</a> by Troy Hunt.</p>
<p>There are three ways to version a RESTful API:</p>
<ul>
<li>Something in the URI (“…/api/v2/…”)</li>
<li>Something in the request header (“api-version: 2”)</li>
<li>Something in the Accept header (“Accept: application/apiname.v2+json”)</li>
</ul>
<p>Troy does them all at the same time. His framework makes it easy.</p>
<h1 id="semantic-versioning">Semantic versioning</h1>
<p>RESTful routes use <a href="http://semver.org/">semantic versioning</a>, but only the major
version number is used because only breaking API changes matter.
Adding a route never hurt anyone. Deleting one, or changing one’s
meaning, does.</p>
<p><a href="https://github.com/Sutto/rocket_pants">rocketpants</a></p>
Rails HATEOAS notes2014-08-01T15:00:00+00:00http://wconrad.github.com/20140801/rails-hateoas-research<p>These are notes summarizing my research into how to fully implement
REST in Rails with JSON responses. These notes do not represent
actual experience. They’re just me remembering links I found useful.</p>
<h1 id="restful-api">Restful API</h1>
<p>REST comes in levels, according to the Richardson Maturity Model as
<a href="http://martinfowler.com/articles/richardsonMaturityModel.html">explained by Martin Fowler</a>.</p>
<ul>
<li>Level 1 - Resources</li>
<li>Level 2 - + verbs</li>
<li>Level 3 - + hypermedia controls</li>
</ul>
<p>Level 2 is what every Rails RESTful application does.</p>
<p>Level 3, Means that the response indicates what further routes the
client may wish to do. If the client just created an object, the
response may include the routes to show or delete that object.</p>
<p>Level 3 is also called <a href="http://en.wikipedia.org/wiki/HATEOAS">HATEOAS<sup>1</sup></a> (wikipedia), for
Hypermedia as the Engine of Application State.</p>
<p>Martin’s article uses XML responses with <link /> elements as defined by
the <a href="http://atompub.org/rfc4287.html">ATOM publishing protocol</a> (<a href="http://tools.ietf.org/html/rfc4287">RFC 4287</a>), because “ATOM
… is generally seen as a leader in level 3 restfulness.”</p>
<h1 id="how-to-get-links-into-the-json-response-in-rails">How to get links into the JSON response in Rails</h1>
<p>Most of this section is cribbed from <a href="http://www.andylindeman.com/2010/11/13/hateoas-in-rails.html">Rails Dilemma: HATEOAS in
XML/JSON Responses</a> by Andy Lindeman.</p>
<p>Here’s an example of HATEOAS in a JSON response:</p>
<div class="highlighter-rouge"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nt">"book"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nt">"id"</span><span class="p">:</span><span class="mi">1</span><span class="p">,</span><span class="w">
</span><span class="nt">"name"</span><span class="p">:</span><span class="s2">"Ender's Game"</span><span class="p">,</span><span class="w">
</span><span class="nt">"isbn"</span><span class="p">:</span><span class="s2">"0812550706"</span><span class="p">,</span><span class="w">
</span><span class="nt">"purchase_url"</span><span class="p">:</span><span class="s2">"http://www.example.com/books/1/purchase"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre>
</div>
<p>The <code class="highlighter-rouge">purchase_url</code> attribute is it. Once you’ve got a book in your
hand, that’s the URI to purchase it.</p>
<p>Andy’s article is about how to properly architect your Rails
application to do this. The issue is:</p>
<ul>
<li>The default rendering method is to call to_json on a model, but</li>
<li>models don’t have access to route helpers.</li>
</ul>
<p>Andy describes several workarounds, but the winner looks like his
option 1: Render a view/partial that builds the JSON as needed. Views
<em>do</em> have access to route helpers.</p>
<p>Andy rejected option 1, but commenter Brandon Beacher explained why he
thinks it’s best, and gave an <a href="https://gist.github.com/brandon-beacher/766646">example</a> of how he’s done it. It
looks like an attractive approach to me. The implementation looks
simple, it doesn’t fight Rails, and his argument is compelling:</p>
<ul>
<li>
<p>“The json and xml representations of a model are just views, no?”</p>
</li>
<li>
<p>“We’d never implement a to_html method on a model.”</p>
</li>
<li>
<p>“And we don’t balk at creating html views because we’ll have to
maintain them as we add and remove attributes.”</p>
</li>
<li>
<p>“The json and xml views will be in the same folder alongside the
html ones.”</p>
</li>
</ul>
<h1 id="nick-sutterers-hal-proposal">Nick Sutterer’s HAL proposal</h1>
<p>Nick Sutterer <a href="http://nicksda.apotomo.de/tag/hateoas/">proposes the use of Mike Kelly’s [HAL][8] protocol</a>
to encode links in the JSON response. His starting example of a
RESTful JSON response differs from Andy Lindeman’s example in that it
includes a <em>rel</em> (relation) attribute. This is the same information
included in the ATOM nodes in the XML responses that <a href="http://martinfowler.com/articles/richardsonMaturityModel.html">Martin Fowler
shows</a>.</p>
<div class="highlighter-rouge"><pre class="highlight"><code><span class="p">{</span><span class="nt">"location"</span><span class="p">:</span><span class="s2">"desk"</span><span class="p">,</span><span class="w">
</span><span class="nt">"fruits"</span><span class="p">:[],</span><span class="w">
</span><span class="nt">"links"</span><span class="p">:[</span><span class="w">
</span><span class="p">{</span><span class="nt">"rel"</span><span class="p">:</span><span class="s2">"self"</span><span class="p">,</span><span class="w"> </span><span class="nt">"href"</span><span class="p">:</span><span class="s2">"http://bowls/desk"</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="nt">"rel"</span><span class="p">:</span><span class="s2">"fruits"</span><span class="p">,</span><span class="nt">"href"</span><span class="p">:</span><span class="s2">"http://bowls/desk/fruits"</span><span class="p">}</span><span class="w">
</span><span class="p">]}</span><span class="w">
</span></code></pre>
</div>
<p>After applying HAL, this becomes:</p>
<div class="highlighter-rouge"><pre class="highlight"><code><span class="p">{</span><span class="nt">"location"</span><span class="p">:</span><span class="s2">"desk"</span><span class="p">,</span><span class="w">
</span><span class="nt">"_embedded"</span><span class="p">:{</span><span class="nt">"fruits"</span><span class="p">:[]},</span><span class="w">
</span><span class="nt">"_links"</span><span class="p">:{</span><span class="w">
</span><span class="nt">"self"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"href"</span><span class="p">:</span><span class="s2">"http://bowls/desk"</span><span class="p">},</span><span class="w">
</span><span class="nt">"fruits"</span><span class="p">:{</span><span class="nt">"href"</span><span class="p">:</span><span class="s2">"http://bowls/desk/fruits"</span><span class="p">}</span><span class="w">
</span><span class="p">}}</span><span class="w">
</span></code></pre>
</div>
<ul>
<li>The _links section seems simple enough.</li>
<li>_Leading _underscores _are _obnoxious.</li>
<li>The _embedded section is for nested resources, I think..</li>
</ul>
<p>I don’t know if HAL is good or not. It makes me uneasy–the responses
are starting to get complicated. I don’t know if it’s as useful as it
is complex.</p>
<p>The money quote from Nick’s article is this:</p>
<blockquote>
<p>“Roy Fielding, the inventor of REST, states that your API is RESTFUL
if and only if it is hypermedia-driven! <em>In my words, that’ll mean
no URL code should be hard-wired into your REST client – except for
the single entry point URL.</em>” (italics mine)</p>
</blockquote>
<hr />
<p><sup>1</sup> <em>HATEOAS</em>? Hate the Organization of American States?</p>
Dynamic dispatch is time travel for if statements2014-07-25T20:00:00+00:00http://wconrad.github.com/20140725/time-travelling-if-statements<p><em>Dynamic dispatch is how conditional logic travels backwards in time</em></p>
<h1 id="what-is-dynamic-dispatch">What is dynamic dispatch?</h1>
<p><a href="http://en.wikipedia.org/wiki/Dynamic_dispatch">Dynamic dispatch</a> is
when your language decides at run time which actual method to call.
For example:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">Foo</span>
<span class="k">def</span> <span class="nf">say</span>
<span class="nb">puts</span> <span class="s2">"foo"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">class</span> <span class="nc">Bar</span> <span class="o"><</span> <span class="no">Foo</span>
<span class="k">def</span> <span class="nf">say</span>
<span class="nb">puts</span> <span class="s2">"bar"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">make_noise</span><span class="p">(</span><span class="n">o</span><span class="p">)</span>
<span class="n">o</span><span class="p">.</span><span class="nf">say</span>
<span class="k">end</span>
<span class="n">make_noise</span><span class="p">(</span><span class="no">Foo</span><span class="p">.</span><span class="nf">new</span><span class="p">)</span> <span class="c1"># => "foo"</span>
<span class="n">make_noise</span><span class="p">(</span><span class="no">Bar</span><span class="p">.</span><span class="nf">new</span><span class="p">)</span> <span class="c1"># => "bar"</span></code></pre></figure>
<p>On encoutering <code class="highlighter-rouge">o.say</code>, Ruby decides <em>at runtime</em>, based on o’s type,
whether to call <code class="highlighter-rouge">Foo#say</code> or <code class="highlighter-rouge">Bar#say</code>. That’s all dynamic dispatch
is. I remember it being a big deal to C programmers when they first
saw C++, but it’s pretty routine now.</p>
<h1 id="logging-without-time-travel">Logging without time travel</h1>
<p>Let’s have an object that needs to do some logging, or not.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">Foo</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">verbose</span><span class="p">)</span>
<span class="vi">@verbose</span> <span class="o">=</span> <span class="n">verbose</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">bar</span>
<span class="nb">print</span> <span class="s2">"bar starting..."</span> <span class="k">if</span> <span class="vi">@verbose</span>
<span class="c1"># ...</span>
<span class="nb">puts</span> <span class="s2">" done"</span> <span class="k">if</span> <span class="vi">@verbose</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>This is no good because we keep repeating <code class="highlighter-rouge">if @verbose</code>. We have
duplicated how to decide whether or not to log.</p>
<p>This class also has too many concerns. It must do whatever it is that
a Foo does, <em>and</em> it must be concerened with how and whether log. We
can fix the duplication, and give this class less responsibility.</p>
<h1 id="logging-gets-a-class-of-its-own-but-still-no-time-travel">Logging gets a class of its own (but still no time travel)</h1>
<p>Let’s move how and whether to log into its own class:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">Log</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">verbose</span><span class="p">)</span>
<span class="vi">@verbose</span> <span class="o">=</span> <span class="n">verbose</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">puts</span><span class="p">(</span><span class="n">s</span><span class="p">)</span>
<span class="nb">puts</span> <span class="n">s</span> <span class="k">if</span> <span class="vi">@verbose</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">print</span><span class="p">(</span><span class="n">s</span><span class="p">)</span>
<span class="nb">print</span> <span class="n">s</span> <span class="k">if</span> <span class="vi">@verbose</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>Foo now takes a log:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">Foo</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">log</span><span class="p">)</span>
<span class="vi">@log</span> <span class="o">=</span> <span class="n">log</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">bar</span>
<span class="vi">@log</span><span class="p">.</span><span class="nf">print</span> <span class="s2">"bar starting..."</span>
<span class="c1"># ...</span>
<span class="vi">@log</span><span class="p">.</span><span class="nf">puts</span> <span class="s2">" done"</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>And in use:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">use_foo</span><span class="p">(</span><span class="n">verbose</span><span class="p">)</span>
<span class="n">log</span> <span class="o">=</span> <span class="no">Log</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">verbose</span><span class="p">)</span>
<span class="no">Foo</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">log</span><span class="p">).</span><span class="nf">bar</span>
<span class="k">end</span>
<span class="n">use_foo</span><span class="p">(</span><span class="kp">true</span><span class="p">)</span> <span class="c1"># => "bar starting... done"</span>
<span class="n">use_foo</span><span class="p">(</span><span class="kp">false</span><span class="p">)</span> <span class="c1"># =></span></code></pre></figure>
<p>This is much better. How and whether to log is now in its own class.</p>
<p>But look at these two lines:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="nb">puts</span> <span class="n">s</span> <span class="k">if</span> <span class="vi">@verbose</span>
<span class="p">.</span><span class="nf">.</span><span class="p">.</span>
<span class="nf">print</span> <span class="n">s</span> <span class="k">if</span> <span class="vi">@verbose</span></code></pre></figure>
<p>A condition that is repeated may be a sign of code that can benefit
from polymorphism. Let’s see how.</p>
<h1 id="applying-the-null-object-pattern">Applying the Null Object pattern</h1>
<p>Let’s apply the <a href="http://en.wikipedia.org/wiki/Null_Object_pattern">null object
pattern</a> to the
logger and see what happens. We’ll remove all of the conditional code
from Log:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">Log</span>
<span class="k">def</span> <span class="nf">puts</span><span class="p">(</span><span class="n">s</span><span class="p">)</span>
<span class="nb">puts</span> <span class="n">s</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">print</span><span class="p">(</span><span class="n">s</span><span class="p">)</span>
<span class="nb">print</span> <span class="n">s</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>and introduce a NullLog which has the same signature, but does nothing:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">NullLog</span>
<span class="k">def</span> <span class="nf">puts</span><span class="p">(</span><span class="n">s</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">print</span><span class="p">(</span><span class="n">s</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>Then, in use:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">use_foo</span><span class="p">(</span><span class="n">verbose</span><span class="p">)</span>
<span class="n">log</span> <span class="o">=</span> <span class="p">(</span><span class="n">verbose</span> <span class="p">?</span> <span class="no">Log</span> <span class="p">:</span> <span class="no">NullLog</span><span class="p">).</span><span class="nf">new</span>
<span class="no">Foo</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">log</span><span class="p">).</span><span class="nf">bar</span>
<span class="k">end</span>
<span class="n">use_foo</span><span class="p">(</span><span class="kp">true</span><span class="p">)</span> <span class="c1"># => "bar starting... done"</span>
<span class="n">use_foo</span><span class="p">(</span><span class="kp">false</span><span class="p">)</span> <span class="c1"># =></span></code></pre></figure>
<p>The Log class has retained the knowledge of <em>how</em> to log, but it no
longer is responsible for knowing <em>whether</em> to log. We’ve given that
responsibility to the class that is creating the log instance.</p>
<h2 id="theres-the-time-travel">There’s the time travel</h2>
<p>Through the use of polymorphism and dynamic dispatch, we have
“time-traveled” the decision of whether to log from when the logging
is done to earlier in the program’s execution, when the log object was
made.</p>
<h1 id="a-factory">A Factory</h1>
<p>There’s another pattern I would usually apply here. The creation of
the Log or NullLog can be moved into a <a href="http://en.wikipedia.org/wiki/Factory_%28object-oriented_programming%29">factory
method</a>:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">Log</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">make</span><span class="p">(</span><span class="n">verbose</span><span class="p">)</span>
<span class="p">(</span><span class="n">verbose</span> <span class="p">?</span> <span class="nb">self</span> <span class="p">:</span> <span class="no">NullLog</span><span class="p">).</span><span class="nf">new</span>
<span class="k">end</span>
<span class="p">.</span><span class="nf">.</span><span class="p">.</span>
<span class="nf">end</span></code></pre></figure>
<p>and in use:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">use_foo</span><span class="p">(</span><span class="n">verbose</span><span class="p">)</span>
<span class="n">log</span> <span class="o">=</span> <span class="no">Log</span><span class="p">.</span><span class="nf">make</span><span class="p">(</span><span class="n">verbose</span><span class="p">)</span>
<span class="no">Foo</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">log</span><span class="p">).</span><span class="nf">bar</span>
<span class="k">end</span>
<span class="n">use_foo</span><span class="p">(</span><span class="kp">true</span><span class="p">)</span> <span class="c1"># => "bar starting... done"</span>
<span class="n">use_foo</span><span class="p">(</span><span class="kp">false</span><span class="p">)</span> <span class="c1"># =></span></code></pre></figure>
Parsing Fortran's Hollerith constant with Rattler2014-06-21T11:30:00+00:00http://wconrad.github.com/20140621/parsing-hollerith-constant-with-rattler<h1 id="the-problem">The problem</h1>
<p>I recent wondered whether Rattler could parse something as nasty
as Fortran’s Hollerith constant. It turns out, it can, quite
easily.</p>
<p>A hollerith constant is a crazy way of specifying a string literal
which includes the length of the string. For example, here’s a Hollerith constant for the string “HELLO”</p>
<figure class="highlight"><pre><code class="language-fortran" data-lang="fortran"><span class="w"> </span><span class="mi">5</span><span class="n">HHELLO</span></code></pre></figure>
<p>It breaks down like this:</p>
<ul>
<li>“5” - There are five characters in this…</li>
<li>“H” - … Hollerith constant</li>
<li>“HELLO” - And they are “HELLO”</li>
</ul>
<p>If you were trying to design a syntax to befuddle parsers, that’d be
it.</p>
<h1 id="the-solution">The solution</h1>
<p>Here’s a parser demonstrating the technique:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="c1">#!/usr/bin/env ruby</span>
<span class="nb">require</span> <span class="s2">"rattler"</span>
<span class="k">class</span> <span class="nc">Hollerith</span> <span class="o"><</span> <span class="no">Rattler</span><span class="o">::</span><span class="no">Runtime</span><span class="o">::</span><span class="no">ExtendedPackratParser</span>
<span class="n">grammar</span> <span class="sx">%{
hollerith <- ~(integer ~{count = _} "H") @(. &{(count -= 1) >= 0} )+
integer <- @(DIGIT+) { _.to_i }
}</span>
<span class="k">end</span>
<span class="nb">p</span> <span class="no">Hollerith</span><span class="p">.</span><span class="nf">parse!</span><span class="p">(</span><span class="s2">"5HHello..."</span><span class="p">)</span> <span class="c1"># "Hello"</span></code></pre></figure>
<p>The input string contains the suffix “…”, which we can see was not
included in the result. It stopped after five characters, just as it
should.</p>
<h1 id="how-it-works">How it works</h1>
<p>Let’s break down the definition of <em>hollerith</em>.</p>
<div class="highlighter-rouge"><pre class="highlight"><code>hollerith <-
~( # Don't include this group in the parse tree
integer ~{count = _} # parse an integer, then set variable
# count to that integer
"H" # Parse an "H"
)
@( # Include this group in the parse tree
# as a single string
. # parse any character
&{(count -= 1) >= 0} # Decrement count. Succeed until
# it goes negative.
)+ # Repeat one or more times
</code></pre>
</div>
<p>Rattler has passed my acid test for parsers with elegance and ease. I
think the next thing to do is to convert the Fortran interpreter from
my hand-rolled parser to Rattler and see how it holds up under fire.</p>
Rattler -- Yet another Parsing Expression Grammar for Ruby2014-06-19T17:00:00+00:00http://wconrad.github.com/20140619/rattler<p>I just noticed this: <a href="http://www.rubyflow.com/items/11157-language-designers-wanted">Language Designers
Wanted</a>.
It announces <a href="https://github.com/jarhart/rattler">rattler</a>, yet
another PEG for Ruby.</p>
<p>A quick look shows that it is an external DSL. Those are prettier to
work with than internal DSLs, but somewhat less flexible. As long as
the DSL gives you a way to do everything you need to do, that’s not a
problem. It also appears to generate useful parse trees pretty
easily. But the most exciting thing about Rattler is that it handles
left recursion. None of the other Ruby PEGs I’ve worked with
(Treetop, Citrus, Parslet) will do that.</p>
<p>The acid test of any parser is how well it handles crazy things like
Fortran’s Hollerith constant<sup>1</sup>, where you precede a string
with the count of how many characters are in it, and then a <code class="highlighter-rouge">H</code>:</p>
<figure class="highlight"><pre><code class="language-fortran" data-lang="fortran"><span class="w"> </span><span class="k">PRINT</span><span class="w"> </span><span class="o">*</span><span class="p">,</span><span class="w"> </span><span class="mi">10</span><span class="w">
</span><span class="mi">10</span><span class="w"> </span><span class="k">FORMAT</span><span class="w"> </span><span class="p">(</span><span class="l">1X</span><span class="p">,</span><span class="w"> </span><span class="l">5HHELLO</span><span class="p">)</span></code></pre></figure>
<p>This is how you might print the word “HELLO”. The “5HHELLO” breaks
down like this:</p>
<ul>
<li>“5” - There are five characters in this…</li>
<li>“H” - … Hollerith constant</li>
<li>“HELLO” - And they are “HELLO”</li>
</ul>
<p>This crazy syntax goes all the way back to Fortran I, and is still
supported in Fortran77, so a FORTRAN compiler has to be able to handle
it.</p>
<p>So, that’s the question. Can Rattler change its parsing behavior
based on some prior parse node? Can it use the count to know how many
characters are in the string? Not because every parser should be able
to parse Fortran, but because being able to parse a Hollerith constant
means that the parser has a way to do unusual things when needed.</p>
<p><sup>1</sup> When used in the FORMAT statement, The <em>nHaaa</em> syntax is
not actually a Hollerith constant. It’s a Hollerith format
specification. Whatever. I only mention this to keep any of the
three people who have read the Fortran specification from bugging me
about it.</p>
Little sibling classes2014-06-18T03:00:00+00:00http://wconrad.github.com/20140618/little-sibling-classes<h1 id="three-classes-squished-into-one">Three classes squished into one</h1>
<p>I was working with a class that held two boolean flags, and a little
bit of conditional logic:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">OutputFragment</span>
<span class="kp">attr_reader</span> <span class="ss">:value</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="n">optional</span><span class="p">,</span> <span class="n">overflow</span><span class="p">)</span>
<span class="vi">@value</span> <span class="o">=</span> <span class="n">value</span>
<span class="vi">@optional</span> <span class="o">=</span> <span class="n">optional</span>
<span class="vi">@overflow</span> <span class="o">=</span> <span class="n">overflow</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">optional?</span>
<span class="vi">@optional</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">overflow?</span>
<span class="vi">@overflow</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">peek</span>
<span class="k">if</span> <span class="vi">@overflow</span>
<span class="s2">"(overflow)"</span>
<span class="k">elsif</span> <span class="vi">@optional</span>
<span class="s2">"[</span><span class="si">#{</span><span class="n">value</span><span class="si">}</span><span class="s2">]"</span>
<span class="k">else</span>
<span class="vi">@value</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>OutputFragment wasn’t bad, but the code that created instances of of
it was:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">add_string</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="ss">optional: </span><span class="kp">false</span><span class="p">)</span>
<span class="vi">@fragments</span> <span class="o"><<</span> <span class="no">OutputFragment</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="n">optional</span><span class="p">,</span> <span class="kp">false</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">set_overflow</span>
<span class="vi">@fragments</span> <span class="o"><<</span> <span class="no">OutputFragment</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s2">""</span><span class="p">,</span> <span class="kp">false</span><span class="p">,</span> <span class="kp">true</span><span class="p">)</span>
<span class="k">end</span></code></pre></figure>
<p>Just… yuck. I could have used named arguments to make it more
readable:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">add_string</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="ss">optional: </span><span class="kp">false</span><span class="p">)</span>
<span class="vi">@fragments</span> <span class="o"><<</span>
<span class="no">OutputFragment</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">value</span><span class="ss">:value</span><span class="p">,</span> <span class="ss">optional: </span><span class="n">optional</span><span class="p">,</span> <span class="ss">overflow: </span><span class="kp">false</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">set_overflow</span>
<span class="vi">@fragments</span> <span class="o"><<</span> <span class="no">OutputFragment</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">value</span><span class="ss">:""</span><span class="p">,</span> <span class="ss">optional: </span><span class="kp">false</span><span class="p">,</span> <span class="ss">overflow: </span><span class="kp">true</span><span class="p">)</span>
<span class="k">end</span></code></pre></figure>
<p>But that’s just lipstick on a pig. The real problem here is that
there are three separate classes being squished into one. Let’s look
at the possible values of the <em>optional</em> and <em>overflow</em> attributes,
and what type of fragment they represent:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>overflow optional type of fragment
F F required
F T optional
T F overflow
</code></pre>
</div>
<h1 id="splitting-the-class">Splitting the class</h1>
<p>So what if we actually make three separate classes, one for each type
of fragment? Let’s see what that looks like:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">RequiredOutputFragment</span>
<span class="kp">attr_reader</span> <span class="ss">:value</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
<span class="vi">@value</span> <span class="o">=</span> <span class="n">value</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">optional?</span>
<span class="kp">false</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">overflow?</span>
<span class="kp">false</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">peek</span>
<span class="n">value</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">OptionalOutputFragment</span>
<span class="kp">attr_reader</span> <span class="ss">:value</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
<span class="vi">@value</span> <span class="o">=</span> <span class="n">value</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">optional?</span>
<span class="kp">true</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">overflow?</span>
<span class="kp">false</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">peek</span>
<span class="s2">"[</span><span class="si">#{</span><span class="vi">@value</span><span class="si">}</span><span class="s2">]"</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">OverflowOutputFragment</span>
<span class="k">def</span> <span class="nf">value</span>
<span class="s2">""</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">optional?</span>
<span class="kp">false</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">overflow?</span>
<span class="kp">true</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">peek</span>
<span class="s2">"(overflow)"</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>In a language without duck typing, these would each derive from some
interface or base class. With duck typing, and in cases like this
where there is no shared behavior, there’s no need (and even in cases
of shared behavior, mixin modules may be better than inheritance).</p>
<p>You might be thinking that the one class kind of exploded into a lot
code, and you’d be right. So what’s good about this? We did get rid
of the two boolean flags, and of the conditional in <code class="highlighter-rouge">#peek</code>. But
the real payoff comes where the classes are made:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">add_string</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="ss">optional: </span><span class="kp">false</span><span class="p">)</span>
<span class="vi">@fragments</span> <span class="o"><<</span> <span class="k">if</span> <span class="n">optional</span>
<span class="no">OptionalOutputFragment</span>
<span class="k">else</span>
<span class="no">RequiredOutputFragment</span>
<span class="k">end</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">set_overflow</span>
<span class="vi">@fragments</span> <span class="o"><<</span> <span class="no">OverflowOutputFragment</span><span class="p">.</span><span class="nf">new</span>
<span class="k">end</span></code></pre></figure>
<p>The removal of those boolean flags (and, in the case of
<code class="highlighter-rouge">OverflowOutputFragment</code>, of the <code class="highlighter-rouge">value</code> argument as well), makes this
code clearer, at least to me. What do you think?</p>
Some code is hard to unit test2014-06-13T12:00:00+00:00http://wconrad.github.com/20140613/hard-to-unit-test<p>I’m bothered that I can’t figure out how to write a reasonable unit
test for a reasonable piece of code.</p>
<p>I’ve been writing unit tests for a long time, long enough that many
unit tests roll off of my fingers pretty easily. I enjoy getting the
code into a shape where it’s both beautiful and easy to unit test, and
for a long time I subscribed to the idea that if the code was hard to
unit test, there was something wrong with it. I’m not so sure any
more. It could be that some code that’s just fine is still hard to
unit test.</p>
<p>Here’s an example of some code that’s nasty to unit test, but looks
good to me:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="c1"># -*- coding: utf-8 -*-</span>
<span class="k">module</span> <span class="nn">Fortran77</span>
<span class="k">class</span> <span class="nc">FormatSpecification</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">edit_descriptors</span><span class="p">)</span>
<span class="vi">@edit_descriptors</span> <span class="o">=</span> <span class="n">edit_descriptors</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">write</span><span class="p">(</span><span class="n">unit</span><span class="p">,</span> <span class="n">io_list_iterator</span><span class="p">,</span> <span class="n">format_flags</span><span class="p">)</span>
<span class="n">record</span> <span class="o">=</span> <span class="n">unit</span><span class="p">.</span><span class="nf">new_formatted_record</span>
<span class="vi">@edit_descriptors</span><span class="p">.</span><span class="nf">error_if_disagrees_with_io_list</span> <span class="n">io_list_iterator</span>
<span class="kp">loop</span> <span class="k">do</span>
<span class="vi">@edit_descriptors</span><span class="p">.</span><span class="nf">write</span> <span class="n">record</span><span class="p">,</span> <span class="n">io_list_iterator</span><span class="p">,</span> <span class="n">format_flags</span>
<span class="k">break</span> <span class="k">if</span> <span class="n">io_list_iterator</span><span class="p">.</span><span class="nf">end?</span>
<span class="n">unit</span><span class="p">.</span><span class="nf">write_formatted</span> <span class="n">record</span>
<span class="n">record</span> <span class="o">=</span> <span class="n">unit</span><span class="p">.</span><span class="nf">new_formatted_record</span>
<span class="k">end</span>
<span class="n">unit</span><span class="p">.</span><span class="nf">write_formatted</span> <span class="n">record</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">to_s</span>
<span class="s2">"(</span><span class="si">#{</span><span class="vi">@edit_descriptors</span><span class="si">}</span><span class="s2">)"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>There isn’t a thing wrong with #write that I can see, and yet it’d be
a chore to test. It interacts with three different objects, calling
five different methods:</p>
<ul>
<li><code class="highlighter-rouge">edit_descriptors.error_if_disagrees_with_io_list</code></li>
<li><code class="highlighter-rouge">edit_descriptors.write_record</code></li>
<li><code class="highlighter-rouge">unit.new_formatted_record</code></li>
<li><code class="highlighter-rouge">unit.write_formatted</code></li>
<li><code class="highlighter-rouge">io_list_iterator.end?</code></li>
</ul>
<p>The test will spend much effort creating objects (or doubles) to
interact with the test subject. If the test uses real objects, they
have to be created, and that’s not trivial. If it uses doubles, it
will be a source of friction, fantastically brittle, having an
intimate awareness of internals that are certain to change.</p>
<p>It feels like defeat to not be able to write a reasonable unit test
for this reasonable code. I think that feeling is wrong, though.
It’s not as important that there’s a unit test for a piece of code as
it is that there is a test <em>somewhere</em>.</p>
<p>So, I think I’m going to let the integration tests cover this one.
It’s not that bad: A great deal of basic101 is covered through
integration tests, and that worked out fine. Integration tests aren’t
as fast to run, but they are the only test you can write that are
immune from how the code does it. They are the ultimate in
resiliancy.</p>
<p>In a program with significant external interactions (database,
network, etc.), integration tests are harder to write, which changes
the math: some unit tests which are hard may become
worthwhile<sup>1</sup>. But in an interpreter which reads from files
and writes to files, integration tests are as easy as anything.</p>
<p><sup>1</sup> But see, for example, <a href="http://www.duncannisbet.co.uk/hexagonal-architecture-for-testers-part-1">hexagonal architecture</a> as a
way of making an application easier to test.</p>
Handling parse errors (cont'd)2014-06-09T12:00:00+00:00http://wconrad.github.com/20140609/handling-parse-errors-2<p>In my investigation (that is, googling) of handling parse errors in a
recursive descent compiler with backtracking, I learned that a
recursive descent compiler with backtracking and ordered choice (which
is my parser) is, in fact a PEG (Parsing Expression Grammar). I was
confused because most PEGs are so-called “packrat” parsers, which do
memoization in order to speed things up. My parser does no
memoization, so I thought it wasn’t a PEG. Wrong. It is. It’s just
not a particularly efficient one.</p>
<p>Anyhow. My problem is that errors like the extra comma at the end of
the third PRINT statement:</p>
<figure class="highlight"><pre><code class="language-fortran" data-lang="fortran"><span class="w"> </span><span class="k">PRINT</span><span class="w"> </span><span class="o">*</span><span class="p">,</span><span class="w"> </span><span class="s1">'a'</span><span class="p">,</span><span class="w"> </span><span class="s1">'bc'</span><span class="w">
</span><span class="k">PRINT</span><span class="w"> </span><span class="o">*</span><span class="w">
</span><span class="k">PRINT</span><span class="w"> </span><span class="o">*</span><span class="p">,</span><span class="w"> </span><span class="s1">'def'</span><span class="p">,</span><span class="w">
</span><span class="k">END</span></code></pre></figure>
<p>Were causing useless error messages like this:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>Expected /\Z/ at " PRINT *, 'a', 'bc'\n PRINT *\n PRINT *, 'def',\n END\n"
</code></pre>
</div>
<p>Yuck.</p>
<h1 id="turning-a-character-offset-into-line-and-column-number">Turning a character offset into line and column number</h1>
<p>Fixing this came in two parts. The first part was being able to
report the line, line number, and column that the error occured at.
This was a problem because none of that information is directly
available at the point the error is recognized. The parser uses
Ruby’s
<a href="http://www.ruby-doc.org/stdlib-2.1.1/libdoc/strscan/rdoc/StringScanner.html">StringScanner</a>
class, which treats the source as one big string. The only position
information the parser has is the byte offset from the beginning of
the file.</p>
<p>I solved that by wrapping the source code in a class named (drumroll,
please)… Source. Source responds to <code class="highlighter-rouge">#to_str</code>, so Ruby’s
StringScanner works fine with it. Yay for Ruby’s type conversion
convensions. Source has these methods:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">line_position</span><span class="p">(</span><span class="n">source_position</span><span class="p">)</span>
<span class="n">i</span> <span class="o">=</span> <span class="mi">0</span>
<span class="vi">@contents</span><span class="p">.</span><span class="nf">each_line</span><span class="p">.</span><span class="nf">with_index</span> <span class="k">do</span> <span class="o">|</span><span class="n">line</span><span class="p">,</span> <span class="n">line_index</span><span class="o">|</span>
<span class="n">j</span> <span class="o">=</span> <span class="n">i</span> <span class="o">+</span> <span class="n">line</span><span class="p">.</span><span class="nf">size</span>
<span class="k">if</span> <span class="p">(</span><span class="n">i</span><span class="p">.</span><span class="nf">.</span><span class="p">.</span><span class="nf">j</span><span class="p">).</span><span class="nf">include?</span><span class="p">(</span><span class="n">source_position</span><span class="p">)</span>
<span class="n">line_number</span> <span class="o">=</span> <span class="n">line_index</span> <span class="o">+</span> <span class="mi">1</span>
<span class="n">column_number</span> <span class="o">=</span> <span class="n">source_position</span> <span class="o">-</span> <span class="n">i</span>
<span class="k">return</span> <span class="no">LinePosition</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">line_number</span><span class="p">,</span> <span class="n">column_number</span><span class="p">)</span>
<span class="k">end</span>
<span class="n">i</span> <span class="o">=</span> <span class="n">j</span>
<span class="k">end</span>
<span class="k">raise</span> <span class="no">ArgumentError</span><span class="p">,</span> <span class="s2">"Invalid source offset"</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">line_at</span><span class="p">(</span><span class="n">line_number</span><span class="p">)</span>
<span class="vi">@contents</span><span class="p">.</span><span class="nf">lines</span><span class="p">[</span><span class="n">line_number</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span>
<span class="k">end</span></code></pre></figure>
<p><code class="highlighter-rouge">Source#line_position</code> takes a character offset from the beginning of
the file and returns the corresponding line number and column number.
Line numbers start from 1 and column numbers from 0 because that’s
what Emacs likes (when I put the cursor on the error message and hit
ENTER, Emacs will take me to that exact position in the source).</p>
<p>I’m not really proud of <code class="highlighter-rouge">Source#line_position</code>. It just seems a lot
like what you’d write in C. A lot of the time, if your algorithm
expressed in Ruby would be expressed similarly in C, there is a better
way to do it. I’d like a more declarative, functional approach, but I
don’t know it.</p>
<p>You’ll notice that <code class="highlighter-rouge">#line_position</code> and <code class="highlighter-rouge">#line</code>_at both convert the
source to lines and then throw the lines away. It doesn’t make sense
here to cache the lines, because these two methods are each called
exactly once, when printing the error message before aborting. Even
if the source is really big, they’ll be fast enough.</p>
<h1 id="which-error-to-report">Which error to report?</h1>
<p>The second part of the problem is having an error at the end of the
file produce an error message that points to the beginning. That’s
because of how I implemented backtracking in the parser. Here’s the
code to parse an arithmetic expression such as “1 + 2 + 3”:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">parse_arithmetic_expression</span>
<span class="n">left</span> <span class="o">=</span> <span class="n">parse_unary_expression</span>
<span class="k">while</span> <span class="p">(</span><span class="n">operator</span> <span class="o">=</span> <span class="n">maybe</span> <span class="p">{</span> <span class="n">parse_additive_operator</span> <span class="p">})</span>
<span class="n">right</span> <span class="o">=</span> <span class="n">parse_term</span>
<span class="n">left</span> <span class="o">=</span> <span class="no">BinaryOperation</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">left</span><span class="p">,</span> <span class="n">operator</span><span class="p">,</span> <span class="n">right</span><span class="p">)</span>
<span class="k">end</span>
<span class="n">left</span>
<span class="k">end</span></code></pre></figure>
<p>It’s interesting how easily this code handles left-associative
operators, something that was harder with the more formal parsers such
as Parslet. But that’s not why I’m showing this piece of code. I
just had to mention it because, well, it is kinda pretty.</p>
<p>The call to <code class="highlighter-rouge">#maybe</code> is speculatively calling the method
<code class="highlighter-rouge">#parse_additive_operator</code>. If that method succeeds, and parses an
additive operator, <code class="highlighter-rouge">#maybe</code> will return that. However, if that method
raises a FortranSyntaxError exception, then maybe will backtrack to
where the parser was when <code class="highlighter-rouge">#maybe</code> was called, and then return nil.
<code class="highlighter-rouge">#maybe</code> is dirt simple:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">zero_or_one</span>
<span class="n">bookmark</span> <span class="o">=</span> <span class="vi">@scanner</span><span class="p">.</span><span class="nf">bookmark</span>
<span class="k">begin</span>
<span class="k">yield</span>
<span class="k">rescue</span> <span class="no">FortranSyntaxError</span>
<span class="vi">@scanner</span><span class="p">.</span><span class="nf">goto_bookmark</span> <span class="n">bookmark</span>
<span class="kp">nil</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">alias</span> <span class="ss">:maybe</span> <span class="ss">:zero_or_one</span></code></pre></figure>
<p><code class="highlighter-rouge">#maybe</code>, and methods like it, are the source of our problem, because
they consume Syntax errors. The syntax error that was the root of the
problem disappears, and the method to parse the print statement ends
up raising its own Syntax error. Each method to parse something was
called by another method to parse something; each of them ends up
eating one Syntax error and emitting another. This continues to the
top:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">parse_program_units</span>
<span class="n">program_units</span> <span class="o">=</span> <span class="n">zero_or_more</span> <span class="p">{</span> <span class="n">parse_program_unit</span> <span class="p">}</span>
<span class="n">parse_end_of_source</span>
<span class="n">program_units</span>
<span class="k">end</span></code></pre></figure>
<p>ultimately, <code class="highlighter-rouge">#parse_program_unit</code> fails, causing zero_or_more to
return an empty array and backtrack to the beginning of the program.
<code class="highlighter-rouge">parse_end_of_source</code> then finds we aren’t at the end of the source
after all. That raises the last, final syntax error, and the program
ends with an error messages no more helpful than the “Check Engine”
light on your car.</p>
<p>It might be more helpful to report on that first error. But how do we
know which error is “first?” This compiler raises syntax error
exceptions as part of how it works normally, so many syntax errors
have already occured on the way to the error that actually revealed
the problem. The answer is to report the “deepest” error, the one
that occured the farthest into the file. That will either be the
location of the actual error, or at least somewhat close to it.</p>
<p>So way, way down, at the point these errors are raised, we’ll keep
track of which one occured the farthest into the file:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">error</span><span class="p">(</span><span class="n">message</span> <span class="o">=</span> <span class="s2">"Syntax error"</span><span class="p">)</span>
<span class="n">error</span> <span class="o">=</span> <span class="no">FortranSyntaxError</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">message</span><span class="p">,</span> <span class="vi">@source</span><span class="p">,</span> <span class="vi">@scanner</span><span class="p">.</span><span class="nf">pos</span><span class="p">)</span>
<span class="vi">@deepest_error</span> <span class="o">=</span> <span class="n">error</span><span class="p">.</span><span class="nf">deepest</span><span class="p">(</span><span class="vi">@deepest_error</span><span class="p">)</span>
<span class="k">raise</span> <span class="n">error</span>
<span class="k">end</span></code></pre></figure>
<p>When creating the exception, we give it the source and the character
offset, which it can use later to give a useful error message. Then,
we update @deepest_error, using <code class="highlighter-rouge">FortranSyntaxError#deepest</code>:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">deepest</span><span class="p">(</span><span class="n">error</span><span class="p">)</span>
<span class="k">if</span> <span class="n">error</span> <span class="o">&&</span> <span class="n">error</span><span class="p">.</span><span class="nf">source_position</span> <span class="o">></span> <span class="vi">@source_position</span>
<span class="n">error</span>
<span class="k">else</span>
<span class="nb">self</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>Finally, when the compiler gets a syntax error, it replaces it with
the deepest syntax error:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">compile</span>
<span class="n">source</span> <span class="o">=</span> <span class="no">Source</span><span class="p">.</span><span class="nf">from_file</span><span class="p">(</span><span class="vi">@source_path</span><span class="p">)</span>
<span class="n">scanner</span> <span class="o">=</span> <span class="no">Scanner</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
<span class="n">parser</span> <span class="o">=</span> <span class="no">Parser</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">scanner</span><span class="p">)</span>
<span class="vi">@program_units</span> <span class="o">=</span> <span class="n">parser</span><span class="p">.</span><span class="nf">parse_program_units</span>
<span class="k">rescue</span> <span class="no">ParseError</span> <span class="o">=></span> <span class="n">e</span>
<span class="k">raise</span> <span class="n">parser</span><span class="p">.</span><span class="nf">deepest_error</span>
<span class="k">end</span></code></pre></figure>
<p>That’s pretty much it.</p>
<h1 id="the-result">The result</h1>
<p>Here’s the error message now:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>/home/wayne/lab/fortran77/gem/examples/print_character.f:3.21:
PRINT *, 'def',
^
Error: Expected "'"
</code></pre>
</div>
<p>The actual text of the error message is as useless as ever, but now we
get an error position that’s either at or close to the actual point of
error. I think that’ll be good enough to let me continue to implement
more of the language. So, after I clean up a few TODO items that I
introduced (some code that could stand to have unit tests, etc.), then
it’s back to the FORMAT statement.</p>
Handling parse errors2014-06-08T12:00:00+00:00http://wconrad.github.com/20140608/handling-parse-errors<p>I’ve plowed through the TODO list. It feels good to have all those
little issues cleaned up. I’ve also got the E, D, F, and G format
specifiers working, and mostly (I think) compliant with the spec.
Eventually I’ll get this thing to the point where I can run the <a href="http://www.fortran-2000.com/ArnaudRecipes/fcvs21_f95.html">NIST
Fortran 77 test
suite</a>, and
then I’ll find out for sure. I expect that I’ll be non-compliant in
enough areas that just getting the tests to run at all will be
difficult, but we’ll see.</p>
<p>I also got a little bit more done on the FORMAT statement, which now
accepts the F (fixed), E (exponential), D (alias for E), G
(fixed/exponential, depending), a L (logical) edit descriptors:</p>
<figure class="highlight"><pre><code class="language-fortran" data-lang="fortran"><span class="w"> </span><span class="k">PRINT</span><span class="w"> </span><span class="mi">30</span><span class="p">,</span><span class="w"> </span><span class="mf">123.456</span><span class="p">,</span><span class="w"> </span><span class="mf">123.456</span><span class="p">,</span><span class="w"> </span><span class="mf">-123.456</span><span class="w">
</span><span class="mi">30</span><span class="w"> </span><span class="k">FORMAT</span><span class="w"> </span><span class="p">(</span><span class="l">F8.3</span><span class="p">,</span><span class="w"> </span><span class="l">E12.3</span><span class="p">,</span><span class="w"> </span><span class="l">D12.3</span><span class="p">)</span><span class="w">
</span><span class="k">PRINT</span><span class="w"> </span><span class="mi">40</span><span class="p">,</span><span class="w"> </span><span class="mf">0.1234</span><span class="p">,</span><span class="w"> </span><span class="mf">0.1234</span><span class="n">E10</span><span class="w">
</span><span class="mi">40</span><span class="w"> </span><span class="k">FORMAT</span><span class="w"> </span><span class="p">(</span><span class="l">G14.4</span><span class="p">)</span><span class="w">
</span><span class="k">PRINT</span><span class="w"> </span><span class="mi">50</span><span class="p">,</span><span class="w"> </span><span class="kc">.TRUE.</span><span class="p">,</span><span class="w"> </span><span class="kc">.FALSE.</span><span class="w">
</span><span class="mi">50</span><span class="w"> </span><span class="k">FORMAT</span><span class="w"> </span><span class="p">(</span><span class="l">L2</span><span class="p">)</span><span class="w">
</span><span class="k">END</span></code></pre></figure>
<div class="highlighter-rouge"><pre class="highlight"><code> 123.456 0.123E+03 -0.123E+03
0.1234
0.1234E+10
T
F
</code></pre>
</div>
<p>I want to go finish all of the other edit descriptors, but there’s a
problem I can’t ignore anymore: the parser’s error reporting stinks.
I could explain it in all it’s gory details, but I’ll just show you
the horror of it. Here’s a short program with an erroneous trailing
comma in the last PRINT statement:</p>
<figure class="highlight"><pre><code class="language-fortran" data-lang="fortran"><span class="w"> </span><span class="k">PRINT</span><span class="w"> </span><span class="o">*</span><span class="p">,</span><span class="w"> </span><span class="s1">'a'</span><span class="p">,</span><span class="w"> </span><span class="s1">'bc'</span><span class="w">
</span><span class="k">PRINT</span><span class="w"> </span><span class="o">*</span><span class="w">
</span><span class="k">PRINT</span><span class="w"> </span><span class="o">*</span><span class="p">,</span><span class="w"> </span><span class="s1">'def'</span><span class="p">,</span><span class="w">
</span><span class="k">END</span></code></pre></figure>
<p>And the actual error message this generates:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>Expected /\Z/ at " PRINT *, 'a', 'bc'\n PRINT *\n PRINT *, 'def',\n END\n"
</code></pre>
</div>
<p>Basically, all you get is what string or regex was expected, and what
is left of the program, as one big string. Because of backtracking,
the parser basically ends up telling you, “This doesn’t match a
program. Since it doesn’t match a program, I expected the end of the
file, but instead found this program. I tried a whole bunch of stuff,
which I’ve forgotten about so I can’t tell you. You’ll have to figure
it out.”</p>
<p>Nice, huh?</p>
<p>So now I’ve got to figure out how to get decent error reporting cooked
into this parser. I’ve been putting it off because I have only a
vague idea how to go about it. I can’t put it off any longer.</p>
We have FORMAT2014-06-07T13:00:00+00:00http://wconrad.github.com/20140607/we-have-format<p>We have FORMAT statements. This code:</p>
<figure class="highlight"><pre><code class="language-fortran" data-lang="fortran"><span class="w"> </span><span class="k">PRINT</span><span class="w"> </span><span class="mi">10</span><span class="p">,</span><span class="w"> </span><span class="mi">1</span><span class="w">
</span><span class="k">PRINT</span><span class="w"> </span><span class="mi">10</span><span class="p">,</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mi">2</span><span class="w">
</span><span class="k">PRINT</span><span class="w"> </span><span class="mi">10</span><span class="p">,</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mi">2</span><span class="p">,</span><span class="w"> </span><span class="mi">3</span><span class="w">
</span><span class="mi">10</span><span class="w"> </span><span class="k">FORMAT</span><span class="w"> </span><span class="p">(</span><span class="l">I3</span><span class="p">,</span><span class="w"> </span><span class="l">I3</span><span class="p">)</span><span class="w">
</span><span class="k">END</span></code></pre></figure>
<p>writes this output:</p>
<div class="highlighter-rouge"><pre class="highlight"><code> 1
1 2
1 2
3
</code></pre>
</div>
<p>It took a lot of thought to do this simple thing in a way that
promises spec compliance.</p>
<p>Now I’ve got to go back and fix all the TODOs, like these:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="c1">#TODO Use a hash to lookup labeled statements</span>
<span class="k">def</span> <span class="nf">labeled_statement</span><span class="p">(</span><span class="n">line_number</span><span class="p">)</span>
<span class="n">statement</span> <span class="o">=</span> <span class="vi">@statements</span><span class="p">.</span><span class="nf">find</span> <span class="k">do</span> <span class="o">|</span><span class="n">statement</span><span class="o">|</span>
<span class="c1">#TODO why is the variable called line_number, but the method</span>
<span class="c1"># talks about a label?</span>
<span class="n">statement</span><span class="p">.</span><span class="nf">label_matches?</span><span class="p">(</span><span class="n">line_number</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">unless</span> <span class="n">statement</span>
<span class="k">raise</span> <span class="no">LabelNotFound</span><span class="p">,</span> <span class="s2">"Label not found: </span><span class="si">#{</span><span class="n">line_number</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span>
<span class="n">statement</span>
<span class="k">end</span></code></pre></figure>
<p>When doing a refactoring or adding a feature, I often see things that
need to be changed. If they’re trivial, I just do it. But if they
require any thought at all (or especially if the tests aren’t passing
at the moment), I defer them to later by adding a TODO comment. After
I commit code, then I find and fix all the TODOs.</p>
<p>Especially early on, each TODO I fix can result in two more being
added. That’s because I’m still exploring the design, trying
different names, etc., and there are a lot of things that aren’t quite
right. But that’s not what happened this time. I was so interested
in getting FORMAT working that I implemented it <em>before</em> doing all the
TODOs. In the course of doing FORMAT, I added quite a few of them.
Now there are 18. They’re not broken windows, just dirty ones, but
they add up to code that isn’t quite right. It’s time to roll up my
sleeves and fix them.</p>
Concatenating Enumerator2014-06-03T22:00:00+00:00http://wconrad.github.com/20140603/concatenating-enumerator<p>First I’m going to complain/brag about how complex Fortran can be, and
then I’m going to show a neat bit of Ruby code that I’m using to help
tame that complexity. If you don’t care that much about ancient
computer languages, just scroll down to the bottom for the Ruby code.</p>
<h1 id="fortran-formatted-io-its-complicated">Fortran formatted I/O: It’s complicated!</h1>
<p>In my Fortran interpreter,
I’m taking a stab at Fortran’s somewhat intimidating format statement.
The format statement is a DSL for formatted I/O, and it is awash in
features. I’ll show some examples. Don’t try too hard to understand
them–this isn’t mean to be a tutorial, so I’m going to gloss over a
<em>lot</em>. Just bask in the complexity.</p>
<p>Here’s a simple example writing an integer, a float (“real”
in Fortran parlance<sup>1</sup>),</p>
<figure class="highlight"><pre><code class="language-fortran" data-lang="fortran"><span class="w"> </span><span class="k">WRITE</span><span class="w"> </span><span class="p">(</span><span class="o">*</span><span class="p">,</span><span class="w"> </span><span class="mi">10</span><span class="p">)</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mf">2.345</span><span class="p">,</span><span class="w"> </span><span class="s1">'FOO'</span><span class="w">
</span><span class="nl">10</span><span class="w"> </span><span class="k">FORMAT</span><span class="w"> </span><span class="p">(</span><span class="l">I4</span><span class="p">,</span><span class="w"> </span><span class="l">F7.2</span><span class="p">,</span><span class="w"> </span><span class="l">A4</span><span class="p">)</span></code></pre></figure>
<p>You can read the format statement as “an integer, printed using 4
columns; a float (sorry, “real”) using 7 characters, 2 of which are
for the fractional part, and a “character” (the Fortran name for a
string <sup>2</sup> ).</p>
<p>This writes:</p>
<div class="highlighter-rouge"><pre class="highlight"><code> 1 2.35 FOO
</code></pre>
</div>
<p>That’s not too complex, but just wait. Let’s have a little bit of
automatic repetition:</p>
<figure class="highlight"><pre><code class="language-fortran" data-lang="fortran"><span class="w"> </span><span class="k">WRITE</span><span class="w"> </span><span class="p">(</span><span class="o">*</span><span class="p">,</span><span class="w"> </span><span class="mi">10</span><span class="p">)</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mi">2</span><span class="p">,</span><span class="w"> </span><span class="mi">3</span><span class="p">,</span><span class="w"> </span><span class="mi">4</span><span class="p">,</span><span class="w"> </span><span class="mi">5</span><span class="w">
</span><span class="nl">10</span><span class="w"> </span><span class="k">FORMAT</span><span class="w"> </span><span class="p">(</span><span class="l">I2</span><span class="p">,</span><span class="w"> </span><span class="l">I2</span><span class="p">)</span></code></pre></figure>
<p>There are five items being written, but only two format specifiers
(<code class="highlighter-rouge">I2, I2</code>). No problem, Fortran’s got your back. It will simply
repeat the entire format specifier, writing two integers on each line,
except for the last line, which only gets one integer:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>1 2
3 4
5
</code></pre>
</div>
<p>You can repeat things by preceding a format specifier with a number,
and you can order the end of a record with a slash:</p>
<figure class="highlight"><pre><code class="language-fortran" data-lang="fortran"><span class="w"> </span><span class="k">WRITE</span><span class="w"> </span><span class="p">(</span><span class="o">*</span><span class="p">,</span><span class="w"> </span><span class="mi">10</span><span class="p">)</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mi">2</span><span class="p">,</span><span class="w"> </span><span class="mi">3</span><span class="p">,</span><span class="w"> </span><span class="mi">4</span><span class="p">,</span><span class="w"> </span><span class="mi">5</span><span class="p">,</span><span class="w"> </span><span class="mi">6</span><span class="w">
</span><span class="nl">10</span><span class="w"> </span><span class="k">FORMAT</span><span class="w"> </span><span class="p">(</span><span class="l">2I2</span><span class="w"> </span><span class="l">/</span><span class="w"> </span><span class="l">I2</span><span class="p">)</span></code></pre></figure>
<p>which writes:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>1 2
3
4 5
6
</code></pre>
</div>
<p>Oh, and did I mention the famous “implied do loop?” Your I/O
statement can have a loop built in. This is often used to write
arrays:</p>
<figure class="highlight"><pre><code class="language-fortran" data-lang="fortran"><span class="w"> </span><span class="k">WRITE</span><span class="w"> </span><span class="p">(</span><span class="mi">6</span><span class="p">,</span><span class="w"> </span><span class="mi">10</span><span class="p">)</span><span class="w"> </span><span class="p">((</span><span class="n">A</span><span class="p">(</span><span class="n">I</span><span class="p">,</span><span class="n">J</span><span class="p">),</span><span class="w"> </span><span class="n">J</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span><span class="mi">10</span><span class="p">),</span><span class="w"> </span><span class="n">I</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span><span class="mi">10</span><span class="p">,</span><span class="mi">2</span><span class="p">)</span><span class="w">
</span><span class="mi">10</span><span class="w"> </span><span class="k">FORMAT</span><span class="p">(</span><span class="l">1X</span><span class="p">,</span><span class="w"> </span><span class="l">10F10.4</span><span class="p">)</span></code></pre></figure>
<p>This outputs the elements of an array in a specific order. In Ruby,
it would look something like this:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="p">(</span><span class="mi">1</span><span class="p">.</span><span class="nf">.</span><span class="mi">10</span><span class="p">).</span><span class="nf">step</span><span class="p">(</span><span class="mi">2</span><span class="p">).</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">i</span><span class="o">|</span>
<span class="p">(</span><span class="mi">1</span><span class="p">.</span><span class="nf">.</span><span class="mi">10</span><span class="p">).</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">j</span><span class="o">|</span>
<span class="nb">print</span> <span class="s2">"10.4f"</span> <span class="o">%</span> <span class="n">a</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="n">j</span><span class="p">]</span>
<span class="k">end</span>
<span class="nb">puts</span>
<span class="k">end</span></code></pre></figure>
<p>That’s just the beginning. So how do we go about taming it?</p>
<h1 id="the-problem">The problem</h1>
<p>The problem is that Formatted I/O is a dance between two entities: The
list of values specified in the WRITE statement, and the FORMAT
statement that tells how the elements are formatted. Through all of
this, it’s actually the format statement that is in control. It pulls
data from the write statement, essentially.</p>
<p>Naively, the write statement could gather up all the values in the I/O
list (the list of values to format) into a single array and pass them
to the format statement, something like this:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="n">values</span> <span class="o">=</span> <span class="n">io_list</span><span class="p">.</span><span class="nf">flat_map</span><span class="p">(</span><span class="o">&</span><span class="ss">:values</span><span class="p">)</span>
<span class="nb">format</span><span class="p">.</span><span class="nf">apply_values</span> <span class="n">values</span></code></pre></figure>
<p>The format statement could then shift values off of the <code class="highlighter-rouge">values</code> array
as needed. This would work, but it’s not optimal: an I/O list can be
big: A single name such as “A” can represent an immense array. We
don’t want to gather up all those values into a big array, and we
don’t need to.</p>
<h1 id="the-solution">The solution</h1>
<p>Instead, what’s wanted is to use external enumerators:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="n">enums</span> <span class="o">=</span> <span class="n">io_list</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span><span class="ss">:to_enum</span><span class="p">)</span>
<span class="nb">format</span><span class="p">.</span><span class="nf">apply_values</span> <span class="n">enums</span></code></pre></figure>
<p>Instead of passing values to the format statement, we pass enumerators
which, when iterated over, return values. The format statement then
pulls values off of the enumerators, one by one, until there are no
values left.</p>
<p>Scalars such as integers and floats (sorry, “reals”) have enumerators
that only yield a single value. Fortran arrays have enumerators that
yield every value in the array. Implied do’s also have enumerators
that yield multiple values.</p>
<p>It’s a good plan, but asking the format statement to deal with
multiple enumerators seems awkward. In order to get the next value,
it will have to get one from the next enumerator, and if that fails,
move on to the next enumerator. It’s not a <em>huge</em> burden, but those
little annoyances add up. So what if we could get multiple
enumerators from the I/O list, but then turn them into a single
enumerator to pass to the format statement?:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="n">enums</span> <span class="o">=</span> <span class="n">io_list</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span><span class="ss">:to_enum</span><span class="p">)</span>
<span class="n">enum</span> <span class="o">=</span> <span class="no">ConcatenatingEnumerator</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">enums</span><span class="p">)</span>
<span class="nb">format</span><span class="p">.</span><span class="nf">apply_values</span> <span class="n">enum</span></code></pre></figure>
<p>Now all we need is a ConcatenatingEnumerator. That, it turns out, is
very simple:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">ConcatenatingEnumerator</span> <span class="o"><</span> <span class="no">Enumerator</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">enumerators</span> <span class="o">=</span> <span class="p">[])</span>
<span class="k">super</span><span class="p">()</span> <span class="k">do</span> <span class="o">|</span><span class="n">yielder</span><span class="o">|</span>
<span class="n">enumerators</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">enumerator</span><span class="o">|</span>
<span class="n">enumerator</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">item</span><span class="o">|</span>
<span class="n">yielder</span><span class="p">.</span><span class="nf">yield</span> <span class="n">item</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>and a simple demonstration of its use:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="n">enum1</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">].</span><span class="nf">to_enum</span>
<span class="n">enum2</span> <span class="o">=</span> <span class="p">[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">].</span><span class="nf">to_enum</span>
<span class="n">enum</span> <span class="o">=</span> <span class="no">ConcatenatingEnumerator</span><span class="p">.</span><span class="nf">new</span><span class="p">([</span><span class="n">enum1</span><span class="p">,</span> <span class="n">enum2</span><span class="p">])</span>
<span class="nb">p</span> <span class="n">enum</span><span class="p">.</span><span class="nf">to_a</span> <span class="c1">#=> [1, 2, 3, 4, 5]</span></code></pre></figure>
<p><strong>2015-03-11</strong> - In <a href="/20150310/more-concatenating-enumerator.html">More Concatenating Enumerator</a>, there’s a minor
enhancement to ConcatenatingEnumerator, and another example of how it
is used.</p>
<p><strong>2015-07-23</strong>: Someone on the Ruby forum <a href="https://www.ruby-forum.com/topic/1965489">asked if enumerators could
be joined</a>. An <a href="https://redmine.ruby-lang.org/issues/709">issue was created asking for the feature</a>, but
was rejected.</p>
<hr />
<p><strong>Footnotes</strong></p>
<p><sup>1</sup> The designers of Fortran had not yet caught on that floating point
numbers are a pretty miserable subset of real numbers.</p>
<p><sup>2</sup> Yes, the data type we now call “string,” Fortran called
“character.” That makes naming some of the variables in a Fortran
interpreter <em>really</em> awkward.</p>
Fortran, take IV2014-06-01T22:00:00+00:00http://wconrad.github.com/20140601/fortran-take-4<p>I’ve failed to make a Fortran interpreter, written in Ruby, three
times now.</p>
<p><em>Take 1</em> used the Treetop parser. It’s a great parser, but I could
not figure out how to generate a parse tree. So I went looking for
another <a href="http://en.wikipedia.org/wiki/Parsing_expression_grammar">PEG</a>
for Ruby, and found Parslet. So, on to:</p>
<p><em>Take 2</em>, using the Parslet parser. This parser seems much nicer than
Treetop, but again, the generation of the parse tree was too
difficult. I just wasn’t “getting it.” I decided to take a break
from Fortran and write a Basic interpreter, in order to learn Parslet
using a simpler language. Having finished that, I went back to
Fortran with renewed confidence. So, on to:</p>
<p><em>Take 3</em>, against using the Parslet parser. At which I failed, again,
when it came to generating the parse tree. I just don’t seem to be
able to wrap my head around the mechanics of building a parse tree in
Parslet. Among other things, left associative operators are a pain in
Parslet (and, if I had to guess, in any PEG). I pretty much gave up
on Fortran after take 3.</p>
<p>Then, I decided to write tutorial on how <a href="http://en.wikipedia.org/wiki/Recursive_descent_parser">recursive descent
compilers</a> can
be constructed in Ruby. A recursive descent compiler is insanely easy
to write. There are a few reasons they’re not used much for serious
work; the most important being that a recursive descent compiler is
only efficent and easy when used to parse a context free grammar.
Fortran, for example, is not context free.</p>
<p>Partway through that tutorial I realized that it would take only a few
lines of code to allow arbitrary backtracking. With backtracking, a
recursive descent compiler can parse languages such as Fortran. So,
with the tutorial still unpublished, I’m on to:</p>
<p><em>Take 4</em>, using a
hand-rolled recursive descent compiler with backtracking.</p>
<p>Finally, I’m getting some traction. For the first time, I’ve got an
interpreter that can execute some simple Fortran statements, such as:</p>
<figure class="highlight"><pre><code class="language-fortran" data-lang="fortran"><span class="w"> </span><span class="n">A</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="mf">1.0</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="mi">2</span><span class="p">)</span><span class="w"> </span><span class="p">/</span><span class="w"> </span><span class="mi">3</span><span class="w"> </span><span class="o">**</span><span class="w"> </span><span class="p">(</span><span class="mi">-4</span><span class="p">)</span><span class="w">
</span><span class="k">PRINT</span><span class="w"> </span><span class="o">*</span><span class="p">,</span><span class="w"> </span><span class="s1">'A ='</span><span class="p">,</span><span class="w"> </span><span class="n">A</span></code></pre></figure>
<p>The interpreter has:</p>
<ul>
<li>PRINT statement with list directed output</li>
<li>INTEGER constants</li>
<li>REAL constants</li>
<li>LOGICAL constants</li>
<li>Simple variables</li>
<li>Math operators (<code class="highlighter-rouge">+</code>, <code class="highlighter-rouge">*</code>, etc.)</li>
<li>Logical operators (<code class="highlighter-rouge">.AND.</code>, <code class="highlighter-rouge">.OR.</code>, etc.)</li>
<li>Comparison operators (<code class="highlighter-rouge">.LE.</code>, <code class="highlighter-rouge">.EQ.</code>, etc.)</li>
<li>Implicit type casting</li>
</ul>
<p>It’s not a lot, and much of it is not spec compliant, or is
incomplete, but it’s a good start. The framework for the type system
has been laid down, and it look much cleaner than it did in basic101.
The lesson from basic101 is to keep a very clear distinction between
Ruby types such as Integer, Float and String, and language types such
as INTEGER, REAL, and CHARACTER. The two type systems should only be
allowed to interact at clearly defined boundaries. I sort of stumbled
into that in basic101, so it’s a bit messy there, but it’s clean here.</p>
<p>It remains to be seen whether the recursive descent compiler with
backtracking is going to fly–Wikipedia warns that there could be
trouble ahead, such as possibly requiring exponential time. But,
forging ahead…</p>
Two ways to decorate a model in Ruby2014-05-15T22:00:00+00:00http://wconrad.github.com/20140515/decorator<p>In applications using the MVC (Model View Controller) architecture, it
is good practice to:</p>
<ul>
<li>Keep the business logic in the model</li>
<li>Keep the controller thin</li>
<li>Keep logic out of the view</li>
</ul>
<p>I’ve had few problems with keeping all of the business logic in the
model. Keeping the controller thin can be more work, but it’s not
usually too difficult. Keeping logic out of the view, however, has
been more of a problem. The two easiest ways to get the logic out of
the view are to move it into either a “helper function,” or into the
model itself. I’d like to explain the problems with these approaches,
and introduce the “decorator” pattern as an alternate solution. I
will show how the decorate pattern is easily applied in Ruby through
the use of dynamically included modules.</p>
<h1 id="lets-begin">Let’s Begin</h1>
<p>Our example application will be a simple schedule with a list of
events. It will make a schedule with some events, and then display
it.</p>
<p>We’ll need some libraries:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="nb">require</span> <span class="s1">'date'</span>
<span class="nb">require</span> <span class="s1">'delegate'</span></code></pre></figure>
<p>The models couldn’t be simpler.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="no">Event</span> <span class="o">=</span> <span class="no">Struct</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">:name</span><span class="p">,</span> <span class="ss">:date</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">Schedule</span>
<span class="kp">attr_reader</span> <span class="ss">:events</span>
<span class="k">def</span> <span class="nf">initialize</span>
<span class="vi">@events</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>The view is just a function that prints the schedule to the console:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">show_schedule</span><span class="p">(</span><span class="n">schedule</span><span class="p">)</span>
<span class="k">case</span> <span class="n">schedule</span><span class="p">.</span><span class="nf">events</span><span class="p">.</span><span class="nf">size</span>
<span class="k">when</span> <span class="mi">0</span>
<span class="nb">puts</span> <span class="s2">"Nothing to do. Perfect!"</span>
<span class="k">when</span> <span class="mi">1</span>
<span class="nb">puts</span> <span class="s2">"Something to do. Sigh."</span>
<span class="k">else</span>
<span class="nb">puts</span> <span class="s2">"Much too much to do!"</span>
<span class="k">end</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">events</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">event</span><span class="o">|</span>
<span class="nb">puts</span> <span class="s2">"</span><span class="si">#{</span><span class="n">event</span><span class="p">.</span><span class="nf">date</span><span class="p">.</span><span class="nf">strftime</span><span class="p">(</span><span class="s2">"%D"</span><span class="p">)</span><span class="si">}</span><span class="s2"> - </span><span class="si">#{</span><span class="n">event</span><span class="p">.</span><span class="nf">name</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>This function is used by the controller to make a populated instance
of the Schedule model:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">make_schedule</span>
<span class="n">schedule</span> <span class="o">=</span> <span class="no">Schedule</span><span class="p">.</span><span class="nf">new</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">events</span> <span class="o"><<</span> <span class="no">Event</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'Mow the lawn'</span><span class="p">,</span> <span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2015</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">events</span> <span class="o"><<</span> <span class="no">Event</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'Stop watering the lawn'</span><span class="p">,</span> <span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2015</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">events</span> <span class="o"><<</span> <span class="no">Event</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'Rake up dead lawn'</span><span class="p">,</span> <span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2015</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span>
<span class="n">schedule</span>
<span class="k">end</span></code></pre></figure>
<p>And this code, the controller proper, ties it all together.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="n">schedule</span> <span class="o">=</span> <span class="n">make_schedule</span>
<span class="n">show_schedule</span> <span class="n">schedule</span></code></pre></figure>
<p>And its outout:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>Much too much to do!
01/01/15 - Mow the lawn
02/01/15 - Stop watering the lawn
06/01/15 - Rake up dead lawn
</code></pre>
</div>
<p>It works, but can it be better? The logic in the view is distracting.
It would be better if the view did not have to concern itself with
details such as how to format the date, or how many tasks is “Much too
much to do!”. If we can move those logics out of the view, the view
becomes simpler. Also, there may be other views of the same models
that wish to reuse those logics. Let’s explore some different ways to
achieve this.</p>
<h1 id="helper-methods">Helper methods</h1>
<p>Let’s try moving the view logics into helper functions. This is an
approach commonly used in Ruby on Rails applications (although rails
calls them “helper methods,” not “helper functions”).</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">Bar</span>
<span class="k">def</span> <span class="nf">foo</span>
<span class="vi">@foo</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>Here are our helper functions:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">format_event_date</span><span class="p">(</span><span class="n">event</span><span class="p">)</span>
<span class="n">event</span><span class="p">.</span><span class="nf">date</span><span class="p">.</span><span class="nf">strftime</span><span class="p">(</span><span class="s2">"%D %R"</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">schedule_burden</span><span class="p">(</span><span class="n">schedule</span><span class="p">)</span>
<span class="k">case</span> <span class="n">schedule</span><span class="p">.</span><span class="nf">events</span><span class="p">.</span><span class="nf">size</span>
<span class="k">when</span> <span class="mi">0</span>
<span class="s2">"Nothing to do. Perfect!"</span>
<span class="k">when</span> <span class="mi">1</span>
<span class="s2">"Something to do. Sigh."</span>
<span class="k">else</span>
<span class="s2">"Much too much to do!"</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>and here is the view that uses them:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">show_schedule</span><span class="p">(</span><span class="n">schedule</span><span class="p">)</span>
<span class="nb">puts</span> <span class="n">schedule_burden</span><span class="p">(</span><span class="n">schedule</span><span class="p">)</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">events</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">event</span><span class="o">|</span>
<span class="nb">puts</span> <span class="s2">"</span><span class="si">#{</span><span class="n">format_event_date</span><span class="p">(</span><span class="n">event</span><span class="p">)</span><span class="si">}</span><span class="s2"> - </span><span class="si">#{</span><span class="n">event</span><span class="p">.</span><span class="nf">name</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>This isn’t bad. The view is certainly cleaner now. But helper
functions have a drawback: Being functions rathern than methods, the
helpers need to have all of the state they will act upon passed into
them. That’s the difference between a function and a method. A
method acts upon an object’s state; a function acts upon its
arguments. Being functions, they are disconnected from the models
they act upon. One of the principles of OOP is that data, and the
code that act upon that data, live together in an object. Now we have
some code just “hanging out there.” An object’s methods are
discoverable:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="nb">p</span> <span class="n">o</span><span class="p">.</span><span class="nf">methods</span></code></pre></figure>
<p>But the functions are unattached to the object. You just have to know
they are there.</p>
<h1 id="adding-view-helpers-to-the-models">Adding view helpers to the models</h1>
<p>One way to avoid the problem of disconnected helper functions is to
move the functions into the models they act upon. Our models become:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="no">Event</span> <span class="o">=</span> <span class="no">Struct</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">:name</span><span class="p">,</span> <span class="ss">:date</span><span class="p">)</span> <span class="k">do</span>
<span class="k">def</span> <span class="nf">format_date</span>
<span class="n">date</span><span class="p">.</span><span class="nf">strftime</span><span class="p">(</span><span class="s2">"%D %R"</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">class</span> <span class="nc">Schedule</span>
<span class="kp">attr_reader</span> <span class="ss">:events</span>
<span class="k">def</span> <span class="nf">initialize</span>
<span class="vi">@events</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">burden</span>
<span class="k">case</span> <span class="vi">@events</span><span class="p">.</span><span class="nf">size</span>
<span class="k">when</span> <span class="mi">0</span>
<span class="s2">"Nothing to do. Perfect!"</span>
<span class="k">when</span> <span class="mi">1</span>
<span class="s2">"Something to do. Sigh."</span>
<span class="k">else</span>
<span class="s2">"Much too much to do!"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>and the view:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">show_schedule</span><span class="p">(</span><span class="n">schedule</span><span class="p">)</span>
<span class="nb">puts</span> <span class="n">schedule</span><span class="p">.</span><span class="nf">burden</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">events</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">event</span><span class="o">|</span>
<span class="nb">puts</span> <span class="s2">"</span><span class="si">#{</span><span class="n">event</span><span class="p">.</span><span class="nf">format_date</span><span class="si">}</span><span class="s2"> - </span><span class="si">#{</span><span class="n">event</span><span class="p">.</span><span class="nf">name</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>This is just about ideal for the view. It doesn’t get any easier than
that. Things aren’t as good in the model. The model’s business
logical is now mixed in with the view’s logic. It’s not so bad with
one view, but what if there were several views, each needing their own
special formatting and other logic? The model ends up becoming a
Caddisfly larvae, covered with little pebbles and twigs that are
useful here and there, but a bit of a mess in the whole. What we want
is the view we just made here, but without having to add those methods
to the model. The decorator pattern can do that.</p>
<h1 id="decorator-using-simpledelegator">Decorator using SimpleDelegator</h1>
<p>What if we could decorate an instance of a model, adding to it the
methods that the view needs? With Ruby’s SimpleDelegator, we can do
it.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">DecoratedSchedule</span> <span class="o"><</span> <span class="no">SimpleDelegator</span>
<span class="k">def</span> <span class="nf">burden</span>
<span class="k">case</span> <span class="n">events</span><span class="p">.</span><span class="nf">size</span>
<span class="k">when</span> <span class="mi">0</span>
<span class="s2">"Nothing to do. Perfect!"</span>
<span class="k">when</span> <span class="mi">1</span>
<span class="s2">"Something to do. Sigh."</span>
<span class="k">else</span>
<span class="s2">"Much too much to do!"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>A SimpleDelegator has an #initialize method that takes a single
argument, the delegatee, or class being delegated to. In this case,
it will be an instance of Schedule. The delegator forwards to the
delegatee any methods it doesn’t know about. It acts as though it is
the delegatee, but with some extra methods. That’s exactly what we
need!</p>
<p>A little change to the controller decorates the schedule before
showing it:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="n">schedule</span> <span class="o">=</span> <span class="n">make_schedule</span>
<span class="n">schedule</span> <span class="o">=</span> <span class="no">DecoratedSchedule</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">schedule</span><span class="p">)</span>
<span class="n">show_schedule</span> <span class="n">schedule</span></code></pre></figure>
<p>We’ll want to decorate the event, too. Unfortunately, there’s not an
easy way for us to do this outside the view. We could do it where we
make the schedule:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">make_schedule</span>
<span class="n">schedule</span> <span class="o">=</span> <span class="no">Schedule</span><span class="p">.</span><span class="nf">new</span>
<span class="n">events</span> <span class="o">=</span> <span class="p">[</span>
<span class="no">Event</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'Mow the lawn'</span><span class="p">,</span> <span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2015</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)),</span>
<span class="no">Event</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'Stop watering the lawn'</span><span class="p">,</span> <span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2015</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">)),</span>
<span class="no">Event</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'Rake up dead lawn'</span><span class="p">,</span> <span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2015</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span>
<span class="p">]</span>
<span class="n">events</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">event</span><span class="o">|</span>
<span class="n">event</span> <span class="o">=</span> <span class="no">DecoratedEvent</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">event</span><span class="p">)</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">events</span> <span class="o"><<</span> <span class="n">event</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>But this is cumbersome. And what if the code that makes the schedules
is used by other views? We don’t want to decorate <em>all</em> events, we
just want to decorate the events used by this view.</p>
<p>We could also have the view do the decorating:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">show_schedule</span><span class="p">(</span><span class="n">schedule</span><span class="p">)</span>
<span class="nb">puts</span> <span class="n">schedule</span><span class="p">.</span><span class="nf">burden</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">events</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">event</span><span class="o">|</span>
<span class="n">event</span> <span class="o">=</span> <span class="no">DecoratedEvent</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">event</span><span class="p">)</span>
<span class="nb">puts</span> <span class="s2">"</span><span class="si">#{</span><span class="n">event</span><span class="p">.</span><span class="nf">format_date</span><span class="si">}</span><span class="s2"> - </span><span class="si">#{</span><span class="n">event</span><span class="p">.</span><span class="nf">name</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>but we’re trying to make views <em>simpler</em>, not more complex. If we
only needed to decorate a single, outer object, using delegators would
be alright. But, in this case, our models are nested: A Schedule has
Tasks. We want to decorate both the schedule and its tasks. Which
leads us to:</p>
<h1 id="mixin-decorations">Mixin decorations</h1>
<p>What if our helper methods were in modules?</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">module</span> <span class="nn">ScheduleDecoration</span>
<span class="k">def</span> <span class="nf">burden</span>
<span class="k">case</span> <span class="vi">@events</span><span class="p">.</span><span class="nf">size</span>
<span class="k">when</span> <span class="mi">0</span>
<span class="s2">"Nothing to do. Perfect!"</span>
<span class="k">when</span> <span class="mi">1</span>
<span class="s2">"Something to do. Sigh."</span>
<span class="k">else</span>
<span class="s2">"Much too much to do!"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">module</span> <span class="nn">EventDecoration</span>
<span class="k">def</span> <span class="nf">format_date</span>
<span class="n">date</span><span class="p">.</span><span class="nf">strftime</span><span class="p">(</span><span class="s2">"%D %R"</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<h1 id="static-mixins">Static mixins</h1>
<p>The usual way people see modules used is at the class level, like this:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">Schedule</span>
<span class="kp">include</span> <span class="no">ScheduleDecoration</span>
<span class="p">.</span><span class="nf">.</span><span class="p">.</span>
<span class="nf">end</span></code></pre></figure>
<p>This is a little better than just including the method directly into
the model: at least the module name serves to group methods according
to their purpose. It’s not ideal, though, since every instance of
that model has the decorations. What happens when one view needs the
date formatted differently? You end up having this:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">module</span> <span class="nn">EventDecoration</span>
<span class="k">def</span> <span class="nf">format_date_for_show</span>
<span class="p">.</span><span class="nf">.</span><span class="p">.</span>
<span class="nf">end</span>
<span class="k">end</span>
<span class="k">module</span> <span class="nn">OtherEventDecoration</span>
<span class="k">def</span> <span class="nf">format_date_for_other_purpose</span>
<span class="p">.</span><span class="nf">.</span><span class="p">.</span>
<span class="nf">end</span>
<span class="k">end</span></code></pre></figure>
<p>This code is <em>not</em> getting better. But what if you could mix modules
into individual objects, so that the model gets decorated by the view
as needed?</p>
<h1 id="dynamic-mixins">Dynamic mixins</h1>
<p>Let’s have the controller mix in the decorations at runtime, dynamically:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">decorate_schedule</span><span class="p">(</span><span class="n">schedule</span><span class="p">)</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">extend</span> <span class="no">ScheduleDecoration</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">events</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">event</span><span class="o">|</span>
<span class="n">event</span><span class="p">.</span><span class="nf">extend</span> <span class="no">EventDecoration</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">schedule</span> <span class="o">=</span> <span class="n">make_schedule</span>
<span class="n">decorate_schedule</span> <span class="n">schedule</span>
<span class="n">show_schedule</span> <span class="n">schedule</span></code></pre></figure>
<p>calling extend on an object and passing it a module, as here:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="n">schedule</span><span class="p">.</span><span class="nf">extend</span> <span class="no">ScheduleDecoration</span></code></pre></figure>
<p>is almost the same as if the class included the module:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">class</span> <span class="nc">Schedule</span>
<span class="kp">include</span> <span class="no">ScheduleDecoration</span>
<span class="p">.</span><span class="nf">.</span><span class="p">.</span>
<span class="nf">end</span></code></pre></figure>
<p><em>except</em> that the module’s methods are only included in that one
instance of Schedule rather than in every instance. This is just what
we were looking for. Now our controller can add the extra methods
that will make the view simple and clean, without polluting the model
class.</p>
<p>There’s one more refinement we can make here. The “decorate_schedule”
method is kind of annoying. It’s detail that our controller should
not have to worry about. Also, if more than one controller needs the
same decoration, they would have to share the <code class="highlighter-rouge">decorate_schedule</code>
method. That’s not that bad, but let’s see if we can tidy things up.</p>
<h1 id="cleaning-things-up-with-moduleextended">Cleaning things up with Module.extended</h1>
<p>You can make a module do things when it is extended. Let’s take the
loop out of <code class="highlighter-rouge">decorate_schedule</code> and move it into ScheduleDecoration:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">module</span> <span class="nn">ScheduleDecoration</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">extended</span><span class="p">(</span><span class="n">schedule</span><span class="p">)</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">events</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">event</span><span class="o">|</span>
<span class="n">event</span><span class="p">.</span><span class="nf">extend</span> <span class="no">EventDecoration</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">burden</span>
<span class="k">case</span> <span class="vi">@events</span><span class="p">.</span><span class="nf">size</span>
<span class="k">when</span> <span class="mi">0</span>
<span class="s2">"Nothing to do. Perfect!"</span>
<span class="k">when</span> <span class="mi">1</span>
<span class="s2">"Something to do. Sigh."</span>
<span class="k">else</span>
<span class="s2">"Much too much to do!"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span></code></pre></figure>
<p>Now the controller can simply decorate the schedule and the schedule’s
tasks will also be decorated:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="n">schedule</span> <span class="o">=</span> <span class="n">make_schedule</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">extend</span> <span class="no">ScheduleDecoration</span>
<span class="n">show_schedule</span> <span class="n">schedule</span></code></pre></figure>
<h1 id="the-final-result">The final result</h1>
<p>We’ve made a lot of changes along the way. Here’s what the code looks
like now:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="nb">require</span> <span class="s1">'date'</span>
<span class="nb">require</span> <span class="s1">'delegate'</span>
<span class="no">Event</span> <span class="o">=</span> <span class="no">Struct</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">:name</span><span class="p">,</span> <span class="ss">:date</span><span class="p">);</span>
<span class="k">class</span> <span class="nc">Schedule</span>
<span class="kp">attr_reader</span> <span class="ss">:events</span>
<span class="k">def</span> <span class="nf">initialize</span>
<span class="vi">@events</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">module</span> <span class="nn">ScheduleDecoration</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">extended</span><span class="p">(</span><span class="n">schedule</span><span class="p">)</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">events</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">event</span><span class="o">|</span>
<span class="n">event</span><span class="p">.</span><span class="nf">extend</span> <span class="no">EventDecoration</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">burden</span>
<span class="k">case</span> <span class="vi">@events</span><span class="p">.</span><span class="nf">size</span>
<span class="k">when</span> <span class="mi">0</span>
<span class="s2">"Nothing to do. Perfect!"</span>
<span class="k">when</span> <span class="mi">1</span>
<span class="s2">"Something to do. Sigh."</span>
<span class="k">else</span>
<span class="s2">"Much too much to do!"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">module</span> <span class="nn">EventDecoration</span>
<span class="k">def</span> <span class="nf">format_date</span>
<span class="n">date</span><span class="p">.</span><span class="nf">strftime</span><span class="p">(</span><span class="s2">"%D %R"</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">make_schedule</span>
<span class="n">schedule</span> <span class="o">=</span> <span class="no">Schedule</span><span class="p">.</span><span class="nf">new</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">events</span> <span class="o"><<</span> <span class="no">Event</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'Mow the lawn'</span><span class="p">,</span> <span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2015</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">events</span> <span class="o"><<</span> <span class="no">Event</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'Stop watering the lawn'</span><span class="p">,</span> <span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2015</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">events</span> <span class="o"><<</span> <span class="no">Event</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'Rake up dead lawn'</span><span class="p">,</span> <span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2015</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span>
<span class="n">schedule</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">show_schedule</span><span class="p">(</span><span class="n">schedule</span><span class="p">)</span>
<span class="nb">puts</span> <span class="n">schedule</span><span class="p">.</span><span class="nf">burden</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">events</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">event</span><span class="o">|</span>
<span class="nb">puts</span> <span class="s2">"</span><span class="si">#{</span><span class="n">event</span><span class="p">.</span><span class="nf">format_date</span><span class="si">}</span><span class="s2"> - </span><span class="si">#{</span><span class="n">event</span><span class="p">.</span><span class="nf">name</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">schedule</span> <span class="o">=</span> <span class="n">make_schedule</span>
<span class="n">schedule</span><span class="p">.</span><span class="nf">extend</span> <span class="no">ScheduleDecoration</span>
<span class="n">show_schedule</span> <span class="n">schedule</span></code></pre></figure>
<p>and its output:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>Much too much to do!
01/01/15 00:00 - Mow the lawn
02/01/15 00:00 - Stop watering the lawn
06/01/15 00:00 - Rake up dead lawn
</code></pre>
</div>
rspec-given: Specs that read like Cucumber2014-05-09T13:00:00+00:00http://wconrad.github.com/20140509/rspec-given<p>I followed a chain of blog entries and comments where people were
discussing the merits of <a href="http://en.wikipedia.org/wiki/Test-driven_development">TDD</a>; those lead to the essay
<a href="http://david.heinemeierhansson.com/2014/test-induced-design-damage.html">Test-induced design damage</a> by DHH of Rails fame. It’s a good
essay. And just to be clear, by “good” I don’t necessarily mean “I
agree with it.” What I mean is that it makes interesting points that
will cause your neurons to fire. At least, it did mine. DHH is a
smart guy, and for heaven’s sake, he <em>invented Rails</em>, so when he has
something to say about how Rails is best tested, it’s worth reading.</p>
<p>DHH argues that slavish application of TDD can cause twisted designs,
code that is less clear than it would be otherwise. I know that’s
true <em>for me</em> – I’ve created less readable code in the name of making
it testable. What I don’t know is how much that’s TDD’s fault, and
how much is just me not seeing a better way to to it.</p>
<p>As an example of code distorting the design, DHH links to <a href="https://www.youtube.com/watch?v=tg5RFeSfBM4">a video by
the late Jim Weirich</a> (you know, the Rake guy), in which Jim
demonstrates the application of <a href="http://alistair.cockburn.us/Hexagonal+architecture">hexagonal architecture</a> to Rails.
It’s a good video, especially in conjunction with DHH’s criticism.
And yes, “good” here means the same thing as it did above.</p>
<p>In the course of the demonstration, Jim showed how he writes rspec
tests (sorry, “specifications”). I saw a wonderful style of rspec I
hadn’t seen before. It looks like this:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="n">describe</span> <span class="no">Stack</span> <span class="k">do</span>
<span class="no">Given</span><span class="p">(</span><span class="ss">:stack</span><span class="p">)</span> <span class="p">{</span> <span class="no">Stack</span><span class="p">.</span><span class="nf">new</span> <span class="p">}</span>
<span class="no">Given</span><span class="p">(</span><span class="ss">:initial_contents</span><span class="p">)</span> <span class="p">{</span> <span class="p">[]</span> <span class="p">}</span>
<span class="no">Given</span> <span class="p">{</span> <span class="n">initial_contents</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">item</span><span class="o">|</span> <span class="n">stack</span><span class="p">.</span><span class="nf">push</span><span class="p">(</span><span class="n">item</span><span class="p">)</span> <span class="k">end</span> <span class="p">}</span>
<span class="no">Invariant</span> <span class="p">{</span> <span class="n">stack</span><span class="p">.</span><span class="nf">empty?</span> <span class="o">==</span> <span class="p">(</span><span class="n">stack</span><span class="p">.</span><span class="nf">depth</span> <span class="o">==</span> <span class="mi">0</span><span class="p">)</span> <span class="p">}</span>
<span class="n">context</span> <span class="s2">"with an empty stack"</span> <span class="k">do</span>
<span class="no">Given</span><span class="p">(</span><span class="ss">:initial_contents</span><span class="p">)</span> <span class="p">{</span> <span class="p">[]</span> <span class="p">}</span>
<span class="no">Then</span> <span class="p">{</span> <span class="n">stack</span><span class="p">.</span><span class="nf">depth</span> <span class="o">==</span> <span class="mi">0</span> <span class="p">}</span>
<span class="n">context</span> <span class="s2">"when pushing"</span> <span class="k">do</span>
<span class="no">When</span> <span class="p">{</span> <span class="n">stack</span><span class="p">.</span><span class="nf">push</span><span class="p">(</span><span class="ss">:an_item</span><span class="p">)</span> <span class="p">}</span>
<span class="no">Then</span> <span class="p">{</span> <span class="n">stack</span><span class="p">.</span><span class="nf">depth</span> <span class="o">==</span> <span class="mi">1</span> <span class="p">}</span>
<span class="no">Then</span> <span class="p">{</span> <span class="n">stack</span><span class="p">.</span><span class="nf">top</span> <span class="o">==</span> <span class="ss">:an_item</span> <span class="p">}</span>
<span class="k">end</span>
<span class="n">context</span> <span class="s2">"when popping"</span> <span class="k">do</span>
<span class="no">When</span><span class="p">(</span><span class="ss">:result</span><span class="p">)</span> <span class="p">{</span> <span class="n">stack</span><span class="p">.</span><span class="nf">pop</span> <span class="p">}</span>
<span class="no">Then</span> <span class="p">{</span> <span class="n">result</span> <span class="o">==</span> <span class="no">Failure</span><span class="p">(</span><span class="no">Stack</span><span class="o">::</span><span class="no">UnderflowError</span><span class="p">,</span> <span class="sr">/empty/</span><span class="p">)</span> <span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="p">.</span><span class="nf">.</span><span class="p">.</span>
<span class="nf">end</span>
</code></pre></figure>
<p>This is an excerpt from <a href="https://github.com/jimweirich/rspec-given/blob/2fd1771f25deaaf9cb58e619ff80bfdb3ddaabe0/examples/stack/stack_spec.rb">stack_spec.rb</a>, which is part of Jim’s
<a href="https://github.com/jimweirich/rspec-given">rspec-given</a> gem. If you’ve ever worked with Cucumber, you’ll
recognize this style. Here’s a piece of a cucumber test so you can
see how similar they are:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>Background:
Given the test server is started
And a successful connection
Scenario: No argument
When the client successfully asks for help
Then the server should return a list of commands
Scenario: Known command
When the client successfully asks for help for "NOOP"
Then the server should return help for "NOOP"
Scenario: Unknown command
When the client successfully asks for help for "FOO"
Then the server should return no help for "FOO"
</code></pre>
</div>
<p>At first, Jim’s “given” syntax looks like it’s just renaming some
methods: <code class="highlighter-rouge">let</code> becomes <code class="highlighter-rouge">Given</code>, <code class="highlighter-rouge">before(:each)</code> becomes <code class="highlighter-rouge">When</code>, and
<code class="highlighter-rouge">specify</code> or <code class="highlighter-rouge">it</code> becomes <code class="highlighter-rouge">Then</code>, but there’s more going on. One
thing I like is the addition of <code class="highlighter-rouge">And</code>: This is like a <code class="highlighter-rouge">Then</code>, but it
does not redo the setup. It inherits the state left by the previous
<code class="highlighter-rouge">Then</code> or <code class="highlighter-rouge">And</code>. That lets you do this (stolen from the gem’s
README):</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="no">Then</span> <span class="p">{</span> <span class="n">pop_result</span> <span class="o">==</span> <span class="ss">:top_item</span> <span class="p">}</span>
<span class="no">And</span> <span class="p">{</span> <span class="n">stack</span><span class="p">.</span><span class="nf">top</span> <span class="o">==</span> <span class="ss">:second_item</span> <span class="p">}</span>
<span class="no">And</span> <span class="p">{</span> <span class="n">stack</span><span class="p">.</span><span class="nf">depth</span> <span class="o">==</span> <span class="n">original_depth</span> <span class="o">-</span> <span class="mi">1</span> <span class="p">}</span>
</code></pre></figure>
<p>Without rspec-given, you would it like this:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="n">specify</span> <span class="k">do</span>
<span class="n">expect</span><span class="p">(</span><span class="n">pop_result</span><span class="p">).</span><span class="nf">to</span> <span class="n">eq</span> <span class="ss">:top_item</span>
<span class="n">expect</span><span class="p">(</span><span class="n">stack</span><span class="p">.</span><span class="nf">top</span><span class="p">).</span><span class="nf">to</span> <span class="n">eq</span> <span class="ss">:second_item</span>
<span class="n">expect</span><span class="p">(</span><span class="n">stack</span><span class="p">.</span><span class="nf">depth</span><span class="p">).</span><span class="nf">to</span> <span class="n">eq</span> <span class="n">original_depth</span> <span class="o">-</span> <span class="mi">1</span>
<span class="k">end</span>
</code></pre></figure>
<p>Now, this kind of test, where you repeatedly poke at an object and
check the results of the accumulated state changes, isn’t my favorite
way of writing tests. I usually prefer that each test checks just one
thing. But there are some tests that are easier to write and read in
this rolling style. Jim’s syntax is nice for those.</p>
<p>Jim’s syntax pleases me very much. Also, the <a href="https://github.com/jimweirich/rspec-given">README</a> is very well
written, with just the right level of detail. I’m going to give
rspec-given a try on my next project.</p>
But PostScript is harder to read than LISP2014-03-17T21:19:00+00:00http://wconrad.github.com/20140317/postscript-is-hard-to-read<p>There is one significant area where Postscript is worse than LISP. In
fact, it’s worse than any other language I’ve used in this regard:
It’s hard to read.</p>
<p>It all comes from Postscript’s minimal syntax and lack of a way to
syntactically group a function’s operands. If you are to understand
what the code does, you have to know the calling sequence of every
function you read. Let me explain.</p>
<p>Here’s some C code that calls a function:</p>
<figure class="highlight"><pre><code class="language-c" data-lang="c"> <span class="n">i</span> <span class="o">=</span> <span class="n">foo</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="mi">2</span><span class="p">);</span>
</code></pre></figure>
<p>What does <code class="highlighter-rouge">foo</code> do? We don’t know. But we know immediately that
<code class="highlighter-rouge">foo</code> takes three arguments and returns one value.</p>
<p>Here’s some Ruby code that calls a method:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="n">foo</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
</code></pre></figure>
<p>What does <code class="highlighter-rouge">foo</code> do? Again, we don’t know. But we know that foo takes
one argument and returns nothing we care about.</p>
<p>And now some LISP to call a function:</p>
<figure class="highlight"><pre><code class="language-lisp" data-lang="lisp"> (foo 1 2)
</code></pre></figure>
<p>Again, we know right away that foo takes two arguments.</p>
<p>Now, to make my point, let’s look at some postscript code:</p>
<figure class="highlight"><pre><code class="language-postscript" data-lang="postscript"> 1 2 3 4 foo
</code></pre></figure>
<p>How many arguments does foo take off of the operand stack? <em>We can’t
tell</em>. How many does it push back into the operand stack? <em>We can’t
tell that, either.</em> It might consume the 3 and the 4, add them, and
leave a 7 on the stack. It might consume no operands and leave
nothing on the stack. Without remembering what <code class="highlighter-rouge">foo</code> does, you have
no way to know what effect <code class="highlighter-rouge">foo</code> has upon the stack except by a lucky
guess. Unfortunately, keeping track of what’s on the operand stack is
important when reading postscript.</p>
<p>The author can help you keep track of the operand stack by writing
short functions so you don’t have to hold too much in your head at
once, and with the judicious use of comments. But that’s all work the
author has to do in order to give you something that most other
languages have out of the box.</p>
<p>Postscript is a strange language like that. It has some fairly
high-level constructs glued together with a syntax that is little more
than a stack-based assembly language.</p>
PostScript is LISP, sort of2014-03-13T19:45:00+00:00http://wconrad.github.com/20140313/postscript-is-lisp<p>I was going doing an exercise from <a href="http://wwwcdf.pd.infn.it/localdoc/tips.pdf">Thinking in
PostScript</a> and became
confused by this piece of code, from Chapter 5, exercise 3
(pp. 64-65):</p>
<figure class="highlight"><pre><code class="language-postscript" data-lang="postscript"> /ushow
{ %def
LOCAL begin
% ...
end
} dup 0 4 dict put def
</code></pre></figure>
<p>This code defines a function <code class="highlighter-rouge">ushow</code>, and uses a local dictionary of
some kind. But where is <code class="highlighter-rouge">LOCAL</code> defined? It’s not defined in the
code, and it isn’t defined in any Adobe documentation. And yet this
code ran in Ghostscript. I ended up <a href="http://stackoverflow.com/q/22385031/238886">asking on Stack
Overflow</a>. The answer I
got showed me what was really happening, and taught me something I
hadn’t quite realized: PostScript has some LISPish qualities to it.</p>
<p>A normal functioin definition in PostScript looks more like this:</p>
<figure class="highlight"><pre><code class="language-postscript" data-lang="postscript"> /ushow
{
} def
</code></pre></figure>
<p>(Function definitions often end with <code class="highlighter-rouge">bind def</code> instead of <code class="highlighter-rouge">def</code>, but
that doesn’t matter here). Notice the difference? Not only is that
<code class="highlighter-rouge">LOCAL begin...end</code> block missing from inside the function definition,
but the end doesn’t contain all of that <code class="highlighter-rouge">dup 0 4 dict put</code> stuff.
<em>That</em> it turns out, is the key to how <code class="highlighter-rouge">LOCAL</code> works here. To
understand what’s really happening, let’s evaluate the puzzling code
as the PostScript interpreter does. First:</p>
<figure class="highlight"><pre><code class="language-postscript" data-lang="postscript"> /ushow
</code></pre></figure>
<p>And the operand stack is now:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>/ushow
</code></pre>
</div>
<p>Then comes the start of the procedure:</p>
<figure class="highlight"><pre><code class="language-postscript" data-lang="postscript"> {
</code></pre></figure>
<p>This puts the PostScript in a special mode where it creates an
<em>executable array</em>. From now on until the <code class="highlighter-rouge">}</code> is parsed, tokens are
not executed, but are simply added to the executable array. That
means that nothing in the procedure needs to be defined until it
executed. <em>That includes <code class="highlighter-rouge">LOCAL</code></em>, so these lines:</p>
<figure class="highlight"><pre><code class="language-postscript" data-lang="postscript"> LOCAL begin
% ...
end
</code></pre></figure>
<p>Simply add the symbols <code class="highlighter-rouge">LOCAL</code>, <code class="highlighter-rouge">begin</code>, whatever code is elided by
<code class="highlighter-rouge">...</code>, and <code class="highlighter-rouge">end</code> to the executable array. None of it gets executed
until the function is actually called, so it doesn’t matter what it is
until then. It can be giberish, or a mix of giberish and regular
code. Do you see where this is going?</p>
<p>The end of the executuble array is signalled the <code class="highlighter-rouge">}</code> at the beginning
of the last line:</p>
<figure class="highlight"><pre><code class="language-postscript" data-lang="postscript"> } dup 0 4 dict put def
</code></pre></figure>
<p>let’s take this one operator at a time. On the right, we’ll show
what’s on the stack at the completion of that operation:</p>
<figure class="highlight"><pre><code class="language-postscript" data-lang="postscript"> } % /ushow proc
</code></pre></figure>
<p>This takes the interpreter out of the “defining a procedure” mode and
back into regular, “executing code” mode, then it pushes the procedure
onto the top of the stack. Now set up some arguments:</p>
<figure class="highlight"><pre><code class="language-postscript" data-lang="postscript"> dup % /ushow proc proc
0 % /ushow proc proc 0
</code></pre></figure>
<p>then create a dictionary with room for at least four entries:</p>
<figure class="highlight"><pre><code class="language-postscript" data-lang="postscript"> 4 % /ushow proc proc 0 4
dict % /ushow proc proc 0 dict
</code></pre></figure>
<p>Finally, here’s the fun part:</p>
<figure class="highlight"><pre><code class="language-postscript" data-lang="postscript"> put % /ushow proc
</code></pre></figure>
<p><code class="highlighter-rouge">put</code> stores the dictionary at index 0 of the procedure. There’s a
reason that PostScript calls it an <em>executable array</em>. It really <em>is</em>
an array, and it can be manipulated as one. This code replaced the
symbol <code class="highlighter-rouge">LOCAL</code> at the beginning of the procedure with a dictionary.
When the procedure executes, <code class="highlighter-rouge">LOCAL</code> won’t be there at all. In its
place will be a dictionary.</p>
<p>After that excitement, the rest is really boring:</p>
<figure class="highlight"><pre><code class="language-postscript" data-lang="postscript"> def %
</code></pre></figure>
<p>Finally, store the procedure in the dictionary with the key /ustore.</p>
<p>This is what makes PostScript a little bit like LISP. In LISP, code
is just <em>data that you execute</em>. A LISP program can manipulate other
code as though it were data (because it is, really): change it, add
things to it, replace it with other code–anything you want. And so
can you in PostScript.</p>
Designing FORTRAN: Setting the Target2014-03-07T12:10:00+00:00http://wconrad.github.com/20140307/fortran-design-target<h1 id="which-fortran">Which FORTRAN?</h1>
<p>My prior attempts to create a FORTRAN interpreter were targeted to
FORTRAN 66. I wanted the earliest version of FORTRAN I could find,
which would presumably be for a smaller language, easier to implement
than later versions. I also wanted a hard specification; not a user
manual, but something targeting compiler writers. The earliest
FORTRAN specification I could find was for FORTRAN 66.</p>
<p>Now, at the beginning of the third try, I’ve decided that the better
target is FORTRAN 77. The biggest reason for this is that the only
specification I can find for FORTRAN 66 is a <a href="http://www.fh-jena.de/~kleine/history/.../ansi-x3dot9-1966-Fortran66.pdf">scanned, non-OCR
document</a>. It’s not searchable. And, really, it’s
not that great a specification. The <a href="http://www.fh-jena.de/~kleine/history/languages/ansi-x3dot9-1978-Fortran77.pdf">FORTRAN 77
spec</a> is a searchable PDF, and pretty well written.</p>
<p>Also, I’m going to need programs to run. If I limit myself to FORTRAN
66, there will be fewer to choose from.</p>
<p>So, FORTRAN 77 it is.</p>
<h1 id="be-strict-on-input">Be strict on input</h1>
<p>A compliant <em>processor</em> (the specification’s word for the compiler or
interpreter) must properly run a compliant program, but there are no
requirements for what it must do with a non-compliant program. It can
do anything it likes, although it probably should not halt and catch
fire.</p>
<p>What I want is for this interpreter to be able to reject <em>all</em>
non-conforming programs. This might not be a good idea, just for the
sheer amount of work involved, but I want it anyhow. It would be a
nice badge to pin on the project.</p>
<h2 id="but-not-always">But not always</h2>
<p>I imagine, once I start to run programs from the real world, I will
find non-compliant programs that exploit some extension or looseness
in the compiler they were written for. I can imagine the interpreter
needing to accept switches, a la gcc, to allow various extensions or
non-compliant behaviors.</p>
<p>A nice, OOP way to handle this would be to keep particulars of what is
allowed and what is not out of the interpreter proper, and keep them
in “pluggable” strategy objects which the interpreter would use. For
example, the FORTRAN 77 spec says that identifiers are no longer than
six characters. Many compilers from the era allowed longer
identifiers, but recognized only so many significant characters of
each identifer–the remaining characters were just noise.</p>
<p>Such a thing in this interpreter might be handled with a strategy
object which validates and returns the significant part of any
identifer. Here’s a strict strategy, adhering to the standard:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="k">class</span> <span class="nc">StrictIdentifierStrategy</span>
<span class="k">def</span> <span class="nf">significant</span><span class="p">(</span><span class="n">identifier</span><span class="p">)</span>
<span class="k">raise</span> <span class="no">IdentifierTooLong</span> <span class="k">if</span> <span class="n">identifier</span><span class="p">.</span><span class="nf">length</span> <span class="o">></span> <span class="mi">6</span>
<span class="n">identifier</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></figure>
<p>Here’s a strategy which allows any length identifier, but only the
first so-many characters are significant (many FORTRANs from the era
did this; the number of significant characters was usually fixed for a
given compiler, but varied from compiler to compiler):</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="k">class</span> <span class="nc">SubstringIdentifierStrategy</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">significant_characters</span><span class="p">)</span>
<span class="vi">@significant_characters</span> <span class="o">=</span> <span class="n">significant_characters</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">significant</span><span class="p">(</span><span class="n">identifier</span><span class="p">)</span>
<span class="n">identifier</span><span class="p">[</span><span class="mi">0</span><span class="p">.</span><span class="nf">.</span><span class="p">.</span><span class="nf">significant_characters</span><span class="p">]</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></figure>
<p>When the interpreter starts, the appropriate strategy is picked,
perhaps depending upon a command-line switch:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="vi">@interpreter</span><span class="p">.</span><span class="nf">identifier_strategy</span> <span class="o">=</span>
<span class="k">if</span> <span class="vi">@args</span><span class="p">.</span><span class="nf">long_identifiers</span>
<span class="no">SubstringIdentifierStrategy</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="vi">@args</span><span class="p">.</span><span class="nf">long_identifiers</span><span class="p">)</span>
<span class="k">else</span>
<span class="no">StrictIdentifierStrategy</span><span class="p">.</span><span class="nf">new</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></figure>
Next Project: FORTRAN2014-03-07T11:23:00+00:00http://wconrad.github.com/20140307/starting-fortran<p>Now that I’m done with <a href="http://www.github.com/wconrad/basic101">basic101</a>, the next program I want
to tackle is a FORTRAN interpreter. I’ve already tried this twice;
this will be my third try.</p>
<h1 id="take-one---fortran-with-treetop">Take one - FORTRAN with treetop</h1>
<p>The first try used the <a href="http://treetop.rubyforge.org/">treetop</a>
parser, but I never could figure out how to cleanly turn the treetop
parse tree into an abstract syntax tree. Unfortunately, I implemented
much of the parser before I tried writing the transform, so a lot of
work was wasted when I abandoned treetop.</p>
<p>I had successfully used treetop for a parser at work, to parse an HTML
markup language used by a proprietary PostScript generator. That was
pretty simple, since no transform needed to use the result of any
other transform. It was a simple matter of walking the parse tree
depth first, gathering up transforms, and throwing away any parse node
that didn’t generate a transform. But the transforms of a language
need to refer to each other: This <em>multiply</em> node needs references to
its two <em>term</em> nodes, and so on. I couldn’t figure out how to get
treetop to do that.</p>
<p>It’s not treetop’s fault. I’m just a bit dense.</p>
<h1 id="take-two---fortran-with-parslet">Take two - FORTRAN with Parslet</h1>
<p>The second try at FORTRAN used <a href="http://kschiess.github.io/parslet/">Parslet</a> instead of treetop, but I
didn’t really understand Parslet’s transform rules. In frustration, I
stopped this project, too. I didn’t get very deep into it, so it
wasn’t much of a loss.</p>
<h1 id="take-two-and-a-half-basic101-with-parslet">Take two-and-a-half: basic101 with Parslet</h1>
<p>I then decided to write basic101 in order to learn Parslet more
thoroughly, with a simpler language, so that I could spend more of my
time learning the mechanics of parsing and transforming, and less time
worrying about the language. FORTRAN is a formidible language.</p>
<p>I remembered the pain of my first try at Fortran, where I plowed all
that work into the parser for nothing, so when I wrote basic101, the
first thing I did was an entire slice of the language, enough to parse
and run a simple <code class="highlighter-rouge">REM</code> statement. After that, <code class="highlighter-rouge">PRINT 1</code>. I wanted to
prove that Parslet, and my general approach to the interpreter, was
viable <em>before</em> I sunk a bunch of work into it.</p>
<p>There are two things about Parslet that I didn’t understand at the
start of basic101:</p>
<ol>
<li>A transform rule must match <em>an entire hash at once</em>.</li>
<li>Left-associative operators need special techniques to handle, lest
you cause the parser to loop infinitely.</li>
</ol>
<p>Writing basic101 taught me the first thing pretty well: I could stand
in front of a white board and explain it to someone. The second, I
can’t. I can muddle through it, but not adeptly.</p>
<h1 id="take-three-fortran-with-parslet-again">Take three: FORTRAN with Parslet, again</h1>
<p>So here we are. I think I can bend Parslet to my will (or me to its)
well enough now to handle FORTRAN. It’s time for FORTRAN, take three.</p>
A BASIC Interpreter in Ruby2014-03-06T23:20:00+00:00http://wconrad.github.com/20140306/basic101<p>Last weekend I published
<a href="https://github.com/wconrad/basic101">basic101</a>, a BASIC interpreter
in Ruby. I am fascinated by what’s inside a compiler or interpreter.
Now, BASIC isn’t that exciting a language, but it’s a good one to
start with before moving on to more difficult things. Even though
basic101 is just a tutorial project, it is reasonably polished. has
decent error detection, runs a fair subset of 1980’s BASIC, and the
code is well organized. Even (especially?) a tutorial project
should be good, clean code.</p>
<h1 id="parslet">Parslet</h1>
<p>I wanted to get some solid experience with
<a href="http://kschiess.github.io/Parslet/">Parslet</a> before moving on to
other things. Parslet is an easy to use <a href="http://en.wikipedia.org/wiki/Parsing_expression_grammar">Parsing Expression
Grammar</a>
(PEG), with a twist: It has a wonderful pattern matching DSL for
turning a parse tree into an abstract syntax tree. Another plus is
that it is an internal DSL rather than external. Not having a
separate compilation step is nice, but that’s not the important thing.
What’s important is that since Parslet’s DSL is all ruby, <em>meta</em> is
possible. For example, suppose you are writing a parser for a
language that must ignore spaces in keywords (FORTRAN). In treetop,
this rule parses the keyword <code class="highlighter-rouge">STOP</code>:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="n">rule</span> <span class="n">keyword_stop</span>
<span class="s1">'S'</span> <span class="n">spaces</span> <span class="s1">'T'</span> <span class="n">spaces</span> <span class="s1">'O'</span> <span class="n">spaces</span> <span class="s1">'P'</span>
<span class="k">end</span>
</code></pre></figure>
<p>(the <em>spaces</em> rule, not listed here, eats zero or more characters of
white space). This is hard to read, repetitive, and litters the parse
rules with a separate rule for each keyword.</p>
<p>With Parslet, you can just define a method like this (don’t worry
about what’s in it. What’s important is that this method takes a
string and returns a little Parslet parser that recognizes keywords
which may have embedded spaces:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="k">def</span> <span class="nf">keyword</span><span class="p">(</span><span class="n">s</span><span class="p">)</span>
<span class="n">s</span><span class="p">.</span><span class="nf">chars</span><span class="p">.</span><span class="nf">map</span> <span class="k">do</span> <span class="o">|</span><span class="n">c</span><span class="o">|</span>
<span class="n">str</span><span class="p">(</span><span class="n">c</span><span class="p">)</span>
<span class="k">end</span><span class="p">.</span><span class="nf">inject</span> <span class="k">do</span> <span class="o">|</span><span class="n">l</span><span class="p">,</span> <span class="n">r</span><span class="o">|</span>
<span class="n">l</span> <span class="o">>></span> <span class="n">spaces</span> <span class="o">>></span> <span class="n">r</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></figure>
<p>And then just use it:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="n">rule</span><span class="p">(</span><span class="ss">:stop</span><span class="p">)</span> <span class="k">do</span>
<span class="n">sym</span><span class="p">(</span><span class="s1">'STOP'</span><span class="p">)</span> <span class="o">>></span> <span class="n">line_break</span>
<span class="k">end</span>
</code></pre></figure>
<p>Parlet is nice like that, and in many other ways.</p>
<h1 id="a-basic-gem">A BASIC <em>gem</em>?!</h1>
<p>I also published it as a gem, even though nobody is likely to use it.
Having it as a gem does make it easier for some lost soul to try it.
And, for publishing a gem, there doesn’t seem to be any real bar to
entry: A great many of them are, in one way or another, not really
useful to anyone, by virtual of being undocumented, obsolete, or
whatever. Hey, my gem has <em>test coverage</em>.</p>
Bug Report == Cucumber Scenario?2014-03-06T13:19:00+00:00http://wconrad.github.com/20140306/bug-report-eq-cucumber-scenario<p>It wasn’t until after I created <a href="https://github.com/gollum/gollum/issues/811">this bug
report</a>:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>When non-bare
If I do this:
$ git clone https://github.com/mojombo/gollum-demo gollum-demo-nonbare
$ cd gollum-demo-nonbare
$ gollum --show-all
And then browse
http://localhost:4567/Mordor/Eye-Of-Sauron
I see the pictures of the eye of Sauron.
</code></pre>
</div>
<p>that I realized how similar it was to a <a href="https://github.com/cucumber/cucumber/wiki/Feature-Introduction">Cucumber
scenario</a>:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>Scenario: Buy last coffee
Given there are 1 coffees left in the machine
And I have deposited 1$
When I press the coffee button
Then I should be served a coffee
</code></pre>
</div>
<p>I like it. I don’t know if this is a good form for every bug report,
but it’s nice to read, and clearly separates the setup from the action
from the result.</p>