PiDP-8/I SoftwareHacking on the PiDP-8/I Software
Not logged in

If you are going to make any changes to the PiDP-8/I software, here are some rules and hints to keep in mind while you work.

Getting Started with Fossil

The PiDP-8/I software project is hosted using the Fossil distributed version control system, which provides most of the features of GitHub without the complexities of Git.

Those new to Fossil should at least read its Quick Start Guide. If you want to go deeper, the the Schimpf book is somewhat outdated, but it is still the best single coherent tutorial on Fossil. The official Fossil docs are much more up to date, but they take a piecemeal approach to topics, rather than the linear tutorial approach of a book, so it is not my first recommendation for learning Fossil. Those docs are better for polishing your skills and for reference after you know Fossil reasonably well.

If you have questions about Fossil, ask on the Fossil forum where I, your humble project maintainer, am active. I also work on the Fossil docs quite a bit, so if your question really isn’t answered somewhere in the above material, I might just solve it by extending the Fossil docs.

Fossil is pre-installed on our binary OS images since April 2017.

When starting from Debian-based OSes released in June 2019 or newer, this will work:

$ sudo apt install fossil

Older Debian-based OSes will give you Fossil 1.x, which won’t work with our repository, which requires Fossil 2.1 or higher. If you can’t upgrade your host OS, you’ll have to build Fossil from source.

Fossil is also available for all common desktop platforms. If your OS package repository doesn’t include Fossil 2.1 or higher already, one of the precompiled binaries may work on your system.

Fossil Anonymous Access

To clone the code repository anonymously, say:

$ mkdir -p ~/museum ~/src/pidp8i/trunk
$ fossil clone https://tangentsoft.com/pidp8i ~/museum/pidp8i.fossil
$ cd ~/src/pidp8i/trunk
$ fossil open ~/museum/pidp8i.fossil

The clone command gets you a file called pidp8i.fossil containing the full history of the PiDP-8/I software project from the upstream 2015.12.15 release onward. You can call that clone file anything you like and put it in any directory you like. Even the .fossil extension is largely a convention, not a requirement.

You only need to clone the repository once per machine. Thereafter, you will just be working with that same clone.

Fossil Developer Access

If you have a developer account on the tangentsoft.com/pidp8i Fossil instance, just add your username to the URL like so:

$ fossil clone https://USERNAME@tangentsoft.com/pidp8i pidp8i.fossil

If you've already cloned anonymously, simply tell Fossil about the new sync URL instead:

$ cd ~/src/pidp8i/trunk
$ fossil sync https://USERNAME@tangentsoft.com/pidp8i

Either way, Fossil will ask you for the password for USERNAME on the remote Fossil instance, and it will offer to remember it for you. If you let it remember the password, operation from then on is scarcely different from working with an anonymous clone, except that on checkin, your changes will be sync’d back to the repository on tangentsoft.com if you're online at the time, and you'll get credit under your developer account name for the checkin.

If you're working offline, Fossil will still do the checkin locally, and it will sync up with the central repository after you get back online. It is best to work on a branch when unable to use Fossil's autosync feature, as you are less likely to have a sync conflict when attempting to send a new branch to the central server than in attempting to merge your changes to the tip of trunk into the current upstream trunk, which may well have changed since you went offline.

You can purposely work offline by disabling autosync mode:

$ fossil set autosync 0

Until you re-enable it (autosync 1) Fossil will stop trying to sync your local changes back to the central repo. In this mode, Fossil works more like Git's default mode, buying you many of the same problems that go along with that working style. I recommend disabling autosync mode only when you are truly going to be offline and don't want Fossil attempting to sync when you know it will fail.

Getting Developer Access

We are pretty open about giving developer access to someone who’s provided at least one good, substantial patch to the software. If we’ve accepted one of your patches, just ask for a developer account on the forum.

Working with Existing Tags and Branches

The directory structure shown in the commands above is more complicated than strictly necessary, but it has a number of nice properties.

First, it collects other software projects under a common top-level directory, which I'm calling ~/src, but you are free to use any scheme you like.

Second, the project directory (~/src/pidp8i) stores multiple separate checkouts, one for each version I'm actively working with at the moment. So, to add a few other checkouts, you could say:

$ cd ~/src/pidp8i
$ mkdir -p release          # another branch
$ mkdir -p v20151215        # a tag this time, not a branch
$ mkdir -p 2019-04-01       # the software as of a particular date
  ...etc...
$ cd release
$ fossil open ~/museum/pidp8i.fossil release
$ cd ../v20151215
$ fossil open ~/museum/pidp8i.fossil v20151215
$ cd ../2019-04-01
$ fossil open ~/museum/pidp8i.fossil 2019-04-01
  ...etc...

This gives you multiple independent checkouts, which allows you to quickly switch between versions with “cd” commands. The alternative (favored by Git and some other version control systems) is to use a single working directory and switch among versions by updating that single working directory in place. The problem with that is that it invalidates all of the build artifacts tied to changed files, so you have a longer rebuild time than simply switching among check-out directories. Since disk space is cheap these days — even on a small Raspberry Pi SD card – it’s better to have multiple working states and just “cd” among them.

When you say fossil update in a check-out directory, you get the “tip” state of that version’s branch. This means that if you created your “release” check-out while version 2017.01.23 was current and you say “fossil update” today, you’ll get the release version 2019.04.25 or later. But, since the v20151215 tag was made on trunk, saying “fossil update” in that check-out directory will fast-forward you to the tip of trunk; you won’t remain pinned to that old version.

The PiDP-8/I project uses tags for each released version, and it has many working branches. You can use any of those names in “fossil open” and “fossil update” commands, and you can also use any of Fossil’s special check-in names.

Creating Branches

Creating a branch in Fossil is scary-simple, to the point that those coming from other version control systems may ask, "Is that really all there is to it?" Yes, really, this is it:

$ fossil ci --branch new-branch-name

That is to say, you make your changes as you normally would; then when you go to check them in, you give the --branch option to the ci/checkin command to put the changes on a new branch, rather than add them to the same branch the changes were made against.

While developers with login rights to the PiDP-8/I Fossil instance are allowed to check in on the trunk at any time, we recommend using branches whenever you're working on something experimental, or where you can't make the necessary changes in a single coherent checkin.

One of this project's core principles is that trunk should always build without error, and it should always function correctly. That's an ideal we have not always achieved, but we do always try to achieve it.

Contrast branches, which PiDP-8/I developers may use to isolate work until it is ready to merge into the trunk. It is okay to check work in on a branch that doesn't work, or doesn't even build, so long as the goal is to get it to a point that it does build and work properly before merging it into trunk.

Here again we have a difference with Git: because Fossil normally syncs your work back to the central repository, this means we get to see the branches you are still working on. This is a good thing. Do not fear committing broken or otherwise bad code to a branch. You are not your code. We are software developers, too: we understand that software development is an iterative process, and that not all ideas spring forth perfect and production-ready from the fingers of its developers. These public branches let your collaborators see what you're up to, and maybe lend advice or a hand in the work; mostly, public branches let your collaborators see what you're up to, so they're not surprised when the change finally lands in trunk.

This is part of what I mean about Fossil fostering close cooperation rather than fostering wild tangents.

Jim McCarthy (author of Dynamics of Software Development) has a presentation on YouTube that touches on this topic at a couple of points:

Fossil's sync-by-default behavior fights these negative tendencies.

PiDP-8/I project developers are welcome to create branches at will. The main rule is to follow the branch naming scheme: all lowercase with hyphens separating words. See the available branch list for examples to emulate.

If you have checkin rights on the repository, it is generally fine to check things in on someone else's feature branch, as long as you do so in a way that cooperates with the purpose of that branch. The same is true of trunk: you should not check something in directly on the trunk that changes the nature of the software in a major way without discussing the idea first. This is yet another use for branches: to make a possibly-controversial change so that it can be discussed before being merged into the trunk.

Special Branches

Most of the branches in the PiDP-8/I project are feature branches of the sort described in the previous section: an isolated line of development by one or more of the project's developers to work towards some new feature, with the goal of merging that feature into the trunk branch.

There are a few branches in the project that are special, which are subject to different rules than other branches:

Developer Discussion Forum

The "Forum" link at the top of the Fossil web interface is for discussing the development of the PiDP-8/I software only. All other traffic should go to the mailing list instead. We're not trying to split the community by providing a second discussion forum; we just think many development-related discussions are too low-level to be of any interest to most of the people on the mailing list.

You can sign up for the forums without having a developer login, and you can even post anonymously. If you have a login, you can sign up for email alerts if you like.

Keep in mind that posts to the Fossil forum are treated much the same way as ticket submissions and wiki articles. They are permanently archived with the project. The "edit" feature of Fossil forums just creates a replacement record for a post, but the old post is still available in the repository. Don't post anything you wouldn't want made part of the permanent record of the project!

Debug Builds

By default, the build system creates a release build, but you can force it to produce a binary without as much optimization and with debug symbols included:

 $ ./configure --debug-mode
 $ make clean
 $ tools/mmake

Manipulating the Build System Source Files

The autosetup build system is composed of these files and directories:

 auto.def
 autosetup/*
 configure
 Makefile.in

Unlike with GNU Autoconf, which you may be familiar with, the configure script is not output from some other tool. It is just a driver for the generic Tcl and C code under the autosetup directory, which in turn runs the project-specific auto.def Tcl script to configure the software. Some knowledge of Tcl syntax will therefore be helpful in modifying auto.def.

If you have to modify any of the files in autosetup/ to get some needed effect, you should try to get that change into the upstream Autosetup project, then merge that change down into the local copy when it lands upstream.

If you do not have Tcl installed on your system, configure builds a minimal Tcl interpreter called jimsh0, based on the Jim Tcl project. Developers working on the build system are encouraged to use this stripped-down version of Tcl rather than "real" Tcl because Jim Tcl is mostly-pure subset of Tcl, and jimsh0 is a subset of the complete Jim Tcl distribution, so any changes you make that work with the jimsh0 interpreter should also work with "real" Tcl, but not vice versa. If you have Tcl installed and don't really need it, consider uninstalling it to force Autosetup to build and use jimsh0.

The Makefile.in file is largely a standard GNU make file excepting only that it has variables substituted into it by Autosetup using its @VARIABLE@ syntax. At this time, we do not attempt to achieve compatibility with other make programs, though in the future we may need it to work with BSD make as well, so if you are adding features, you might want to stick to the common subset of features implemented by both the GNU and BSD flavors of make. We do not anticipate any need to support any other make flavors.

This, by the way, is why we're not using some heavy-weight build system such as the GNU Autotools, CMake, etc. The primary advantage of GNU Autotools is that you can generate source packages that will configure and build on weird and ancient flavors of Unix; we don't need that. Cross-platform build systems such as CMake ease building the same software on multiple disparate platforms straightforward, but the PiDP-8/I software is built primarily on and for a single operating system, Raspbian Linux. It also happens to build and run on several other OSes, for which we also do not need the full power of something like CMake. Autosetup and GNU make suffice for our purposes here.

Directory Structure

The directory structure of the PiDP-8/I project is as follows:

Submitting Patches

If you do not have a developer login on the PiDP-8/I software repository, you can still send changes to the project.

The simplest way is to say this after developing your change against the trunk of PiDP-8/I:

$ fossil diff > my-changes.patch

Then either upload that file somewhere (e.g. Pastebin) and point to it from a forum post or attach the patch to a new PiDP-8/I mailing list message. Either way, include a declaration of the license you wish to contribute your changes under. We suggest using the SIMH license, but any non-viral OSI-approved license should suffice. We’re willing to tolerate viral licenses for standalone products; for example, CC8 is under the GPL, but it’s fine because it isn’t hard-linked into any other part of the PiDP-8/I software system.

If your change is more than a small patch, fossil diff might not incorporate all of the changes you have made. The old unified diff format can't encode branch names, file renamings, file deletions, tags, checkin comments, and other Fossil-specific information. For such changes, it is better to send a Fossil bundle:

$ fossil set autosync 0                # disable autosync
$ fossil checkin --branch my-changes
  ...followed by more checkins on that branch...
$ fossil bundle export --branch my-changes my-changes.bundle

After that first fossil checkin --branch ... command, any subsequent fossil ci commands will check your changes in on that branch without needing a --branch option until you explicitly switch that checkout directory to some other branch. This lets you build up a larger change on a private branch until you're ready to submit the whole thing as a bundle.

Because you are working on a branch on your private copy of the PiDP-8/I Fossil repository, you are free to make as many checkins as you like on the new branch before giving the bundle export command.

Once you are done with the bundle, send it to the developers the same way you should a patch file.

If you provide a quality patch, we are likely to offer you a developer login on the repository so you don't have to continue with the patch or bundle methods.

Please make your patches or experimental branch bundles against the tip of the current trunk. PiDP-8/I often drifts enough during development that a patch against a stable release may not apply to the trunk cleanly otherwise.

The PiDP-8/I Software Project Code Style Rules

Every code base should have a common code style. Love it or hate it, here are PiDP-8/I's current code style rules:

C Source Code

File types: c, h, c.in

We follow the SIMH project's pre-existing code style when modifying one of its source files:

I find that this style is mostly sensible, but with two serious problems: I find the indented closing curly braces confusing, and I find that the loss of the first indent level for the statements inside a function makes functions all visually run together in a screenful of code. Therefore, when we have the luxury to be working on a file separate from SIMH, we use a variant of its style with these two changes, which you can produce with the tools/restyle command. See its internal comments for details.

That gives the following, when applied to the above example:

    int some_function (char some_parameter)
    {
        int some_variable = 0;

        if (some_parameter != '\0') {
            int nbytes = sizeof (some_parameter);
            char *buffer = malloc (4 * nbytes);

            switch (some_parameter) {
                case 'a':
                    do_something_with_buffer ((char *)buffer); 
                default:
                    do_something_else ();
            }
        }
        else {
            some_other_function (with_a_large, "number of parameters",
                wraps_with_a_single, "indent level");
            printf (stderr, "Failed to allocate buffer.\n");
        }
    }

If that looks greatly different, realize that it is just two indenting level differences: add one indent at function level, except for the closing braces, which we leave at their previous position.

SIMH occasionally exceeds 100-column lines. I recommend breaking long lines at 72 columns. Call me an 80-column traditionalist.

When in doubt, mimic what you see in the current code. When still in doubt, ask on the project forum.

Plain Text Files

File types: md, txt

Ticket Processes

Normal end users of the Fossil ticket system are not expected to understand it properly or to fill out tickets properly. Without certain permissions, it is in fact not possible to completely fill out a ticket properly.

Therefore, the first thing that should happen to a ticket is that someone with sufficient privilege should triage it and fix up any incorrect settings.

The Status of a ticket initially starts out as Open. If the person triaging a ticket takes the time to check that the problem actually occurs as the ticket filer claims or that the requested feature is in fact missing, the Status should change to Verified.

If a developer implements a fix or feature in response to a ticket, he has two choices: change the ticket’s Status to “Review” if he wants someone to check out the change before closing it, or go straight to Closed status. Closing a ticket hides it from the Wishes and Bugs ticket reports.

Larger teams often require all ticket changes to go through Review status before getting to Closed, but the PiDP-8/I project is too small to require such ceremony: if we’ve given you a developer account on the repository, you’re expected to resolve and close tickets in the same step, most of the time. If you cannot confidently close a ticket when resolving it, you should probably not be assigning a resolution yet anyway. Do whatever you have to with tests and such to become certain before you resolve a ticket.

There is a process interlock between the Resolution and Status settings for a ticket: while Status is not Closed, Resolution should be Open. When Resolution changes from Open, Status should change to either Review or, preferentially, Closed. A resolution is an end state, not an expression of changeable intent: no more ceremony than setting a ticket’s Resolution state away from Open and its Status to Closed is required.

If you do not intend to close a ticket but wish to advocate for a particular resolution, just add a comment to the ticket and let someone else choose whether to close the ticket or not. Don’t change the Resolution value until the issue has been resolved for good.

For example, the resolution "Works as Designed" does not merely mean, “Yes, we know it works that way,” it also implies “...and we have no intention to change that behavior, ever.” If there is a chance that the behavior described in the ticket could change, you should not assign any resolution. Just leave it open until someone decides to do something final with the ticket.

This is not to say that a ticket can never be re-opened once it’s had a resolution assigned and been closed, but that this is a rare occurrence. When a developer makes a decision about a ticket, it should be difficult to re-open the issue. A rejected ticket probably shouldn’t be re-opened with anything short of a working patch, for example:

User A: I want feature X.

Dev B: No, we’re not going to do that. Ticket closed and rejected.

User C: Here’s a patch to implement feature X.

Dev B: Well, that’s different, then. Thanks for the patch! Ticket marked Implemented, but still Closed.

License

Copyright © 2016-2019 by Warren Young. This document is licensed under the terms of the SIMH license.