PiDP-8/I Software

Contributing to the PiDP-8/I Project
Log In

If you wish to make any changes to the project’s files, here are some rules and hints to keep in mind while you work.

Getting Started with Fossil

The PiDP-8/I software project is hosted using the Fossil distributed version control system, which provides most of the features of GitHub without the complexities of Git.

Those new to Fossil should at least read its Quick Start Guide. If you want to go deeper, the Schimpf book is somewhat outdated, but it is still the best single coherent tutorial on Fossil. The official Fossil docs are much more up to date, but they take a piecemeal approach to topics, rather than the linear tutorial approach of a book, so it is not my first recommendation for learning Fossil. Those docs are better for polishing your skills and for reference after you know Fossil reasonably well.

If you have questions about Fossil, ask on the Fossil forum where I, your humble project maintainer, am active. I also work on the Fossil docs quite a bit, so if your question really isn’t answered somewhere in the above material, I might just solve it by extending the Fossil docs.

Fossil is pre-installed on our binary OS images since April 2017.

When starting from Debian-based OSes released in June 2019 or newer, this will work:

$ sudo apt install fossil

Older Debian-based OSes will give you Fossil 1.x, which won’t work with our repository, which requires Fossil 2.1 or higher. If you can’t upgrade your host OS, you’ll have to build Fossil from source.

Fossil is also available for all common desktop platforms. If your OS package repository doesn’t include Fossil 2.1 or higher already, one of the precompiled binaries may work on your system.

Fossil Anonymous Access

There are three ways to clone the repository anonymously. The easiest way requires Fossil 2.14 or higher, currently unreleased but available in development form:

$ fossil clone https://tangentsoft.com/pidp8i
$ cd pidp8i

That gets you a clone of the pidp8i.fossil repository plus a check-out of the current trunk in a pidp8i/ directory alongside it. We recommend that you do this in a directory like ~/src so you don’t commingle these files with other things in your current working directory.

If you have Fossil 2.12 or 2.13, the next-easiest method is:

$ mkdir -p ~/src/pidp8i
$ cd ~/src/pidp8i
$ fossil open https://tangentsoft.com/pidp8i

This creates a pidp8i directory then clones the repository as pidp8i/pidp8i.fossil, then opens that repo into that same subdirectory. The repo file ends up inside the check-out tree with this method.

The complicated method works with all versions of Fossil back to 2.1, and it is the one we recommend to people who want to get involved with the project, because it has numerous advantages over the easy methods. We’ll explain those benefits later, but for now, the method is:

$ mkdir -p ~/museum ~/src/pidp8i/trunk
$ fossil clone https://tangentsoft.com/pidp8i ~/museum/pidp8i.fossil
$ cd ~/src/pidp8i/trunk
$ fossil open ~/museum/pidp8i.fossil

All three methods get you a file called pidp8i.fossil containing the full history of the PiDP-8/I software project from the upstream 2015.12.15 release onward.

You only need to do this once per machine. Thereafter, you will just be working with the cloned repository.

Fossil Developer Access

If you have a developer account on the tangentsoft.com/pidp8i Fossil instance, just add your username to the URL like so:

$ fossil clone https://USERNAME@tangentsoft.com/pidp8i pidp8i.fossil

If you’ve already cloned anonymously, simply tell Fossil about the new sync URL instead:

$ cd ~/src/pidp8i/trunk
$ fossil sync https://USERNAME@tangentsoft.com/pidp8i

Either way, Fossil will ask you for the password for USERNAME on the remote Fossil instance, and it will offer to remember it for you. If you let it remember the password, operation from then on is scarcely different from working with an anonymous clone, except that on checkin, your changes will be sync’d back to the repository on tangentsoft.com if you’re online at the time, and you’ll get credit under your developer account name for the checkin.

If you’re working offline, Fossil will still do the checkin locally, and it will sync up with the central repository after you get back online. It is best to work on a branch when unable to use Fossil’s autosync feature, as you are less likely to have a sync conflict when attempting to send a new branch to the central server than in attempting to merge your changes to the tip of trunk into the current upstream trunk, which may well have changed since you went offline.

You can purposely work offline by disabling autosync mode:

$ fossil set autosync 0

Until you re-enable it (autosync 1) Fossil will stop trying to sync your local changes back to the central repo. In this mode, Fossil works more like Git’s default mode, buying you many of the same problems that go along with that working style. I recommend disabling autosync mode only when you are truly going to be offline and don’t want Fossil attempting to sync when you know it will fail.

Getting Developer Access

We are pretty open about giving developer access to someone who’s provided at least one good, substantial patch to the software. If we’ve accepted one of your patches, just ask for a developer account on the forum.

Working with Existing Tags and Branches

The directory structure shown in second “clone and open” sequence above is more complicated than strictly necessary, but it has a number of nice properties.

First, it collects software projects under a common top-level directory. I’ve used ~/src for this example, but you are free to use any scheme you like.

Second, the level underneath the project directory (~/src/pidp8i) stores multiple separate checkouts, one for each version the developer is actively working with at the moment, so to add a few other checkouts, you could say:

$ cd ~/src/pidp8i
$ mkdir -p release          # another branch
$ mkdir -p v20151215        # a tag this time, not a branch
$ mkdir -p 2019-04-01       # the software as of a particular date
  ...etc...
$ cd release
$ fossil open ~/museum/pidp8i.fossil release
$ cd ../v20151215
$ fossil open ~/museum/pidp8i.fossil v20151215
$ cd ../2019-04-01
$ fossil open ~/museum/pidp8i.fossil 2019-04-01
  ...etc...

This gives you multiple independent checkouts, which allows you to quickly switch between versions with “cd” commands. The alternative (favored by Git and some other version control systems) is to use a single working directory and switch among versions by updating that single working directory in place. The problem is that this invalidates all of the build artifacts tied to changed files, so you have a longer rebuild time than simply switching among check-out directories. Since disk space is cheap these days — even on a small Raspberry Pi SD card – it’s better to have multiple working states and just “cd” among them.

When you say fossil update in a check-out directory, you get the “tip” state of that version’s branch. This means that if you created your “release” check-out while version 2017.01.23 was current and you say “fossil update” today, you’ll get the release version 2019.04.25 or later. But, since the v20151215 tag was made on trunk, saying “fossil update” in that check-out directory will fast-forward you to the tip of trunk; you won’t remain pinned to that old version. This is one of the essential differences between tags and branches in Fossil, which are at bottom otherwise nearly identical.

The PiDP-8/I project uses tags for each released version, and it has many working branches. You can use any of those names in “fossil open” and “fossil update” commands, and you can also use any of Fossil’s special check-in names.

Creating Branches

Creating a branch in Fossil is scary-simple, to the point that those coming from other version control systems may ask, “Is that really all there is to it?” Yes, really, this is it:

$ fossil ci --branch new-branch-name

That is to say, you make your changes as you normally would; then when you go to check them in, you give the --branch option to the ci/checkin command to put the changes on a new branch, rather than add them to the same branch the changes were made against.

While developers with login rights to the PiDP-8/I Fossil instance are allowed to check in on the trunk at any time, we recommend using branches whenever you’re working on something experimental, or where you can’t make the necessary changes in a single coherent checkin.

One of this project’s core principles is that trunk should always build without error, and it should always function correctly. That’s an ideal we have not always achieved, but we do always try to achieve it.

Contrast branches, which PiDP-8/I developers may use to isolate work until it is ready to merge into the trunk. It is okay to check work in on a branch that doesn’t work, or doesn’t even build, so long as the goal is to get it to a point that it does build and work properly before merging it into trunk.

Here again we have a difference with Git: because Fossil normally syncs your work back to the central repository, this means we get to see the branches you are still working on. This is a good thing. Do not fear committing broken or otherwise bad code to a branch. You are not your code. We are software developers, too: we understand that software development is an iterative process, that not all ideas spring forth perfect and production-ready from the fingers of its developers. These public branches let your collaborators see what you’re up to; they may be able to lend advice, to help with the work, or to at least be unsurprised when your change finally lands in trunk.

Fossil fosters close cooperation, whereas Git fosters wild tangents that never come back home.

Jim McCarthy (author of Dynamics of Software Development) has a presentation on YouTube that touches on this topic at a couple of points:

Fossil’s sync-by-default behavior fights these negative tendencies.

PiDP-8/I project developers are welcome to create branches at will. The main rule is to follow the branch naming scheme: all lowercase with hyphens separating words. See the available branch list for examples to emulate.

If you have checkin rights on the repository, it is generally fine to check things in on someone else’s feature branch, as long as you do so in a way that cooperates with the purpose of that branch. The same is true of trunk: you should not check something in directly on the trunk that changes the nature of the software in a major way without discussing the idea first. This is yet another use for branches: to make a possibly-controversial change so that it can be discussed before being merged into the trunk.

Special Branches

Most of the branches in the PiDP-8/I project are feature branches of the sort described in the previous section: an isolated line of development by one or more of the project’s developers to work towards some new feature, with the goal of merging that feature into the trunk branch.

There are a few branches in the project that are special, which are subject to different rules than other branches:

Developer Discussion Forum

The “Forum” link at the top of the Fossil web interface is for discussing the development of the PiDP-8/I software only. All other traffic should go to the end-user focused mailing list instead. We’re not trying to split the community by providing a second discussion forum; we just think many development-related discussions are too low-level to be of any interest to most of the people on the mailing list.

You can sign up for the forums without having a developer login, and you can even post anonymously. If you have a login, you can sign up for email alerts if you like.

Keep in mind that posts to the Fossil forum are treated much the same way as ticket submissions and wiki articles. They are permanently archived with the project. The “edit” feature of Fossil forums just creates a replacement record for a post, but the old post is still available in the repository. Don’t post anything you wouldn’t want made part of the permanent record of the project!

Debug Builds

By default, the build system creates a release build, but you can force it to produce a binary without as much optimization and with debug symbols included:

$ ./configure --debug-mode
$ make clean
$ tools/mmake

Manipulating the Build System Source Files

The autosetup build system is composed of these files and directories:

auto.def
autosetup/*
configure
Makefile.in

Unlike with GNU Autoconf, which you may be familiar with, the configure script is not output from some other tool. It is just a driver for the generic Tcl and C code under the autosetup directory, which in turn runs the project-specific auto.def Tcl script to configure the software. Some knowledge of Tcl syntax will therefore be helpful in modifying auto.def.

If you have to modify any of the files in autosetup/ to get some needed effect, you should try to get that change into the upstream Autosetup project, then merge that change down into the local copy when it lands upstream.

If you do not have Tcl installed on your system, configure builds a minimal Tcl interpreter called jimsh0, based on the Jim Tcl project. Developers working on the build system are encouraged to use this stripped-down version of Tcl rather than “real” Tcl because Jim Tcl is a mostly-pure subset of Tcl, and jimsh0 is a subset of the complete Jim Tcl distribution, so any changes you make that work with the jimsh0 interpreter should also work with “real” Tcl, but not vice versa. If you have Tcl installed and don’t really need it, consider uninstalling it to force Autosetup to build and use jimsh0 to ensure that your changes to auto.def work on both interpreters.

The Makefile.in file is largely a standard GNU make file excepting only that it has variables substituted into it by Autosetup using its @VARIABLE@ syntax. At this time, we do not attempt to achieve compatibility with other make programs, though in the future we may need it to work with BSD make as well, so if you are adding features, you might want to stick to the common subset of features implemented by both the GNU and BSD flavors of make. We do not anticipate any need to support any other make flavors.

This, by the way, is why we’re not using some heavy-weight build system such as the GNU Autotools, CMake, etc.

The primary advantage of GNU Autotools is that you can generate standalone source packages that will configure and build on weird and ancient flavors of Unix. We don’t need that.

Cross-platform build systems such as CMake ease building the same software on multiple disparate platforms, but the PiDP-8/I software is built primarily on and for a single operating system, Raspberry Pi OS, né Raspbian. It also happens to build and run on several other OSes, for which we also do not need the full power of something like CMake. Autosetup and GNU make suffice for our purposes here.

Directory Structure

The directory structure of the PiDP-8/I project is as follows:

Submitting Patches

If you do not have a developer login on the project repository, you can still send changes to the project.

The simplest way is to say this after developing your change against trunk:

$ fossil diff > my-changes.patch

Then paste that into a forum post using a fenced code block. We will also accept trivial patches not needing discussion as text or attachments on a Fossil ticket.

If you're making a patch against a PiDP-8/I distribution tarball, you can generate a patch this way:

$ diff -ruN pidp8i-olddir pidp8i-newdir > mychange.patch

The diff command is part of every Unix and Linux system, and should be installed by default. If you're on a Windows machine, GNU diff is part of Cygwin and WSL. Fossil is also available for all of these systems. There are no excuses for not being able to make unified diffs. :)

Bundles Instead of Patches

If your change is more than a small patch, fossil diff might not incorporate all of the changes you have made. The old unified diff format can’t encode branch names, file renamings, file deletions, tags, checkin comments, and other Fossil-specific information. For such changes, it is better to send a Fossil bundle:

$ fossil set autosync 0                # disable autosync
$ fossil checkin --branch my-changes
  ...followed by more checkins on that branch...
$ fossil bundle export --branch my-changes my-changes.bundle

After that first fossil checkin --branch ... command, any subsequent fossil ci commands will check your changes in on that branch without needing a --branch option until you explicitly switch that checkout directory to some other branch. This lets you build up a larger change on a private branch until you’re ready to submit the whole thing as a bundle.

Because you are working on a branch on your private copy of the project’s Fossil repository, you are free to make as many checkins as you like on the new branch before giving the bundle export command.

Once you are done with the bundle, upload it somewhere public and point to it from a forum post or ticket.

Contribution Licensing

Submissions should include a declaration of the license you wish to contribute your changes under. We suggest using the SIMH license, but any non-viral OSI-approved license should suffice. We’re willing to tolerate viral licenses for standalone products; for example, CC8 is under the GPL, but it’s fine because it isn’t statically linked into any other part of the PiDP-8/I software system.

Can I Use GitHub Instead?

Although the PiDP-8/I project does have a GitHub mirror, it is intended as a read-only mirror for those heavily tied into Git-based tooling. You’re welcome to send us a PR anyway, but realize that what’s going to happen on the back end is that we’ll generate a patch, apply it to the Fossil repo by hand, test it, and then commit it to the repository under one of our existing Fossil developer accounts. Only then do we update the mirror so that the change appears on GitHub; thus, you don’t get GitHub credit for the PR. You avoid these problems by simply asking for a developer account on the Fossil repo, so you can commit there instead.

This is not simply because setting up bidirectional mirroring is difficult, it is actually impossible to achieve 100% fidelity due to limitations of Git and/or GitHub. If you want a faithful clone of the project repo, or if you wish to contribute to the project’s development with full credit for your contributions, it’s best done via Fossil, not via GitHub.

The PiDP-8/I Software Project Code Style Rules

Every code base should have a common code style. Love it or hate it, here are PiDP-8/I’s current code style rules:

C Source Code

File types: *.c, *.h, *.c.in

We follow the SIMH project’s pre-existing code style when modifying one of its source files:

Plain Text Files

File types: *.md, *.txt

Ticket Workflow

Normal end users of the Fossil ticket system are not expected to understand it properly or to fill out tickets properly. Without certain permissions, it is in fact not possible to completely fill out a ticket properly. Project developers typically must triage, augment, and correct submissions from the start.

Here’s the basic workflow for a “code defect” ticket, colloquially called a bug report:

SUBMIT TICKET New bug ticket marked "Open" Triage, augment & correct Developer comments Filer responds Reject? Yes Mark ticket "Rejected" & "Resolved" Rejected ticket Reopen?   No; fix it Developer changes code Fixed? No Optional: Update ticket resolution: "Partial Fix", etc.   Yes Mark ticket "Fixed" & "Closed" Resolved ticket END fast reject path No Yes
      fill = bisque
      linerad = 15px

      define diamond { \
        box wid 150% invis
        line from last.w to last.n to last.e to last.s close rad 0 $1
      }

      oval "SUBMIT TICKET" width 150%
      down
      arrow 50%
NEW:  file "New bug ticket" "marked \"Open\"" fit
      arrow same
      box "Triage," "augment &" "correct" fit
      arrow same
DC:   box "Developer comments" fit
      arrow same
FR:   box "Filer responds" fit
      arrow 100%
REJ:  diamond("Reject?")
      right
      arrow 100% "Yes" above
      box "Mark ticket" "\"Rejected\" &" "\"Resolved\"" fit with .w at previous.e
      arrow right 50%
REJF: file "Rejected" "ticket" fit
      arrow right 50%
REOP: diamond("Reopen?")
      down
REJA: arrow 75% from REJ.s "  No; fix it" ljust
CHNG: box "Developer changes code" with .n at last arrow.s fit
      arrow 50%
FIXD: diamond("Fixed?")
      right
FNO:  arrow "No" above
RES:  box "Optional:" "Update ticket resolution:" "\"Partial Fix\", etc." fit
      down
      arrow 75% "  Yes" ljust from FIXD.s
      box "Mark ticket" "\"Fixed\" & \"Closed\"" fit
      arrow 50%
RESF: file "Resolved ticket" fit
      arrow same
      oval "END"

      line from 0.3<FR.ne,FR.se> right even with 0.25 right of DC.e then \
          up even with DC.e then to DC.e ->

      line from NEW.w left 0.5 then down even with REJ.w then to REJ.w ->
      line invis from 2nd vertex of last line to 3rd vertex of last line \
          "fast reject path" above aligned

      line from RES.e right 0.3 then up even with CHNG.e then to CHNG.e ->

      line from REOP.s "No" aligned above down 0.4
      line from previous.s down to (previous.s, RESF.e) then to RESF.e ->

      line from REOP.n "Yes" aligned below up 0.3
      line from previous.n up even with 0.6<FR.ne,FR.se> then to 0.6<FR.ne,FR.se> ->

On noticing a new-filed ticket — such as because you are subscribed to email notifications on ticket changes — someone with sufficient privilege triages it, sets values for the fields not exposed to the ticket’s filer, and fixes up any incorrect values given in the initial submission.

The Status of a ticket initially starts out as Open; the filer cannot change that default value, short-circuiting the process. If the person triaging a ticket takes the time to check that the bug actually occurs as the ticket filer claims, the Status should change to Verified.

If a developer implements a fix in response to a ticket, he has two choices: change the ticket’s Status to “Review” if he wants someone to check out the change before closing it, or go straight to Closed status. Closing a ticket hides it from the Wishes and Bugs ticket reports.

The process for software feature request and documentation improvement request tickets is essentially the same, differing mainly in terminology rather than process flow: instead of verifying the existence of a bug, the one triaging the ticket verifies that the feature does in fact not exist yet, and so on.

A common requrement in larger teams is that all ticket changes to go through Review status before getting to Closed, but the PiDP-8/I project is too small to require such ceremony: if we’ve given you a developer account on the repository, you’re expected to resolve and close tickets in the same step, most of the time. If you cannot confidently close a ticket when resolving it, you should probably not be assigning a resolution yet anyway. Do whatever you have to with tests and such to become certain before you resolve a ticket.

There is a process interlock between the Resolution and Status settings for a ticket: while Status is not Closed, Resolution should be Open. When Resolution changes from Open, Status should change to either Review or, preferentially, Closed. A resolution is an end state, not an expression of changeable intent: no more ceremony than setting a ticket’s Resolution state away from Open and its Status to Closed is required.

If you do not intend to close a ticket but wish to advocate for a particular resolution, just add a comment to the ticket and let someone else choose whether to close the ticket or not. Don’t change the Resolution value until the issue has been resolved for good.

For example, the resolution “Works as Designed” does not merely mean, “Yes, we know it works that way,” it also implies “…and we have no intention to change that behavior, ever.” If there is a chance that the behavior described in the ticket could change, you should not assign any resolution. Just leave it open until someone decides to do something final with the ticket.

This is not to say that a ticket can never be re-opened once it’s had a resolution assigned and been closed, but that this is a rare occurrence. When a developer makes a decision about a ticket, it should be difficult to re-open the issue. A rejected ticket probably shouldn’t be re-opened with anything short of a working patch, for example:

User A: I want feature X.

Dev B: No, we’re not going to do that. Ticket closed and rejected.

User C: Here’s a patch to implement feature X.

Dev B: Well, that’s different, then. Thanks for the patch! Ticket marked Implemented, but still Closed.

License

Copyright © 2016-2020 by Warren Young. This document is licensed under the terms of the SIMH license.