PiDP-8/I SoftwareRelease 2020 plan
Not logged in

Release 2020 plan

(1) By Bill Cattey (poetnerd) on 2020-06-17 04:21:51 [link] [source]

Rather than continue off-topic discussion of punchlist items for the 2020 release, I decided to start this new thread.

The previous TODOs:

  1. Integrate e8 is done.
  2. Integrate LCBAS.BI and UCBAS.BI into uni is done.
  3. Assemble UF.PA if possible. I've scoped this work.

Surprisingly, for all the talk of how the Combined Kit is supposed to have integrated all patches up to that point, the patch 31.5.1M that is required for proper operation of BASIC.UF was never done at the source code level. Along the way I reviewed other patches applied to BASIC, and found that the mandatory BASIC patch 31.12.1M that Lee Nichols put into BASIC.PA was renamed in August 1979 in a way that made it impossible to use the Cumulative Index to find it after that. Happily, I went forward in time when I did my original patch inventory so I properly curated the patch.

I think this means that I have one new TODO:

Review the patches. Confirm they've been applied in source form. Apply as needed.

The scope of that work is tricky. I know that UF.PA needs to be applied because otherwise it completely breaks. Applying it is quick.

However, having done the patch review once before, the other review work might take a while. My next step is to scope out the work and propose a cutoff point for such patching to be in/out of the release.

(2) By Bill Cattey (poetnerd) on 2020-06-19 03:59:48 [link] [source] in reply to 1

I have corrected what would have been a regression of BASIC.UF.

I have not yet scoped out the other Patch work.

(3) By Bill Cattey (poetnerd) on 2020-06-20 04:10:53 [link] [source] in reply to 2

I have scoped out some of the work. The bad news is that some patches that were applied to V3D were not applied to the Combined Kit in source form and need to be.

Fifty patches were originally reviewed and 40 of them were applied.

Of the 50 patches to review, I've re-reviewed 20 as of this moment.) Of that 20, 15 need no further work.

I'll expedite review of the remaining 30 and reply to this thread with an inventory of what further work, be it binary or source application is under consideration.

(4) By Bill Cattey (poetnerd) on 2020-06-20 23:39:38 [link] [source] in reply to 3

I have completed my re-review of the patches applied to OS/8 V3D to scope out follow-on work.

To prevent regression, 16 patches should be applied. The rest are either irrelevant because of major version upgrades, already applied in source form, or (as was the case for UF.PA) fixed up already.

Of those 16 patches, one is optional that I applied to V3D. All 16 could simply be applied. (Although one is a FUTIL patch, and the FUTIL patch runner has not been extensively tested.)

Five came out after the documented patch application date. Ten came out before the documented patch application date: Eight affecting TECO, one affecting F4, one affecting CREF. So the former five, to prevent regression, drive the creation of a patching setup for the Combined Kit.

I've looked at the source code for the latter ten. I like the way Lee Nichols performed source code integration, where the patch ID is mentioned and both old and new code is present, with the old code commented out.

I'm going to take a position that minimizes my work, and leans on trusting Lee's work where he said, "The built binaries match what was on the Combined Kit binary image" that he had in hand.

Next step:

Create patching infrastructure for the Combined Kit, and apply the 16 patches. This is probably a lot less work than integrating the patches in source form.

(5) By Bill Cattey (poetnerd) on 2020-06-21 16:32:14 [link] [source] in reply to 4

I have just finished verifying (i.e. confirming that the intended bits got onto the intended files) and testing (i.e. running basic run tests of patched programs) of all the patches needed to prevent known regressions between the V3D distribution and the OCK distribution.

The next step that occurs to me is:

Add in a compile-time choice between v3d and ock for the default boot disk. This seems a bit tricky to me because we DO need the v3d image (preferably v3d-patched.rk05) to create the ock system.

I see that auto.def and Makefile.in have the notion of @OS8_BOOT_DISK@ that enables some choice there. However, I've not been able to convince myself that the right think will happen if we choose ock.rk05 as the OS8_BOOT_DISK.

Warren, can you tease out the dependencies, and create the right setup so we can make a compile-time option between v3d and ock as the default boot disk after install, such that v3d-patched.rk05 is what's used to build stuff, but then fades into the background if we choose ock.rk05 as OS8_BOOT_DISK?

This now is a time for considering what, if anything we do with v3f, and the bootable tu56 images. I suggest we make creating v3f a compile time option with a default of "no".

(6) By Bill Cattey (poetnerd) on 2020-06-25 02:28:38 [link] [source] in reply to 5

I've thought some more about this. I inventoried the places where OS8_BOOT_DISK is used:

  • .../tools/os8-cmp.in
  • .../boot/0.script.in
  • .../boot/run.script.in
  • .../bin/os8-cp.in
  • .../Makefile.in

There are actually three questions that are currently conflated:

  1. What disk image gets booted when the installed system is run?
  2. What disk image gets booted when the os8-cmp and os8-cp utilities are run?
  3. What disk image gets booted to build things?

For the purposes of bootstrapping, we have the following layers:

  1. Boot the OS/8 V3D Distribution DECtape image, .../media/os8/v3d/al-4711c-ba-os8-v3d-1.1978.tu56. This creates v3d-dist.rk05 which is booted to continue the build process.
  2. Boot v3d-dist.rk05 to apply patches to create a useful image with the latest bug fixes, v3d-patched.rk05. Strictly speaking, this is the optimal, but minimum platform for continuing to build, and to operate utilities.
  3. Boot v3d-patched.rk05 to build e8, cc8, and anything else that needs OS/8. This is the platform that should be used to build v3f from source, and the OS/8 Combined Kit from source. Creating the images used to assemble v3f and ock from source requires os8-cp.
  4. Install packages built with OS/8 onto runable packs. This is where v3d-patched.rk05 becomes v3d.rk05. At the present moment, the component rk05 images that will be gathered into ock-dist.rk05 have been built using v3d-patched.rk05. They could have been built using v3d.rk05. The choice is made in the build scripts. The ock-dist.rk05 image is constructed similarly to layer 1. The al-4711c-ba-os8-v3d-1.1978.tu56 image is booted, and used to create an rk05 image, which is then populated with the other components built in layer 3. At this point we have bootable tu56 images for v3d and v3f built from v3d.rk05. We also have ock-dist.rk05 built from source.
  5. Boot ock-dist.rk05 and apply patches to create ock-patched.rk05.
  6. Install packages such as cc8 and e8 on ock-patched.rk05 to create ock.rk05. This completes all building.

At the completion of the build process, we now have a choice of bootable RK05 images: v3d.rk05, or ock.rk05.

After describing all this, I see that these layers actually constrain question 2, because v3f and ock both use os8-cp. Since they don't exist yet, we have to use either v3d-dist.rk05, v3d-patched.rk05, or v3d.rk05.

Proposal:

@OS8_BUILDER_DISK@ is v3d-patched.rk05. That image has the most complete and up to date binaries. Convert use of @OS8_BOOT_DISK@ in Makefile.in, os8-cp.in, and os8-cmp.in to this.

@OS8_RUNTIME@ is a choice between v3d.rk05 and ock.rk05. Convert use of @OS8_BOOT_DISK@ in 0.script.in, and run.script.in to this.

Question:

How do we want to express the choice for OS8_RUNTIME?

(7) By Warren Young (tangent) on 2020-06-25 03:33:58 [link] [source] in reply to 6

I thought it was clear already: *-dist for bootstrapping and for use by tools (e.g. os8-cmp) and either v3d.rk05 or ock.rk05 for interactive booting. Everything else is a build-time intermediary file. Some of those may end up installed and be useful beyond that, but that's a secondary matter.

On the V3F issue, that's an historical oddity, worth reproducing for the curious, but now that we have OCK, I don't see much reason to put a lot more time into it.

Expressed this way, I think the boot options become simple: OCK or V3D. Done. No -dist, no -patched, no nothing. Just the completely built-up versions of each, with all selected packages, patches, and add-ons.

(8) By Bill Cattey (poetnerd) on 2020-06-25 13:37:00 [link] [source] in reply to 7

"The distribution media" has a particular meaning defined by recommended DEC site practice:

Distribution media is what shipped from DEC.
You only run it long enough to make a copy.
The copy is for regular use.
When patches come out you apply them to the copy.
In the event of a bad patch, you go back to the distribution media.
You record all this copying and patching in your system maintenance log.

So actually *-patched is for bootstrapping and use by tools.

Add-ons like 3rd party software have their own distribution media, and then get copied onto the patched running copy.

To me, it seems sensible to separate out the patched copy of OS distribution media from "the thing we run because it has all our third party software on it." What this probably means is that the optional software like FOCAL, ADVENT and the BASIC games should get installed into ock.rk05 and v3d.rk05 instead of current build practice of installing them onto -dist.rk05.

It occurs to me, that, if we move the installation of such optional 3rd party software to later in the build, the testing process becomes simpler! We build the "boot packs", test that for repeatability, and then perform testing on the permutations of copy-ins.

Additional consideration: Up to now we have made OS/8 components BASIC, FORTRAN II, and FORTRAN IV conditional/optional. This comes from the idea that disk space is limited, and we wanted to give the option of having boot packs with more space. What if, instead, we said, "those are system components, and you get them from the build process. If you need space, go ahead and delete them."? That too simplifies our build and test process.

If we really wanted to offer removing them as a convenient option, we could write a tool that simply removes the optional pieces.

To review and summarize:

  • I've been hewing to some "best practice" stuff to create -dist and -patched.
  • Reviewing the current conflation of OS and 3rd party optional software exposes ideas for simplifying build and test.
  • I agree that the "thing you run from boot and run scripts" should be a choice between v3d.rk05 and ock.rk05.
  • Not stated above in detail, but yes, I agree that it's not worth putting much more time into v3f. I would say, however, that it's a useful, interesting capability to give users of the pidp8i the experience of "booting from TD8E or TC08 virtual DECtape." It doesn't have to be v3f. It should be v3d and/or ock.

(9) By Bill Cattey (poetnerd) on 2020-07-18 16:41:29 [link] [source] in reply to 8

Hi Warren,

I want to check in to confirm we're on the same page after my clarification of the concepts behind dist and patched.

I think the boot options become simple: OCK or V3D. Done. Agreed.

I thought it was clear already: *-dist for bootstrapping and for use by tools (e.g. os8-cmp) and either v3d.rk05 or ock.rk05 for interactive booting. Everything else is a build-time intermediary file.

The change I'm proposing is that we use *-patched for use by tools, and for building of components like cc8 and e8. A simpler way to state my rationale is: It covers the case where building requires the latest patch level.

Right now we code to the -dist least common denominator. But there are patches to PAL8, BATCH, BUILD etc. that we may come to want/need.

Reviewing the multi-layered bootstrapping I described, I realize that there may come a time in the future where we wish to rely more heavily on the OS/8 environment built from source (I.E. ock.) By this I mean that the build process would be re-factored to make a minimal use of v3d binary distribution media to create an environment that would build ock, and then ock-patched would be used to build add-ons like cc8 and e8.

But such re-factoring would only come after a burn-in period with ock as the default boot option.

But that is for the future. For now, I want to replace use of OS8_BOOT_DISK with a set of names that clarifies the use being made:

  • @OS8_RUNTIME@ is a choice between v3d.rk05 and ock.rk05. Convert use of @OS8_BOOT_DISK@ in 0.script.in, and run.script.in to this.

  • @OS8_BASELINE@ is v3d-dist. It's used at the lowest levels of build time.

  • @OS8_TOOLTIME@ is v3d-patched. It's used by tools and higher levels of build time.

I didn't want to just make these changes without discussion. Do you see problems with making this replacement?

(10) By Warren Young (tangent) on 2020-07-18 17:39:27 [source] in reply to 9

All of this seems sensible.

I may apply a bit of oar occasionally, but you've got the rudder on this project.