Hello Everyone,

Attending VCF East last week, I met up with Oscar Vermeulen, and he was sharing with me how people are still buying PiDP-8 systems from him but having trouble getting them going with Pi4 and Pi5 hardware.

He shared with me some stuff he had produced to get the Pi5 working for him, but he was having trouble figuring out how to get it into even a branch in the tangentsoft build tree.

I'd like to help move the pi4 and pi5 support forward.

I know that there are several issues, like how SIMH forked into Open SIMH, and how the gpio support in Raspberry OS has changed out from under how it's all currently working for us.

In anticipation of owning a PiDP-10, I've just placed my order for a Raspberry Pi 5, which I will use first here to do testing of the evolving Pi5 support.

I learned the build system, and the release process so I can help shepherd those aspects, but I don't know any details about the gpio API issues, so I'll need to be educated on what needs to happen with Oscar's first draft.

At the present time I understand not one bit of the current gpio implementation, nor Oscar's amendments. Where can I go to learn more?

Specifically, does anyone have a pointer to the thread about the gpio API situation? I think that's the first step.

-Bill Cattey

aka PoetNerd

In out of band communication with Oscar, I was told that the pinctrl interface he proposed was backward compatible with old Pi hardware, and the way forward for current and new hardware.

I have created a branch "pi5" that integrates what Oscar obtained, "One of the developers at the Pi Foundation".

He further asserted:

[T]hat code package as it is both future-proof (they will maintain it for any future Pi) and much faster than any of the other remaining GPIO driver packages. As the Pi 5 has a very different setup (GPIO now goes through the separate RP1 chip, over the southbridge). Pinctrl's code is a factor 10 faster, in fact, than any of the other libraries that support the Pi 5. So pinctrl is the future for higher-speed applications, as far as the Pi Foundation developers are concerned.

I have gotten the pi5 branch to compile on my Pi3b running Buster.

It won't build on my Mac because the pinctrl code uses the non-standard reallocarray malloc call, and so some sort of platform independent implementation of that will be needed.

The switches and lights seem to work just fine when I run bin/pidp8i-sim

The switches seem to work when I run bin/pidp8i-test.

When I run bin/pidp8i-test, the switches work, but the lights don't light up.

Oscar, if you see this thread, can you suggest what might be going on with no lights lighting for pidp8i-test?

I've dug into this some more.

It looks like the update to gpio-common.c.in has some issues, some of which, i think come from lack of clarity of how to deal with non-pi, and non-piddp8i blinking light platforms.

The current distribution has SIMH calling start_pidp8i_gpio_thread with must_map=0. That routine calls map_gpio_for_pidp8i which calls map_peripheral without bothering to check the return status. Basically gpio initialization code keeps trudging along even if it fails, returns happy 0 status, but leaves leaves pidp8i_gpio_present at 0 which other code tests as the definitive indication that there's no gpio present.

For platforms with no gpio hardware there's a stub:

//// bcm_host_get_peripheral_address ///////////////////////////////////
// Provide fallback for non-Pi case to avoid a link error.

#if !defined(HAVE_BCM_HOST_H)
static unsigned bcm_host_get_peripheral_address(void) { return 0; }
#endif

The proposed new gpio-common.c.in gets rid of that stub, and calls out to a new interface pinctrl.

I believe the root cause of pidp8i-test not blinking lights is that the new gpio-common.c.in lost support for "the old ledstatus[] interface for updating the display" as it is described in gpio-common.c.in:swap_displays.

Although the pidp8i_simple_gpio_mode flag is still set and tested, I believe that with the incorporation of setting ledrow GPIO pins into gpio-common.c.in:map_peripherals, that legacy simple interface disappeared.

I may be wrong about this, and am hoping someone with more experience with the code can help here.

I think perhaps the build flag PIDP8I should be split into two flags, one that says we support the PDP-8 instruction set emulation, and one that says that yes, we expect to see gpio hardware.

Making that split would allow elimination of building a lot of gpio-specific modules that strain POSIX portability. (Because the proposed pinctrl code uses reallocarray which is present on the Pi platform where we expect gpio hardware, but is not part of POSIX, and isn't actually needed when we don't have gpio hardware. And also because the construct in pinctrl/gpiochip.h uses an attribute #DEFINE construct that the Apple C compiler chokes on (but which can be coded around to get compilation to work, but actually is only needed for gpio stuff anyway.)

Bottom line: making a clearer split between with and without gpio platforms makes sense to me. If it makes sense to others, I'll code it up.

Oh and I forgot to mention, that whoever wrote misc/test.c to implement pidp8i-test calls gpio-common.c:start_pidp8i_gpio_thread with the string "test program" rather than the expected must_map flag.

It appears that the effect is that, since the call has must_map non-zero, when the gpio setup fails, we get a non-zero status that causes pidp8i-test to exit with EXIT_FAILURE status. I.E. the behavior is as expected, but the utilization of the code is inconsistent with its definition.

The proposed new gpio-common.c.in gets rid of that stub, and calls out to a new interface pinctrl.

Explain to me why that doesn't cause the reallocarray() problem you describe elsewhere. Surely the purpose of this stubbed-out code — when translated to this new model — is to not call pinctrl at all, thus bring none of its dependencies into the macOS and non-Pi Linux worlds?

The whole thing should be compiled out in these cases. Then you don't need to solve the reallocarray() portability problem at all. No PiDP-8/I LED/switch panel, no need for pinctrl.

two flags, one that says we support the PDP-8 instruction set emulation, and one that says that yes, we expect to see gpio hardware.

I don't see the point of the first. Without panel hardware, the software should devolve to a fork of SIMH's "PDP8" simulator.

One of the developers at the Pi Foundation

Yes, here.

I'm not sure we should be "vendoring" this in our source tree. If there's a way to say "apt install pinctrl" or similar on the Pi, then detecting whether it builds and works against the installed header and library, that would be better.

Two other things worry me about this plan:

1. Quoting pinctrl's top-level README: "It accesses the hardware directly, bypassing the kernel drivers, and as such requires root privilege (run with "sudo")." We went to a lot of trouble in the past to make PiDP-8/I software run nonprivileged. Either this just threw all that away, or we'll have to do some clever dancing to start as root and shortly after drop root privs.

2. This code uses the 3-clause BSD license which isn't checked in on the tree or referenced from our top-level COPYING.md file. If we vendor the library, we're obligated to do both. If not, then not.

the utilization of the code is inconsistent with its definition.

Feel free to sort that out. I'm not going to be jumping back into the code any time soon.

Hi Warren,

Thanks for thinking about this. Let me respond point by point:

Sorry for not being clearer, yes, I'm complaining about reallocarray(), and I think the way forward is to detect at configure time that we are or are not building on a pi.

I agree with your questioning why we need a separate PDP-8 instruction set flag at all. Perhaps I misunderstood when I read the code in src/SIMH/scp.c:

#ifdef PIDP8I
    if (strstr (argv[0], "pidp8i-sim") == 0) use_pidp8i_extensions = 0;
    else if (start_pidp8i_gpio_thread (0) != 0) exit (EXIT_FAILURE);
#endif

I thought it was doing more than enabling the gpio. It's defined when I build on my Mac, and perhaps it should not be. If it were conditional on either the existence of libraspberrypi-dev or on UNAME_SYSTEM = Linux and UNAME_MACHINE = arm*, rather than universally set, a bunch of stuff that we don't use would properly go away.

The issues you flagged with pinctrl: Needs root, should be a apt package, and licensing are serious.

In Oscar's note, he said that pidp8i-sim would run with user privileges, but that lights would flicker.

I swam around in that link you provided to the pinctrl github repo. Apparently there's a one-line fix to pinctrl.c that fixes the (admittedly minor) issue of the declaration for reallocarray on the Buster platform.

There was also mention of an update where under conditions I don't actually understand, root is not required: pinctrl: Failed to mmap gpiolib #33 .

I had not mentioned licensing, but it was on my mind.

I agree with you that not vendoring it ourselves is preferable. If we do so, I understand that we'd need to add yet another license to our COPYING.md file, and to our tree. At the present moment, it appears there isn't a debian package for it. :-(

I presume it was Oscar who made the mods to gpio-common.c.in. Let's see what he has to say about root privileges and the legacy lamp code.

I'll put "fix the args on test.c" on my TODO list.

I have discovered in auto.defs a place where I think we should differentiate gpio vs. non-gpio platforms. There is current code there that alleges to flag pi vs. non-pi operation and detects the broadcom periphal interface that the new pinctrl and gpio-common.c.in stuff seems to have abandoned:

# Check for libraspberrypi-dev stuff
set old_LIBS [get-define LIBS]      ;# don't want LIBS modded directly
if {![file isdirectory "/opt/vc"]} {
    msg-result "WARNING: Not building on a Pi or libraspberrypi-dev not installed!"
    define PI_CFLAGS ""
    define PI_LFLAGS ""
    define PI_LIBS   ""
} elseif {![cc-with {-cflags "-L/opt/vc/lib"} {
             cc-check-function-in-lib bcm_host_get_peripheral_address bcm_host
           }]} {
    user-error "Found /opt/vc but cannot link to -lbcm_host!"
} elseif {![cc-with {-cflags "-I/opt/vc/include"} {
             cc-check-includes bcm_host.h
           }]} {
    user-error "Found /opt/vc but cannot #include bcm_host.h!"
} else {
    define PI_CFLAGS "-I/opt/vc/include"
    define PI_LFLAGS "-L/opt/vc/lib"
    define PI_LIBS   "-lbcm_host"
}
define LIBS $old_LIBS

I think, to properly integrate pinctrl, we need to get rid of the bcm_host stuff, to detect we're on the pi, either by looking for libraspberrypi-dev or for matching other autosetup variables: UNAME_SYSTEM = Linux AND UNAME_MACHINE = arm* seemed available. I'd suggest then doing #DEFINE BUILD_GPIO 1, and if it's not 1, have Makefile not even try to build pinctrl, and to conditionalize gpio-common.c.in to chop out most of the stuff it currently tries to do, but really shouldn't.

I will note in passing that the pinctrl code that uses reallocarray does get a warning about an implicitly defined function (which is a fatal error on the Mac), and so if it does exist, and is found the proper conditionals to pull it in from malloc.h have not actually been set.

And with this, I am going to try and stop thinking about this stuff until someone chimes in with some feedback on my ideas here. I'd very much like to hear what Warren and Oscar have to say.

I've just checked to the pi5 branch changes that:

1. Detect in auto.defs whether or not we are building on a Linux Pi platform.

2. Build all the relevant gpio stuff on the Linux Pi platform.

3. Carefully do not build anything that requires pidp8 hardware if we're not on the Pi platform. This means that scanswitch, pidp8i-test, and deeper won't even try to build since they require the pidp8i hardware. I put a bunch of #ifdef #endif constructs into our SIMH/PDP8/pdp8_cpu.c to quit even compiling code that uses the pidp8i hardware. (Made VERY easy by the quite nice demarcation labels. THANK YOU to whoever did that!) This avoids the problem of Apple clang not liking the construct used to define the devices, and the problem of the non-standardized reallocarray use.

4. Builds and runs bin/pidp8-simh on both my Mac and my Pi3b.

5. Updates to the latest upstream pinctrl stuff. Root access confirmed not to be required.

6. Cleans up the integration a bit.

7. Copied the license into our tree.

A couple little TODOs:

1. Document the force-pidp8i boolean option to force creation of the gpio stuff on non-arm/Linux platforms.

2. Point to the License from the our top-level COPYING.md.

3. Fix misc/test.c to send "1" not "test program" in the call to start_pidp8i_gpio_thread

---

Although a pinctrl package would be vastly preferable, that's not available as such as of yet. I think as a temporary measure, if we can confirm that this code works on all Pi platforms, it's worth the effort to vendor it ourselves.

---

LED light-up in pidp8i-test does not work and I haven't been able to figure out why. I understand more about how the legacy direct access to ledstatus works. I don't see a problem. That method just writes to one side of the double-buffered display -- the "cur" display.

Maybe there's something in the gpio thread startup that differs.

Maybe cur isn't actually the current, live, display buffer now?

I've run out of time and brain cells to devote to this for a while. But I'll pick it back up if anyone can offer insight into why the LEDs don't light up for pidp8i-test.

In my opinion this failure is a blocker to taking the proposed changes.

Made VERY easy by the quite nice demarcation labels. THANK YOU to whoever did that!

You can find out with "fossil blame" on the file in question, up on trunk.

I'm not sure what labels you refer to, but if we're talking about the minimization of the SIMH CPU core side of our patches and the isolation of the PiDP-8/I specific stuff in src/pidp8i/*, then that was me. Compare it to the original upstream sources.

This in turn is likely part of the reason why Oscar was having such trouble doing the integration himself: this tree isn't organized the same way as his, preventing a simple copy-over. As you have seen, there's a good reason for the reorg.

Root access confirmed not to be required.

Interesting. I wonder if that's true for Pi 4 and Pi 5. It could be that Pi 3 continues working as before, but one of the two new platforms is where root access becomes required.

If so, it won't be a trivial thing to restore root compatibility. I changed a bunch of stuff to make this software run rootless.

it's worth the effort to vendor it ourselves.

Yes. We vendor SIMH and Autosetup, after all. It isn't a black-and-white decision. Given a good-enough reason and a compatible license, it's fine.

At the same time, as soon as "libpinctrl" appears in the Raspberry Pi OS package repos, we should pluck our copy right back out.

this failure is a blocker to taking the proposed changes.

Yes, but I don't expect it to take you much time to work out where the mismatch lies.

I can't do it myself. My interest in the PiDP-8/I has ebbed considerably, to the point that the last time it roached its SD card, I didn't bother rebuilding it. I could bring it back to life easily enough, but I lack the will to bother.

I'll continue hosting the development platform, but there's an excellent chance I'm done working on it myself.

Consider this thread an attempt to extend the bus factor of this project beyond "1". 😉

In the simh4 branch, I have integrated the latest Open SIMH epoch. It was easier than I expected.

I expect I'll merge it into the trunk soon.

I've merged the Open SIMH code from the simh branch into trunk and successfully tested it on my Mac and Pi3b PiDP-8/i platform.

Heinz-Berndt Eggenstein provided the key insight into getting pidp8i-test working with the new pinctrl stuff. Theres a bit of sanity checking, but that fix should be getting published to the pi5 branch very soon. Looks like it should slide right into trunk now without problems.

Note also, I've pulled in Open SIMH in from trunk. We should use Open SIMH as our baseline going forward.

Heinz-Berndt has identified a performance issue with the pinctrl integration that wasn't even on my radar:

On his Pi2b, the load goes to 150% when OS/8 is idling. I think this is unacceptable. I want the new code to be useful on legacy platforms, not to only support the beefiest ones.

Heinz-Berndt suggests one possible source is the replacement of bit-banging in-line macros with procedure calls.

And a bit more investigation leads me to believe the pinctrl performance isn't as bad as I feared.

I chose to gather my own performance measurements on my Pi3b system:

In one terminal window I started pidp8i-sim, booted into OS/8 and let it go idle. In another window I ran top.

Sure enough, the pidp8i-sim job said it wanted 141% of the CPU, within striking distance of what Heinz-Berndt reported (which was 150%).

But going to the old pidp8i-sim from trunk, the job reported 138% CPU.

I think what really may be happening is that the Incandescent Light Simulator may be being the hog.

Confirmed.

Linking against nls instead of ils caused the CPU utilization of pidp8i-sim as reported by top to drop to 100% for both the pinctrl and the current gpio interface.