Formal way to launch code after reset

[dansanderson]

We currently tell people to avoid touching KERNAL internal variables because, in theory, we don't know what will need to change to make desireable improvements, and we have the opportunity to add accessors. Currently, Etherload depends on the "top of BASIC" variable (the end of a loaded BASIC program) and the BASIC init jump table at ROM $2000 to manually load and run a PRG in lieu of a formal routine. Others, such as the Intro Disk menus, may have use for this as well.

One option would be to add a formal routine to support this and keep the internals private. This would probably involve coupling the KERNAL and BASIC modules, but that's probably ok.

I'm ok with declaring a few specific vars as public APIs. Costs include 1) no future way to wrap access, 2) must support being updated between any two instructions, 3) ROM code must be documented and instrumented (.assert directive) to prevent accidental breakage. Benefit is that existing Etherload code doesn't have to change and uses the same mechanism for older ROM versions (though because it's first-party it's probably OK for it to test for the ROM version and switch methods).

The BASIC init jump table at ROM $2000 is unlikely to change. it's not a nice API but it works for this obscure purpose to allow for running ML code after a ROM reset. It might be nice to provide a more formal API and continue to support the current Etherload method, for completeness.

[lgblgblgb]

Btw, for example Xemu also depends on similar things to be able to "auto-inject" programs without user interaction to the emulated machine (ie, from menu, or from CLI, or drag-and-dropping a file into the emulator window). What I do is to adjust pointers $AE/$AF and and $82/$83. Also, in this case I do a RESET and checking for READY. appeared on the screen (I check VIC settings to know where is the screen and what mode it is in) then placing "RUN" into the video RAM directly (just below the sensed READY. place) and then simulating a keypress of RETURN. This is a mess, but I need to be compatible with about every ROMs (maybe even C65 ROMs) also with GO64 mode (though then the pointer needed to be adjust is a bit different).

This is indeed a complicated and error-sensitive process, however I cannot rely on things like VICE/C64 emulator where the ROM is patched, or at least checked for a certain PC value, since we have no "single ROM".

I'm not sure if the situation can be different from view point of Xemu with the plans here, though.

[dansanderson]

@lgblgblgb You're right to call out this use case. A formal API for this is all we'd need to support Xemu cleanly. There is already a formalism for detecting the ROM version, so Xemu could use the current method for ROMs older than some new API, and the new API going forward. I suggested this for Etherload, and @ki-bo points out the Etherload is perhaps too space constrained to support two methods and a ROM version check, so we'll probably support at least $82/$83 as a fixed point. (I would put BASTOP on the short list of C64 KERNAL vars with useful external applications.)

$AE/$AF are two bytes in the middle of a multi-purpose scratch variable for DOS (eal), so I'm more nervous about making promises about it. How does Xemu use this? Perhaps it's a suitable "input" register for some entry point, but we'd have to formalize that. This is a good example of how defining behavior around KERNAL internal variables can quickly get out of hand, and we'd be better off building robust APIs through accessors and entry points, even if it means a bit of up-front hassle for migrating legacy use cases.

[lgblgblgb]

@dansanderson ROM version detection is not a problem, Xemu detects ROM version already on start-up/reset, even the SHA256 checksum is calculated if it's useful to check very precisely (other than the "version/date string" at the beginning). It also checks if it's open ROM, if someone uses that, etc. This info is also used to set the default DMA mode, though it's very rare nowadays that someone needs the older DMA revision, it's maybe only with C65 ROMs and even then only the older ones, but that was the original motivation of mine to determine the ROM version by the emulator at least.

This is what I do currently, not sure if it's 100% correct, but at least seems to work (the C64 mode is for the case when the loaded/injected program has start address of $801 indicating C64/GO64 mode material rather than native MEGA65):

            // We must modify BASIC pointers ... Important to know the C64/C65 differences!
            if (prg.c64_mode) {
                    main_ram[0x2D] =  prg.size + prg.load_addr;
                    main_ram[0x2E] = (prg.size + prg.load_addr) >> 8;
            } else {
                    main_ram[0xAE] =  prg.size + prg.load_addr;
                    main_ram[0xAF] = (prg.size + prg.load_addr) >> 8;
                    main_ram[0x82] =  prg.size + prg.load_addr;
                    main_ram[0x83] = (prg.size + prg.load_addr) >> 8;
            }

As far as I remember, it was BitShifter who gave me the hints what should be adjusted in MEGA65 mode.

Btw, I really think, some ZP locations used by kernel/BASIC should be made "official" and "fixed" if we feel it's safe to say that for a given location. However indeed it can cause problem in the future when it turns out a change would be needed. This is an issue as - I think - most ppl treat a 8 bit platform as a "digging in" architecture (compared to modern OSes) when it's OK to do low level modifications here and there eg on variables used by the system. Even if we say "hey! that's not safe, pls do not do that!". But maybe at least some stuff could/should (???) be made official and fixed part of the official public API/ABI or whatever. For example in this topic, I wouldn't think there is a need to ever change the meaning of AE/AF and 82/83 ZP locs meaning, however who knows maybe in the future even if these can be fixed, some BASIC routine should be called to really know the change of those vars, or whatever. Bit lame example, but still.

[ki-bo]

Thinking about this further, I think I like the idea of having a formal API in kernal available that allows to RUN a program injected into memory. Probably by providing just the size of the program, the kernal routine would then initialise all needed internal state and execute it. I guess this would still fit into Etherload, and I would just keep supporting older ROMs knowing that their ZP locations were stable with the former way of doing it.

[dansanderson]

I'm very familiar with the interest in hacking into ROM internals for recreation, and I understand why people want it. It's diametrically opposed to supporting software that runs across multiple ROM versions. I get that some people would rather have ROM hacking be fully supported than have the ROM receive updates (the "freeze the ROM" crowd), but there are other people that value the potential for ROM updates more highly, and we can't make both groups happy.

I also believe some people want access to ROM internals just because they're used to this being a documented way to program the C64, and what they actually want is a documented API surface on which to build. When someone asks for the "memory map" (by which they mean documentation about KERNAL internals), I ask them why they're asking, to clarify this distinction. For this group, freezing the ROM is an extreme solution to their problem. Yes, making specific KERNAL variables public is one way of building such an API, but this is difficult and brittle. If you just want an API that does a thing, an accessor that does the thing is a much better solution, pretty much always.

To wit: I believe Xemu writing to $AE-$AF does nothing in recent ROMs. These are the middle two bytes of a 32-bit address used internally by some disk routines, and it doesn't make sense to set them this way. If there was ever a reason to do that, either the reason no longer exists or it's some other location now and nobody knows. Xemu would be better served by a KERNAL routine it can call to RUN an injected program.

kibo and I are also discussing a RESET+RUN KERNAL routine that should be plausible and useful. This would parameterize the reset process such that it ignores cartridge bootstrap at $8004, skips loading AUTOLOAD.C65, doesn't perform a NEW on BASIC memory, and performs a RUN after reset is complete (as if AUTOLOAD.C65 were loaded). With this, Etherload and Xemu can simply install a PRG at $2001 and call this routine, without touching any other internals. A separate "RUN" KERNAL call would also allow for faster simpler cases where a reset isn't necessary.

[lgblgblgb]

No idea why I got that AE/AF from, I guess it was told by Bitshifter maybe. https://65site.de/downloads/c65_rom_memory_map_20201109.pdf This also indicates similar function for those addresses, though I am not sure if this map is correct at all (and surly it's about BASIC10 not BASIC65 ...). Besides this, I should keep all the stuff needed to modify, as some ppl may try to use C65 ROM with MEGA65 emulator: yes, does not make too much sense, but ... On the longer term, probably a ROM version based solution can be applied though by Xemu at least, so using some of the KERNAL stuff you mention for new enough ROMs.

[dansanderson]

sal and eal were widened to 32-bit to support loading and saving data from any RAM location. Prior to formalizing this support via SETBNK et al., there was a single Discord post years ago recommending that programs write to eal directly when calling disk routines with wide addresses. I'm sure leaving this undocumented made it easy to forget that he told you to write directly to eal in a different way. Luckily, I don't think it matters. I can't think of a reason Xemu would need to set eal for program injection—at least not in recent ROMs.

[lgblgblgb]

sal and eal were widened to 32-bit to support loading and saving data from any RAM location. Prior to formalizing this support via SETBNK et al., there was a single Discord post years ago recommending that programs write to eal directly when calling disk routines with wide addresses. I'm sure leaving this undocumented made it easy to forget that he told you to write directly to eal in a different way. Luckily, I don't think it matters. I can't think of a reason Xemu would need to set eal for program injection—at least not in recent ROMs.

Yeah, do not misunderstand me, I don't say it's needed today, but I still know ppl want to try C65 ROMs and other very old MEGA65 ROMs with the emulator (even if it does not make too much sense for me), so I don't want to stop support those. However - as I've said - in the future, with some API, I will opt to use that if I detect recent enough ROM, what I wanted to say, and only stick with the direct ZP modification method for older ROMs.

[dansanderson]

Use cases for a formal RESET+RUN call are looking compelling for the Intro Disk menu project, especially if it can help reset some hardware registers to their boot state. Note that the ROM warm boot doesn't reset every register in the system, and some cases will still rely on a VHDL cold boot for edge cases. But we can evaluate the current behavior and possibly extend it with more resets if needed. (We don't have a strict definition of what the warm boot does and does not reset.)

It sounds like the Intro Disk use case would benefit from some control over post-reset disk mounting. The MOUNT command has a local mechanism for post-warm boot persistence, but this is not exposed to ML programs. Earlier I suggested a RESET+RUN that executes the program in BASIC memory after a reset, skipping the NEW that typically happens on warm boot. With this design, the Intro Disk could install a small bootstrap BASIC program with a MOUNT and a DLOAD command, which gets RUN after reset. This would be the general mechanism by which a program can perform arbitrary BASIC commands after a reset. This would exclude certain direct mode only commands, but they're out of scope for the use cases we've discussed so far.

Another option might be a parameter to RESET+RUN that points to a null-terminated string <160 chars containing PETSCII characters to auto-type at the READY prompt after a reset. This would spare the caller from having to tokenize the commands into program memory, and wouldn't have an exception for direct mode only commands. Maybe this also skips NEW on reset, such that a caller could still install a PRG into memory and auto-type RUN to run it after reset.

Anything else we'd need to consider for Intro Disk or Xemu launcher use cases?