CD functions without the system card

[elmer]

The ring buffer very much exists, but only if you write the decode to use a ring buffer instead of a sliding window. The two work slightly differently. I modified Pucrunch's sliding LZ window to a ring buffer function so that I could decompress large chunks of data to vram without needing to decompress the whole thing into ram first (which I could for hucard projects because of the ram limitations). The ring buffer concept didn't exist inside the original decompressor.

I wasn't sure if you were one of those folks that still believed that the ring-buffer was a integral part of the LZSS algorithm ... which, it seems we actually both agree, it isn't. 

It is an "implementation-detail", a method of keeping the history window in a fixed 4KB location as a work-around for when you don't have easy random access to the window through normal pointer manipulation.

As such, I totally agree that the method may have its place, when used by someone that understands the trade offs.

I just have some difficulty imagining how you'd write a good inner-loop that can deal with the window wrapping.

Are you using self-modifying "abs,X" or "abs,Y" to address the window in order to avoid "[zp],y"? 

I rarely ever need to decompress anything LZ level compression to ram. Almost all assets are vram bound (tiles, sprites, but not tilemaps).

Yes, that makes sense. It all depends upon your needs and your code design.

I suspect that Falcom's usage of so many compressed code segments may not be typical.

I did take a quick look at Anearth to see what it was doing, and found Hudson using a lot of compressed text and graphics segments, with multiple decompression runs for a single text-box.

Yuk! 

They were also using the preloaded ring-buffer technique that you have mentioned before, so I finally got to see that in action.

Given the values that they were preloading it with, and the time taken to do the preloading, I'm absolutely amazed that they thought/found that the benefits outweighed the costs.

[elmer]

Time to resurrect this old thread!

I'm pretty close to having the fast CD code from Xanadu 2 completely disassembled-and-commented so that it can be included as source code in homebrew games.

It's got replacements for the BIOS functions AD_RESET, AD_TRANS, AD_CPLAY, AD_STOP and CD_READ.

It's approx 2KB of code, and I can see some wasted space in there that could be optimized away to make it smaller ... for instance, there's still code in there to read from Hudson's Development Kit SCSI.

This opens up the possibility completely disabling the CDROM BIOS, taking over the whole system, and mapping RAM into bank 7 instead of ROM.

Does that idea interest the other programmers here?

What other BIOS functions would you really need (in practice) during a game's execution?

Certain things like CD_DINFO and CD_BASE come to mind ... but they could easily be done at boot time and the results could be cached before switching to the "replacement" BIOS.

Does anyone ever use things like CD_SEEK, CD_SEARCH, CD_SUBRD, or CD_PCMRD?

Wasn't there a mention of someone having source code for some BM_* backup memory replacements?


<EDIT>

I guess that there's also he question of AD_READ and AD_WRITE.

Has anyone found a practical use for those since the Super System Card came out and cured the worst of the PCE's RAM problems?


<EDIT2>

D'oh!    Of course ... AD_READ does still have a potential benefit, in that it can be used for background streaming of data off the CD.

It *might* just be faster to read a sector or two at a time from the CD directly ... or it might not.

Perhaps someone else has investigated the HuVideo player and knows how it handles its streaming?

[elmer]

The disassembly/analysis of the Xanadu2 CD fast code is now done, and it has been tested to confirm that it actually assembles correctly and matches the original code in the game.

So now it's on to reverse-engineering some of the missing functions from the CD BIOS, and then doing some optimization on the results.

I just took a look at the bank of auto-disassembled code that Bonknuts posted from Seiya Monogatari earlier in the thread.

Just to make sure that everyone understands ... the work that I've done is the next (and very laborious) step beyond that kind of automated disassembly.

The Xanadu 2 code has been investigated and commented, and is fully-ready to be modified and changed.

A quick look the the Seiya Monogatari code shows that it's based on the same Hudson source-code.

For instance this routine, from my Xanadu 2 disassembly, is clearly identifiable in the Seiya Monogatari code ...

Code: [Select]

;
; Send SCSI "Request Sense" command (used to clear drive's error status).
;

_retry_recovery:jsr     _stop_scsi_bus          ; Release the SCSI bus.

_error_recovery:jsr     _init_scsi_bus          ; Acquire the SCSI bus.
                bcs     _retry_recovery

.test_scsi_bus: ldx     #$14
                cly
.wait_scsi_bus: jsr     _get_cd_status
                and     #$80
                bne     .proc_scsi_loop
                dex
                bne     .wait_scsi_bus
                dey
                bne     .wait_scsi_bus

                ; Timeout, retry.

                bra     _error_recovery

                ; Process SCSI states.

.proc_scsi_loop:jsr     _get_cd_status
                cmp     #$d0
                beq     .send_command
                cmp     #$c8
                beq     .read_reply
                cmp     #$d8
                beq     .command_done
                bra     .proc_scsi_loop

                ; Send SCSI command.

.send_command:  lda     #< _request_sense
                sta     <_bl
                lda     #> _request_sense
                sta     <_bh
                jsr     _send_scsi_cmd
                bra     .proc_scsi_loop

                ; Read SCSI reply.

.read_reply:    lda     #< scsi_recv_buf        ; Buffer for result.
                sta     <_bl
                lda     #> scsi_recv_buf
                sta     <_bh
                lda     #$0a                    ; # of bytes to read.
                sta     <_al
                stz     <_ah
                jsr     _get_scsi_reply
                bra     .proc_scsi_loop

                ; SCSI command done.

.command_done:  jsr     _get_exit_code
                cmp     #$00
                bne     _error_recovery

                lda     scsi_recv_buf + 9       ; Last byte of SENSE reply is either
                rts                             ; an error code or a retry count.

;
; Use return code from _error_recovery to decide what to do.
;

_pick_next_base:cmp     #$00
                beq     .try_swap_track
                cmp     #$04
                bne     .skip
                bra     .try_swap_track
.skip:          cmp     #$0b
                beq     .try_same_track
                cmp     #$0d
                beq     .try_same_track
                cmp     #$11
                beq     .try_swap_track
                cmp     #$15
                beq     .try_swap_track
                cmp     #$16
                beq     .try_swap_track
                cmp     #$1c
                beq     .try_same_track
                cmp     #$1d
                beq     .try_same_track
                cmp     #$20
                beq     .try_same_track
                cmp     #$21
                beq     .try_same_track
                cmp     #$22
                beq     .try_same_track
                cmp     #$25
                beq     .try_same_track
                cmp     #$2a
                beq     .try_same_track
                cmp     #$2c
                beq     .try_same_track

                ; Swap to alternate data track.

.try_swap_track:lda     #$01
                sec
                rts

                ; Stay on current data track.

.try_same_track:cla
                clc
                rts


[touko]

I'd love to see tuokos chuck no-rice game with loadable characters to use

It's already here for the two players mode 
It works fine,but i don't know if it's slow or not, only tested on mednafen,you'll see it soon with the demo version .

[Bonknuts]

It *might* just be faster to read a sector or two at a time from the CD directly ... or it might not.

Perhaps someone else has investigated the HuVideo player and knows how it handles its streaming?

 From what I remember, and it's been years since I looked at it, it's reading directly from the CD data port. It writes updates to ADPCM ram, and after it reads out data from the CD port, it does a few things. And returns back to polls the CD status.

 The strange thing I found, was the very tail end of the data block (2k I think?) is read out much slower than reading out the rest. I suspect because the MCU interface must be triggered at some point (by reading the CD port X number of times/bytes) and starts filling in the buffer from the front. It's only speculation, but I can't figure out why the last bit of data is read out slower. Some tests should be done, but I'm suspecting this is how the next sector read from the CD is triggered. 

 elmer: Do you have the disassembly of the system card?

[elmer]

elmer: Do you have the disassembly of the system card?

I've got the one that Dave Shadoff and Oliver Jolly (NeoGrad) did ... do you have anything more complete than that?

It's a great starting-point, but putting *meaning* to the code is still a laborious process.

[TheOldMan]

It's a great starting-point, but putting *meaning* to the code is still a laborious process.

What version of bios is that?
Would it help to have an over-commented, re-assembleable version using names?
If so, I'll zip the files I have for banks 0-3, along with matching variable definitions, and mail them to you.
There's still stuff in them I don't completely understand, but most of it I got. I just wish I understoof the irq stuff better, like why there are 2 irqs for the adpcm stuff. (Top half/Bottom Half?)

PM me with an email address if you want it.

[elmer]

What version of bios is that?

It's version 3.00 ... which is the Japanese SuperCD BIOS IIRC.

Would it help to have an over-commented, re-assembleable version using names?

Absolutely!

I'm about 75% of the way through labeling/commenting the BIOS CD/ADPCM routines that aren't included in the fast CD code that I've already done, but it would be wonderful to have someone else's version of the same stuff to compare notes with.

As annoying as it's been ... I really did need to do it myself in order to get a good handle on the differences between the BIOS code and the fast-read code. That's the only way that I'll be able to merge the two methods together in order to save space in a new ROM-replacement library.

Having all the extra functions available that you've already done, and that I've not even looked at, may well open up the potential for that BIOS-compatible replacement that I seem to remember either you or Bonknuts talking about.

Personally, I'd just like to get to where I can have a fully-functional fast CD/ADPCM library that lives in RAM in $F000-$FFFF, and that leaves the rest of that bank usable by a game.

I just wish I understood the irq stuff better, like why there are 2 irqs for the adpcm stuff. (Top half/Bottom Half?)

Hudson's fast-cd code only uses a single IRQ hook for the AD_CPLAY function. You'll get a better idea when I'm finished and it's all tested, and I can put it up on github or somewhere else.

[TheOldMan]

It's version 3.00 ... which is the Japanese SuperCD BIOS IIRC.

Okay, I have the backup files removed, and it's all zipped up. What's the best way to send it to you? (Please don't post e-mail here. PM it, if needed)
It may take a day or two; I have to figure out how to do it in gmail. (no, I seldom use e-mail so I don't remember all the gotchas; I vaguely remember I can't send a .zip, though)

This is Bios 3.01 - region desn't matter, they are just bit-flipped. Only difference is 3.01 has a bunch of nop's at boot to allow reset to settle, i presume. Other pages beyond the boot page should match. You can see where I left off, somewhere in bank 3 with the grp functions...

I really did need to do it myself in order to get a good handle on the differences between the BIOS code and the fast-read code.

Understood. It's one thing to read the code and say "It does this". It's an enirely different matter to disassemble the code and say "It does this because....".
My goal is to split bios apart; I'm 99% sure a lot of it is 'library' code, used across multiple games with little change. I'd like to bring that across to Huc, so we can build cd/Hucard versions against the same library code. Hopefully, we can then get rid of the HuC library stuff when we need the space in a cd game, and still test it as a HuCard.


And before I forget: What setup are you using to re-build Huc? I need to know what os/tools I need just in case...

[elmer]

And before I forget: What setup are you using to re-build Huc? I need to know what os/tools I need just in case...

I use msys2/mingw-w64 to build it. It's a modern 32bit and 64bit replacement for the old (and out-of-date) mingw project.

You can find it here ...

https://sourceforge.net/projects/msys2/

I've written some batch files that make the initial setup and updating easier, and they're being used by the guys that are using my V810 GCC compiler patches for the VirtualBoy.

I really need to get them put up on github, but there always seems to be another "priority".

I can send you those if it would help.