Author Topic: Xanadu II Translation Development Blog  (Read 39453 times)

elmer

  • Hero Member
  • *****
  • Posts: 2153
Re: Xanadu II Translation Development Blog
« Reply #120 on: November 10, 2015, 06:48:39 AM »
As for this hack ... I'm happy to say that the script-assembler is now approx 3,000 lines of C code, and is verified to be compiling the original scripts with 100% accuracy, including all the script parameters and address fix-ups.  :)

Now that it's basically "done", and relocating scripts as the English translations make them overflow their original space, it's approx 3,500 lines of C code. Phew!  ](*,)


SamIAm + elmer =

Hahaha!   8)

I think that SamIAm is the metal dude ... and I'm just full of the "hot air" in the background.  :wink:


Oh wow, you got a font hack going already! MUCH BETTER! :)

I'm happy with the results ... but it's an embarrassingly small hack, just like Team Innocent.

It's really just filling the game's glyph buffer with my font data instead of the ROM font data, and then modifying the cursor increment.


; ***************************************************************************
;
; Get the 8x12 ASCII glyph data (replaces call to EX_GETFNT).

bank_font equ $84

get_acsii_glyph:
        lda   <_al
        sec
        sbc   #$20

        stz   <_bl
        asl   a
        rol   <_bl
        asl   a
        rol   <_bl
        sta   (.smod + 1)
        ldy   <_bl
        asl   a
        rol   <_bl
.smod:  adc   #$00
        sta   (.loop + 1)
        tya
        adc   <_bl
        adc   #high($a000)    ; Font is 256-byte aligned, at $a000.
        sta   (.loop + 2)

        tma   #$05
        pha
        lda   #bank_font
        tam   #$05

        ldy   #11             ; Font data is 12-bytes (0-11).
        ldx   #22             ; Copy it as 16-bit wide.
.loop:  lda   $0000,y
        sta   $280a,x
        stz   $280b,x
        dex
        dex
        dey
        bpl   .loop

        pla
        tam   #$05

        rts


; ***************************************************************************
;
; Increment the text cursor by one or two 4-pixel steps.

inc_text_cursor:
        lda   <_ah            ; Hi-byte of glyph code from EX_GETFNT.
        bne   .double
        lda   <_al            ; Lo-byte of glyph code from EX_GETFNT.
        bmi   .double
        sec
        sbc   #$20
        bcc   .double

        tax
        lsr   a
        lsr   a
        lsr   a
        sax
        and   #$07
        tay
        lda   idx2msk,y
        and   tbl8x12,x
        beq   .single

.double:inc   $2805
.single:inc   $2805
.done:  rts

idx2msk:.db   $01, $02, $04, $08, $10, $20, $40, $80

tbl8x12:.db   $7D ; 01111101
                  ; '&%$#"!

        .db   $AF ; 10101111
                  ; /.-,+*)(

        .db   $FF ; 11111111
                  ; 76543210

        .db   $F3 ; 11110011
                  ; ?>=<;:98

        .db   $FF ; 11111111
                  ; GFEDCBA@

        .db   $FD ; 11111101
                  ; ONMLKJIH

        .db   $FF ; 11111111
                  ; WVUTSRQP

        .db   $FF ; 11111111
                  ; _^]\[ZYX

        .db   $FE ; 11111110
                  ; gfedcba`

        .db   $E9 ; 11101001
                  ; onmlkjih

        .db   $FF ; 11111111
                  ; wvutsrqp

        .db   $FF ; 11111111
                  ;  ~}|{zyx



Finally! I can't retire from all things translations. I've waited so long... ;_; ... . . :mrgreen:

Haha ... you don't get off that easily!  :wink:

Just like you, I'd much rather be writing new code than hacking around in someone's old code.

I'm hoping that you'll want to do Anearth Fantasy Stories with SamIAm when you've got the time.

The community needs to keep him busy ... it's rare to find someone with his combination of passion and capability.  :D

elmer

  • Hero Member
  • *****
  • Posts: 2153
Re: Xanadu II Translation Development Blog
« Reply #121 on: November 10, 2015, 07:59:48 AM »
Well, later games did a bunch of things with fonts:

- Koei (and others) made their own font instead of using system font

- lots of games started using multiple different fonts in different spots

- hang-shadows were used in lots of games, but not necessarily on all text (Tokimeki Memorial seems to have this).  This is done by taking the orginal font, shifting it right one pix and down one pix, and printing it in black. Then white (or whatever color) on top of that, without shift.  All on transparent background, so both the black and foreground colors are apparent.

- I think Riverhill (Gunbuster) might have just doen a bold effect on some text by shifting right and blending.

- several companies all added large amounts of text as graphics splashes instead of font/text.

Thanks for the list.  :)

I'm not really talking about custom fonts or text that's built into graphics ... what really interests me is developers that did unusual "real-time" processing of the font data.

"Hang-shadows" aka "drop-shadows" were the most common effect that's used almost-everywhere.

"Outlines" are another common and easy-to-do effect ... especially when the text is a layer on top of a moving background.

I'll have to take a look at Gunbuster and see what it's doing.  :-k

Xanadu 1's "bold" really sparks my curiosity because someone had the smart idea of how to preserve the small details in the font.  :clap:

Take a look at these examples ...


     Original    Double      Xanadu      Outline

01   .........   .........   .........   .OO......
02   .X.......   .XX......   .XX......   O..O.....
03   .X.......   .XX......   .XX......   O..O.....
04   .X.......   .XX......   .XX......   O..OOOO..
05   .X.XXX...   .XXXXXX..   .X.XXXX..   O.O....O.
06   .XX...X..   .XXX..XX.   .XXX..XX.   O...OO..O
07   .X....X..   .XX...XX.   .XX...XX.   O..OOO..O
08   .X....X..   .XX...XX.   .XX...XX.   O..O.O..O
09   .X....X..   .XX...XX.   .XX...XX.   O..O.O..O
10   .X....X..   .XX...XX.   .XX...XX.   O..OOO..O
11   .XXXXX...   .XXXXXX..   .XXXXXX..   O......O.
12   .........   .........   .........   .OOOOOO..

     Original    Double      Xanadu      Outline

01   .........   .........   .........   ......OO.
02   ......X..   ......XX.   ......XX.   .....O..O
03   ......X..   ......XX.   ......XX.   .....O..O
04   ......X..   ......XX.   ......XX.   ..OOOO..O
05   ..XXX.X..   ..XXXXXX.   ..XXX.XX.   .O...O..O
06   .X...XX..   .XX..XXX.   .XX..XXX.   O..OO...O
07   .X....X..   .XX...XX.   .XX...XX.   O..O.O..O
08   .X....X..   .XX...XX.   .XX...XX.   O..O.O..O
09   .X....X..   .XX...XX.   .XX...XX.   O..O.O..O
10   .X....X..   .XX...XX.   .XX...XX.   O..OOO..O
11   ..XXXXX..   ..XXXXXX.   ..XXXXXX.   .O......O
12   .........   .........   .........   ..OOOOOO.



Xanadu's "bold" processing preserves the blank "." pixel where round part of the "b" and "d" join the stem on line 05.

That really helps to preserve the roundness of the original glyph and IMHO is vital to making the "bold" effect look good.

When you look carefully at the difference between the "b" and the "d", you can analyze how they must be doing it.

From what I can see, they're shifting the original data right by 1 pixel and then "or"ing it back in to produce the "double" version.

Then they're looking for "X.X" sequences in the original data and making sure to clear any "." pixels like that from the "double" version.

That's probably as simple as ...

        lda   font_data,y     ; Double the font data.
        lsr   a
        sta   (.and + 1)
        or    font_data,y
        sta   (.msk + 1)

        lda   font_data,y     ; Mask out the "0" pixel
        asl   a               ; data in "101" sequences.
.and:   and   #$00
        eor   #$ff
        or    font_data,y
.msk:   and   #$00
        sta   bold_data,y


Now I'm going to actually take a look at their code and see if they're actually doing it any more efficiently than that.  :wink:
« Last Edit: November 10, 2015, 05:52:34 PM by elmer »

Bonknuts

  • Hero Member
  • *****
  • Posts: 3292
Re: Xanadu II Translation Development Blog
« Reply #122 on: November 10, 2015, 11:56:59 AM »
Finally! I can't retire from all things translations. I've waited so long... ;_; ... . . :mrgreen:
Haha ... you don't get off that easily!  :wink:

Just like you, I'd much rather be writing new code than hacking around in someone's old code.

I'm hoping that you'll want to do Anearth Fantasy Stories with SamIAm when you've got the time.

The community needs to keep him busy ... it's rare to find someone with his combination of passion and capability.  :D

 SamIam is probably more valuable than any of us hackers. It's hard to find a good translator that wants to do PCE translations. Everything in the translation scene is soo Nintendo centric, or counter-nintendo centric (sega stuff), or just plain out skips this generation (PS2, etc). When it comes to RPGS, it's like the PCE is the system no one wants to touch (outside of niche communities that surround it).

 To be honest, Dead of the Brain should be finished. It's almost there. The script was dumped, Dave made tools for script insertion, the font routine works (with some odd ball convoluted support needed to prop it up). And the translator has been sitting in this for years. I would hate to start anything new when other projects are so close.

 I always held this idea that a community could work on projects as a whole.. a team. Members might come and go, but if the translation process is organized enough - people can pick up where others have left off.

 Cosmic Fantasy I has a working font routine; Dave figured out the script format. That just needs a translation. Stuff like that.

SamIAm

  • Hero Member
  • *****
  • Posts: 1835
Re: Xanadu II Translation Development Blog
« Reply #123 on: November 12, 2015, 01:52:56 AM »
I don't know whether I am really more valuable than any hackers, but I am all in favor of being kept busy.

It really takes two to tango, unless you're some freak of nature who can hack and translate and find the time for both. I'm pretty useless by myself.

Also, morale is critical during projects like these, and knowing that the other guy is working his tail off too is critical. The fact that elmer has been doing amazing work motivated me to squeeze in four hours of translating today...and I hope that fact is motivating to elmer.  :wink:
« Last Edit: November 12, 2015, 02:00:59 AM by SamIAm »

elmer

  • Hero Member
  • *****
  • Posts: 2153
Re: Xanadu II Translation Development Blog
« Reply #124 on: November 27, 2015, 05:48:34 AM »
Also, morale is critical during projects like these, and knowing that the other guy is working his tail off too is critical. The fact that elmer has been doing amazing work motivated me to squeeze in four hours of translating today...and I hope that fact is motivating to elmer.  :wink:

Definitely!  :wink:

I've been feeling guilty that SamIAm has been working so hard on the translation, and that I've been "playing" with the Turbo Everdrive 2.

The reason is that I got the complete 1st-draft of the 1st-level of Xanadu 1 and tried to insert it into the game ... and it immediately ran out-of-memory on the very first script chunk in the game.

So that meant that I've got to implement the entire game-recompression and decompressed-script-overflow code before we can move forward.

Yuk! That's a lot of work.  #-o

That really brought up the question of whether we even have any good alternative to use if the translation needs extra RAM, or whether it would then be Mednafen-only (or CD Stupid Card for those of us lucky enough to have them).

Well, after a lot of messing around (as documented in various other threads), it looks like the Turbo Everdrive 2 is definitely going to be an alternative that people can actually go out and buy if there are any CD translations/homebrew in the future that need extra RAM above-and-beyond the 256KB that the Super System Card provides.

There's also the possibility of TheOldMan's 512KB ROM/RAM card that he's designing, or the mythical 1MB-RAM card that I've been thinking about ... but the Turbo Everdrive 2 actually exists, and can be purchased, and is a really nice card (if a little bit more expensive than some people would like).

So, with that concern out of the way, now it's time to get back to Xanadu!  :)

MNKyDeth

  • Hero Member
  • *****
  • Posts: 715
Re: Xanadu II Translation Development Blog
« Reply #125 on: November 27, 2015, 05:57:34 AM »
Was it stated earlier that the Arcade Card Duo/Pro are not viable options in this scenario Elmer? Or could we use those for those of us that have them?

spenoza

  • Hero Member
  • *****
  • Posts: 2751
Re: Xanadu II Translation Development Blog
« Reply #126 on: November 27, 2015, 06:16:03 AM »
Elmer has brought up in multiple threads, on multiple occasions, why the Arcade Card standard is not effective/appropriate for this.
<a href="http://www.pcedaisakusen.net/2/34/103/show-collection.htm" class="bbc_link" target="_blank">My meager PC Engine Collection so far.</a><br><a href="https://www.pcenginefx.com/forums/" class="bbc_link" target="_blank">PC Engine Software Bible</a><br><a href="http://www.racketboy.com/forum/" c

MNKyDeth

  • Hero Member
  • *****
  • Posts: 715
Re: Xanadu II Translation Development Blog
« Reply #127 on: November 27, 2015, 06:24:27 AM »
Elmer has brought up in multiple threads, on multiple occasions, why the Arcade Card standard is not effective/appropriate for this.

Ok, thank you.

I think I remember reading something on it but wasn't for sure.

I recently bought both of these games and have them now mostly because of this project. I also have a 2.4 TED enroute so I should be good to go hopefully soon. I am just excited for this project as I have always heard good things about the Xanadu games.

Black Tiger

  • Hero Member
  • *****
  • Posts: 11242
Re: Xanadu II Translation Development Blog
« Reply #128 on: November 27, 2015, 06:30:36 AM »
Elmer has brought up in multiple threads, on multiple occasions, why the Arcade Card standard is not effective/appropriate for this.

Ok, thank you.

I think I remember reading something on it but wasn't for sure.

I recently bought both of these games and have them now mostly because of this project. I also have a 2.4 TED enroute so I should be good to go hopefully soon. I am just excited for this project as I have always heard good things about the Xanadu games.

You can play just the action/boss sequences of both games through the debug modes in the meantime.
http://www.superpcenginegrafx.net/forum

Active and drama free PC Engine forum

elmer

  • Hero Member
  • *****
  • Posts: 2153
Re: Xanadu II Translation Development Blog
« Reply #129 on: November 27, 2015, 06:31:18 AM »
Was it stated earlier that the Arcade Card Duo/Pro are not viable options in this scenario Elmer? Or could we use those for those of us that have them?

I'm still hoping to avoid needing the extra RAM ... but Xanadu 2 is the game that's probably going to cause the most trouble.

That's because its FALCOM2 compression is already doing a good job, and so the amount of memory that I get back by switching to SWD4 or SWD5 is going to be less than with the FALCOM1 compression on Xanadu 1.

I think that I'm going to need quite a bit of extra space for the compressed translations, and I've also got to free up 8KB for the decompressed translation.

Let's just hope that everything fits, and that we don't need the extra memory.

elmer

  • Hero Member
  • *****
  • Posts: 2153
Re: Xanadu II Translation Development Blog
« Reply #130 on: December 01, 2015, 12:05:26 PM »
Time for an update ...

As you, Dear Reader, may have seen, the font hacks that I took screen grabs of a few weeks ago are now properly integrated into the CD on both Xanadu 1 and Xanadu 2.

That's taken a fair bit of behind-the-scenes messing around to actually locate/extract the different code overlays for both games and then put together a PCEAS framework to let me modify them all and then reinsert them into the CD image.

Part of that has involved the discovery of how both games actually locate the files that they're loading from CD.

***************

They're both using a 2KB "directory" structure that's loaded as part of the IPL sectors and is then kept permanently in memory.

They're then referring to files by a 2 byte code when they want to load them ...

The 1st byte is a file type "X" for executable, "Z" for compressed META-BLOCK, and "A" for something else (presumably audio).
The 2nd byte is a binary number, and seemingly refers to the game section.

For instance, in Xanadu 1, the "Main Menu" is actually made up of 3 files "X00", "A00" and "Z00".

It also makes it really clear that there's a single "game" overlay, and 12 different sets of META-BLOCKs for each level ...

"X10", "Z10", "Z11", "Z12", "Z13", "Z14", "Z15", "Z16", "Z17", "Z18", "Z19", "Z1A", and "Z1B".

It's an interesting discovery ... and I wish that I'd found it a long time ago.

Having a list of files like this makes it a lot easier to see how the game is put together.

It also shows that there's a lot of the game that we've not touched at all, yet!

***************

One nice side-effect of finding where the directory structure actually lives when it's loaded into in memory, is that there's enough free space in that bank for the English font, rather than just putting it in the space that I'm freeing up by recompressing the game data.

That's going to be really important on Xanadu 2, because the META-BLOCKs in that game just won't shrink down as much, and I'm already worried if there's going to be enough memory.

***************

Which leads on to the compression/decompression.

I've got a 1st-version of the SWD5 decompressor for the PCE, and I've managed to squeeze it down into being a few bytes shorter that the FALCOM1 decompression code, so it can just live in the same location and overwrite the old code.

I've also hacked the SWD5 format so that it can automatically decompress some data into a different block after it decompresses a script chunk.

That seems like the easiest way to get the game to allow SamIAm's translated script chunks to expand in size and overflow into the space that we're creating by recompressing all those game files.

Now that I've found the directory structure, I'm in the position of being able to rewrite all those old files in the new compression format ... and hope that it all doesn't just blow-up!  :pray:

That's the next task.

***************

Anyway, for anyone that's interested in a challenge, here are the directories for Xanadu 1 & 2.

See if you can guess which groups of files are for the Main Menu, the Cinemas, the Top-Down Game, the Side-Scrolling Game, the Boss fights, etc.  :wink:

***************

Xanadu 1 Files ...

X00, A00, Z00,
X01, Z01,
X02, Z02,
X03, Z03, A03,
X04, Z04, A04,
X05, Z05, A05,
X06, Z06, A06,

X10, Z10, Z11, Z12, Z13, Z14, Z15, Z16, Z17, Z18, Z19, Z1A, Z1B,

X30,
Z30, A40, A50, A60,
Z31, A41, A51, A61,
Z32, A42, A52, A62, A72, A82,
Z33, A43, A53, A63,
Z34, A44, A54, A64,
Z35, A45, A55, A65,
Z36, A46, A56, A66,
Z37, A47, A57, A67,
Z38, A48, A58, A68,
Z39, A49, A59,
Z3A, A4A, A5A, A6A, A7A,
Z3B, A4B, A5B,
Z3C, A4C, A5C,

X20, Z20, A20,
X21, Z21, A21,
X22, Z22, A22,
X23, Z23, A23, A33,
X24, Z24, A24,
X25, Z25, A25,
X26, Z26, A26,
X27, Z27, A27,
X28, Z28, A28,
X29, Z29, A29,
X2A, Z2A, A2A,
X2B, Z2B,


***************

Xanadu 2 Files ...

XF0, ZF0,

X10, Z90, Z0F, A0F,

Z00, Z01, Z02,
Z10, Z11, Z12, Z13, Z14, Z15, Z16, Z17,
Z20, Z21, Z22, Z23, Z24, Z25, Z26, Z27,
Z30, Z31, Z32, Z33, Z34, Z35, Z36, Z37,
Z40, Z41, Z42, Z43, Z44, Z45, Z46, Z47, Z48, Z49,
Z50, Z51, Z52, Z53, Z54, Z55, Z56, Z57,
Z60, Z61, Z62, Z63, Z64, Z65, Z66,
Z70, Z71, Z72, Z73, Z74, Z75, Z76,
Z80, Z81, Z82, Z83, Z84, Z85,

A15,
A20, A24, A25, A26,
A35,
A41, A42,
A51,
A60,
A70, A74, A75,
A82,

XA0,
Z98, A98,
Z99, A99,
Z9A, A9A,
Z9B, A9B,
Z9C, A9C,
Z9D, A9D,
Z9E, A9E,
Z9F, A9F,
ZA0, AA0,
ZA1, AA1,
ZA2, AA2,
ZA3, AA3,
ZA4, AA4,
ZA5, AA5,
ZA6, AA6,
ZA7, AA7,
ZA8, AA8,
ZA9, AA9,
ZAA, AAA,
ZAB, AAB,
ZAC, AAC,
ZAD, AAD,

AB0, AB1, AB2, AB3, AB4, AB5, AB6, AB7,

XC0, ZC0, AC0,
XC1, ZC1,
XC2, ZC2,
XC3, ZC3,
XC4, ZC4,
XC5, ZC5,
XC6, ZC6,
XC7, ZC7,
XC8, ZC8,
XC9, ZC9,
XCA, ZCA,
XCB, ZCB,


***************

Bonknuts

  • Hero Member
  • *****
  • Posts: 3292
Re: Xanadu II Translation Development Blog
« Reply #131 on: December 01, 2015, 02:43:15 PM »
elmer, you definitely impress me.. lol

You should put up a public blog site of these two projects, so people outside this forum can see your progress.

elmer

  • Hero Member
  • *****
  • Posts: 2153
Re: Xanadu II Translation Development Blog
« Reply #132 on: December 03, 2015, 09:15:32 AM »
You should put up a public blog site of these two projects, so people outside this forum can see your progress.

Nah, I can't be bothered. IMHO, the folks here are the ones most interested in the PC Engine (except, maybe, for that one French forum that I keep on forgetting about).

I'm just making sure that the people here can see that the translation keeps on slowly moving forwards, and that it's not just going to die like it did the last time, and perhaps, to also see just how much work is involved.

Oh ... it's also a chance to be a noisy Old Fart and pontificate about "the good old days" ... hahaha!  :wink:

**********

Since the subject of sound drivers is being talked-about again, I thought that I'd take a quick look at the Xanadu games.

It looks like they're both using Falcom's own proprietary sound driver, and not the one that's built into the System Card.

I'm going to hazard a guess and say that that is probably because it was easier for them to recode their existing PC-88/PC-98 driver to work on the PCE, than get their current music guy(s) to switch to a completely different way of working.

Of course, if it turns out that they were using MML on the PC-98 ... then I'd be totally wrong!   8-[

**********

And since I've also not mentioned it, yet, it's interesting to see that while Xanadu 1 used the System Card's CD routines to load a complete level of data at one time; when it came to Xanadu 2, and the need to quickly switch out large chunks of data as you move around the level, they appear to have abandoned the System Card's routines and are using their own custom code for all the CD loading.  :-k

ccovell

  • Hero Member
  • *****
  • Posts: 2245
Re: Xanadu II Translation Development Blog
« Reply #133 on: December 03, 2015, 09:57:49 AM »
I gotta say that Xanadu I has absolutely fantastic, awesome music.... but the instruments aren't that complex and they don't change all that much.  It's quite typical of a JP computer composition/translation, I think.  Jinmu Denshou (Yaksa) on the PCE has a similar thing going on.  The focus is on polyphony with a lot of sawtooth sounds.

Compare that to Ankoku Densetsu or Dungeon Explorer, where in a lot of situations simple square waves are used, but 2 channels are paired, playing almost the same note.  The phasing between the square waves does all of the heavy lifting to make a powerful sound.

Bonknuts

  • Hero Member
  • *****
  • Posts: 3292
Re: Xanadu II Translation Development Blog
« Reply #134 on: December 03, 2015, 12:18:02 PM »
So Xanadu 2 is another later gen PCE game to use custom cd read routines. The system card one isn't that great for running multiple threads of code. I've seen games do it, but it's hack-y. Besides, it's slow too. Only 90k per second transfer bandwidth. The newer routines are at least 122k/sec.

 If you get a chance, see if you can rip the custom cd read part. I have one from Seiya Monogatari that I'd like to compare.