Author Topic: Anearth Fantasy Stories - Translation Development Blog  (Read 4889 times)

elmer

  • Hero Member
  • *****
  • Posts: 2153
Anearth Fantasy Stories - Translation Development Blog
« on: July 05, 2017, 05:20:50 AM »



A long, long time ago, in a galaxy really quite close ...

I'm going to go ahead and give Anearth a green light for translation, as long as Esperknight is in (and I'm 99% sure he is). I can start as soon as the script gets a proper dump.

I am undeniably having fun playing this, but the clincher is that I don't think translation projects on this system will ever come any easier. The script is stored in a simple way, the dialogue is not particularly dense or complex, the game is short, and maybe best of all, it won't need a dub or custom subtitling.
I just reached the end of my first playthrough of AFS.
...
Basically it was good, and if the script gets extracted easily, I'm all for doing it. It gives me something to do while Esperknight gets the Xanadu II translation into the playable-beta stage.

Well, moving on a few years, and with a change of programmer, it looks like this is on-the-move again!  8)
« Last Edit: July 19, 2017, 07:51:06 AM by elmer »

esteban

  • Hero Member
  • *****
  • Posts: 24063
Anearth Fantasy Stories - Translation Development Blog
« Reply #1 on: July 05, 2017, 05:28:50 AM »
Well, damn. That's an unexpected, and awesome, update.

:)

FOR FOLKS WHO ARE CURIOUS:
http://tg-16.com/anearth/intro.html
« Last Edit: July 05, 2017, 07:42:40 AM by esteban »
  |    | 

elmer

  • Hero Member
  • *****
  • Posts: 2153
Re: Anearth Fantasy Stories - Translation Development Blog
« Reply #2 on: July 05, 2017, 05:30:54 AM »
Imagine if Elmer and Sam secretly work on Anearth Fantasy Stories and release all 3 games simultaneously?

It's a nice idea, and you're partially right!  :wink:

I've done some work on this while dealing with the administrative side of the LoX auditions, and that work will continue as SamIAm deals with the LoX recordings.

The idea is to make sure that he's got something to do and won't be bored while I'm dealing with fixing the LoX lip-sync in the cutscenes.

But there is absolutely no way that these different projects are going to be released anywhere near each other!  [-(

There's just too much work involved in one of these translations.  :-({|=


But, it is a good time to start talking about AFS, and about how it differs from the LoX games in terms of how it was put together, and how it needs to be hacked in order for a translation to work.

Again, like the LoX Blog, this thread will concentrate on the technical side of what it takes to get a translation done, and why having a programmer involved is sometimes needed.

Necromancer

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 21366
Re: Anearth Fantasy Stories - Translation Development Blog
« Reply #3 on: July 05, 2017, 05:53:05 AM »
Awww, yisss.
U.S. Collection: 97% complete    155/159 titles

elmer

  • Hero Member
  • *****
  • Posts: 2153
Re: Anearth Fantasy Stories - Translation Development Blog
« Reply #4 on: July 05, 2017, 06:04:22 AM »
So where do things stand right now ...
[ul][li]The basic program architecture and runtime memory layout is known.[/li][li]The CD layout and the location of the code/graphics/script data is known.[/li][li]The decompression code has been disassembled and anaylzed.[/li][li]The compression code has been created to produce compatible data.[/li][li]The scripting language has been disassembled and anaylzed.[/li][li]The RPG script data (including code and Japanese text) are all extracted.[/li][li]The extracted scripts can be recompiled and give identical script data.[/li][li]The font drawing system has been disassembled and anaylzed.[/li][li]The font code has been converted into source code so that it can be changed.[/li][li]The script language core has been converted into source code so that it can be changed.[/li][li]The game's original Debug Menu has been hacked and fixed so that it can be used again.[/li][/ul]
What needs to be done ...
[ul][li]Lots!  ](*,)[/li][/ul]

LentFilms

  • Guest
Re: Anearth Fantasy Stories - Translation Development Blog
« Reply #5 on: July 05, 2017, 08:05:30 AM »
This is really awesome news! Anearth Fantasy Story is at the top of my list of awesome looking RPGs on the PCE-CD that I've always wanted to play in English. Based off some of the off-handed comments you had been making on the Xanadu threads I was guessing that maybe this was going to be Sam and your's next project. I barely understand most of the technical posts in the Xanadu Development thread but I look forward to reading about your progress on Anearth!

dshadoff

  • Full Member
  • ***
  • Posts: 175
Re: Anearth Fantasy Stories - Translation Development Blog
« Reply #6 on: July 05, 2017, 11:46:05 AM »
This is fantastic news !

As we are talking about the technical side of these patches, I was wondering whether you could describe your development setup (and workflow if possible).

1) What IDE/language are you using ?  Eclipse ?  Microsoft's IDE ?  JAVA ?  C ?
2) What emulator are you using ?  Is it one where you execute the TOC, or do you use a CD emulator like Daemon Tools ?
3) For the SJIS, are you displaying in a different codepage ?  Or are you converting to Unicode and back ?

I'm pretty sure that I'm not the only one who's interested.

Dave

jtucci31

  • Hero Member
  • *****
  • Posts: 1648
Re: Anearth Fantasy Stories - Translation Development Blog
« Reply #7 on: July 05, 2017, 03:08:37 PM »
Let us plebs who just want more of these to happen know what we can do. This is awesome news! This was always one of those lofty RPGs to one day be translated, and now we are inching closer!

ParanoiaDragon

  • Hero Member
  • *****
  • Posts: 4619
Re: Anearth Fantasy Stories - Translation Development Blog
« Reply #8 on: July 05, 2017, 06:30:20 PM »
As for dubbing for this game, I assume there cinemas that use redbook audio.  If so, how feasible is it to dub, & find replacement sounds & tunes for the BG?

Michirin9801

  • Hero Member
  • *****
  • Posts: 589
Re: Anearth Fantasy Stories - Translation Development Blog
« Reply #9 on: July 05, 2017, 07:40:51 PM »
I've only heard of this game in passing, but after looking it up on YouTube I gotta say, it looks pretty sweet! I'll be looking forward to more news on this project as well ^^

I'm pretty sure that I'm not the only one who's interested.
You're definitely not!

elmer

  • Hero Member
  • *****
  • Posts: 2153
Re: Anearth Fantasy Stories - Translation Development Blog
« Reply #10 on: July 06, 2017, 05:14:10 AM »
I've only heard of this game in passing, but after looking it up on YouTube I gotta say, it looks pretty sweet! I'll be looking forward to more news on this project as well ^^

There's a great review on Brothers Duomazov ...
http://www.thebrothersduomazov.com/2013/01/seiya-monogatari-anearth-fantasy-stories.html

Black Tiger posted a bunch of screenshots in the old translation thread ...
https://www.pcenginefx.com/forums/index.php?topic=14331.msg341951#msg341951


I barely understand most of the technical posts in the Xanadu Development thread but I look forward to reading about your progress on Anearth!

I'll try to be a little less boring this time around!  :wink:

This time, there definitely shouldn't be any need for posts full of long lists of dull numbers, written while I try to improve my compression code!  :pray:


As for dubbing for this game, I assume there cinemas that use redbook audio.  If so, how feasible is it to dub, & find replacement sounds & tunes for the BG?

Every scene that is voiced also has a message box with the text of the speech inside it.

We're not planning on doing a dub, it just doesn't seem necessary ... we can change the text instead.


Now, if someone really wanted to do a dub, it *might* be possible.  :-k

25 out of the 29 CD-Audio tracks are pure music ... no voice or sound effects.

Only 4 of the tracks contain voice, and (I believe) those are for the 4 different character-class choices right at the start of the game.

It sounds like the BGM for those particular tracks is already available in one of the other CD tracks.

I have no idea about the rest of the game's VO tracks, but they're obviously in ADPCM somewhere in the data portion. I can see where VO is triggered in the script data, so I can probably find all of the ADPCM ... it's just not been necessary.

As far as mixing new ADPCM tracks for a dub, I suspect, but don't know for sure, that the BGM for the existing ADPCM VO might actually be in the list of CD Audio tracks.

I think that there's a decent chance that Hudson/MediaWorks recycled those, unlike Falcom and the LoX games, where you can only find the pure CD Audio tracks in the separate LoX CD Audio releases.
« Last Edit: July 06, 2017, 07:01:07 AM by elmer »

elmer

  • Hero Member
  • *****
  • Posts: 2153
Re: Anearth Fantasy Stories - Translation Development Blog
« Reply #11 on: July 06, 2017, 07:40:59 AM »
As we are talking about the technical side of these patches, I was wondering whether you could describe your development setup (and workflow if possible).

Sure!

I'm a bit old-fashioned, so development is being done on a Windows 7 x64 PC.


Quote
1) What IDE/language are you using ?  Eclipse ?  Microsoft's IDE ?  JAVA ?  C ?

The custom toolset is comprised of a PCE-VDC-data-to-PNG converter, and PNG-to-PCE-VDC-data converter, and the script-extractor-compiler-inserter. These are all command-line programs so that they can easily be used in batch files.

They're written in C/C++, because I'm so familiar with it, and it's easy to knock up quick programs.

They could be written in C#, Python, or anything else that can memory-map and modify existing binary files.

I'm using Visual Studio 2010 Professional, because the debugger is excellent, and it's the last version before Microsoft screwed-up the IDE.

Although I actually prefer to edit the source code in a separate and more-powerful text editor. That's just personal preference.

But most of my time is either spent at a command-line in TCC/LE, or MSYS2 if I need some Unix tool like hexdump/grep/diff.

I tend to prefer using batch files for building and testing things (unless I'm debugging the extraction/insertion tools).

I'm using grafx2 to make all of the graphics changes, because it's quick and simple, and I don't need all of the extra features that something like ProMotion brings to the table.


Quote
2) What emulator are you using ?  Is it one where you execute the TOC, or do you use a CD emulator like Daemon Tools ?

I use my customized version of Mednafen that I posted here ages ago.

Since I posted that version, Mednafen has finally added some better font choices, but the overall debugger layout still hasn't incorporated the UI layout changes that I made in order to make it prettier to look at and usable.

Because I spend so many hours looking at things in Mednafen's debugger, those changes that I made are really important to my overall comfort in using it.

And without Mednafen's excellent debugger, I wouldn't have done any of this translation work, it would just have been too painful.

When you're at the command line all of the time anyway, it's trivial to launch Mednafen from a batch file or build script, with no need to mess around with a slow GUI.


Quote
3) For the SJIS, are you displaying in a different codepage ?  Or are you converting to Unicode and back ?

Well, Windows is just in the standard US-English mode.

The scripts are extracted as SJIS, and not Unicode, because it's easier to compile them in SJIS format in order to verify that they produce identical data to the original script chunks.

That's the only way that I can be sure that I understand the complete structure of the script language, and will be able to change the scripts without breaking the game.

It's easy to edit the script files in any text editor that supports SJIS ... like Notepad++

Here's an example of one of the AFS script files ...

.script003:
  _call( .chunk01, .script005 )
  _call( .chunk00, .script111 )
  {買い物なら}*
  {カウンターごしに}*
  {話しかけてね}
  _end()

.script004:
  _call( .chunk01, .script005 )
  _call( .chunk00, .script111 )
  {お買いものですか}*
  {何になさいます?}
  _wait_for_keypress_then_clr()
  _code253A( $00, $00 )

.script122:
  _call( .chunk00, .script016 )
  _choose( .chunk00, .script006, $00, $10,  0,  2,
           22, 10,   6,  2,      22, 12,   6,  2,      22, 14,   6,  2 )
  _chosen( .chunk00, .script121, .chunk00, .script124, .chunk00, .script050, $FF )

.script059:
.script124:
  _code250E()
  _call( .chunk01, .script005 )
  _call( .chunk00, .script111 )
  {持ち物を買い取れって?これだから田舎者は}*
  {困るのよね}
  _wait_for_keypress_then_clr()
  _jump( .chunk00, .script127 )



You can notice a couple of things from that example ...
[ul][li]For a complex script language, it's important to write custom extraction/compilation tools, or else you won't be able to follow the flow of the langauge.[/li][li]You don't need to understand exactly what all of the script commands do in order to work with the scripts.[/li][li]This particular language is a bit of a mess, with scripts calling scripts in different script chunks all over the place.[/li][/ul]
It's the last aspect that probably killed EsperKnight's ability to make any real headway with the script extraction, because until you understand that, you're just looking at fragments of Japanese text that make little sense.

Pretty much the same with the LoX games.

Unless you know that you're looking at a full embedded programming-language inside the game, you really can't understand what's going on, and you'll never be able to successfully modify things (i.e. translate the text) without breaking everything.

Not every game is this complicated, but embedded interpreted application-specific scripting languages were really popular in some genres as an alternative to simple text control-codes within strings.

But ... I'd be surprised if you'd encounter them much on the PCE outside of RPG games, so I hope that people reading this won't be put off trying to do a translation of any of the many other games on our favorite machine!  :wink:

dshadoff

  • Full Member
  • ***
  • Posts: 175
Re: Anearth Fantasy Stories - Translation Development Blog
« Reply #12 on: July 06, 2017, 02:14:05 PM »
Quote
1) What IDE/language are you using ?  Eclipse ?  Microsoft's IDE ?  JAVA ?  C ?

The custom toolset is comprised of a PCE-VDC-data-to-PNG converter, and PNG-to-PCE-VDC-data converter, and the script-extractor-compiler-inserter. These are all command-line programs so that they can easily be used in batch files.

They're written in C/C++, because I'm so familiar with it, and it's easy to knock up quick programs.

They could be written in C#, Python, or anything else that can memory-map and modify existing binary files.

I'm using Visual Studio 2010 Professional, because the debugger is excellent, and it's the last version before Microsoft screwed-up the IDE.

Although I actually prefer to edit the source code in a separate and more-powerful text editor. That's just personal preference.

But most of my time is either spent at a command-line in TCC/LE, or MSYS2 if I need some Unix tool like hexdump/grep/diff.

I tend to prefer using batch files for building and testing things (unless I'm debugging the extraction/insertion tools).

I'm using grafx2 to make all of the graphics changes, because it's quick and simple, and I don't need all of the extra features that something like ProMotion brings to the table.


This is really similar to what I use, actually - although I'm a bit more UNIXy (Eclipse IDE for C/C++, bash or ksh instead of BAT, gvim for editing, etc.).  I also write in C, because it's easy to write things quickly.

Would you be interested in sharing your PCE-VDC-data-to PNG converters ?

I wrote something similar to create ASCII art files for a Bazaru de Gozaru project - where almost all of the text is stored in graphics - but I'd like to pass along all of my work for somebody else to continue on, and nobody wants to touch ASCII art.  Sounds like your way would be easier for normal humans than my way.

(I'd be happy to convert all my extraction/insertion scripts to aid whoever wants to take it on - it would be mostly adjusting graphics...)


Quote
Quote
3) For the SJIS, are you displaying in a different codepage ?  Or are you converting to Unicode and back ?

Well, Windows is just in the standard US-English mode.

The scripts are extracted as SJIS, and not Unicode, because it's easier to compile them in SJIS format in order to verify that they produce identical data to the original script chunks.

That's the only way that I can be sure that I understand the complete structure of the script language, and will be able to change the scripts without breaking the game.

It's easy to edit the script files in any text editor that supports SJIS ... like Notepad++


Thanks for that... I've generally seen the following format, in non-compressed data:
[array of pointers to blocks]

block1:
[array of pointers to text strings]

text string1:
A bunch of text, with a few interesting control codes to pause, wait for keypress, change color, switch character set, and so on - but intermingled with the text itself...followed by a terminator, which is usually a 0x00 or sometimes 0xFF.

Occasionally, you'll find that "block1" is not an array of pointers, but rather the start of the first string, and the strings are read sequentially.

Finding a way to express the meaning of those interspersed codes - while still making it intelligible to the translator/editor, and keeping it parseable for reinsertion - was always challenging.

Quote
It's the last aspect that probably killed EsperKnight's ability to make any real headway with the script extraction, because until you understand that, you're just looking at fragments of Japanese text that make little sense.


I got the same feeling, looking at his work on Tengai Makyo - Ziria.  He was able to find text quickly, but had more difficulty trying to figure out a larger organizational structure of those blocks.

[/quote]

ParanoiaDragon

  • Hero Member
  • *****
  • Posts: 4619
Re: Anearth Fantasy Stories - Translation Development Blog
« Reply #13 on: July 06, 2017, 06:58:15 PM »
That's right, I seem to remember the mention of not doing dubbing on Anearth.  I always prefer dubbing, but of coarse, on a homebrew translation project, it's an added bonus.  It's nice that there's subtitles, that helps!

elmer

  • Hero Member
  • *****
  • Posts: 2153
Re: Anearth Fantasy Stories - Translation Development Blog
« Reply #14 on: July 07, 2017, 04:23:58 AM »
Would you be interested in sharing your PCE-VDC-data-to PNG converters ?

I can give you 32-bit Windows command-line binaries immediately, but I'm not quite ready to share the source code quite yet, if that's OK.

I want to do so ... I've just got a whole lot of cleanup to do on the source.

The pce2png utility has only been hacked-together for my own use, and it's not very friendly or robust.

I'll PM you.