Elmer is off for a few days. After the work he's put in, he certainly deserves a break. Though he'll probably want to weigh in later with his programmer's perspective, you might like to know what's going on now.
The in-game text, the in-game icons, the menus, the title screen, the continue screen, the post-death/stage-clear screen...all of that is basically in its final form, fully translated. We spent a while settling on a color for the in-game text, and I played through the game again and again to polish the display timings. It's all looking very slick.
Now, we are focusing on optimising the FMVs for quality.
-----
The process of subtitling the FMVs has always looked something like this:
1. Convert FMV files ripped directly from the game to lossless AVI. Audio is lost at this stage for whatever reason. We used tools made by some Japanese guy to do this.
2. Cut off the 16 pixel tall black-ish borders at top and bottom, apply a 32 pixel pure black strip to bottom, apply subtitles over that, and output to AVI again.
3. Take subtitled AVI and convert it with MPCONV2. This is a program included with the PC-FXGA GMAKER development kit that encodes AVIs to MIX, which is a format the PC-FX's HuC6271 video playback chip can understand.
4. Merge the audio from original rips with the .mix files produced by MPCONV2, sprinkle additional formatting fairy dust, and insert into the game.
The big deal is stage 3, where we convert the subtitled AVIs to MIX. Allow me to talk about MPCONV2 for a moment. This tool is very limited...almost as if it wasn't made with real developers in mind...and when it comes to quality, it only allows you to specify a ceiling for the data rate of the FMVs. For example, you could set it at 200KB/s, and the data rate would never exceed that as it plays the entire video.
Alternatively, you can tell it to encode at certain "Quality" settings which don't have ceilings. Quality=0 effectively tells it to use as much data as it could want, and Quality=1, 2, 3 etc. step that amount down by about 10% each successively.
-----
The PC-FX has a 300KB/s rated drive, and so until recently I had been encoding the videos with a 295KB/s ceiling (for peace of mind). Basically, this output looks very good. However, there are three problems:
1. There is a minor drop in overall quality due to lossy-encoding. Of course, the source is also lossy, but we inevitably loose just a little bit more quality on top of that. This could be alleviated somewhat if we had a higher data rate ceiling.
2. The subtitles, which are yellow, occasionally have some blue dots appear around them. They're still easy to read, but the effect is distracting. This is merely another case of compression artifacts, and its caused by moments when the video image needs an unusually large share of the bandwidth.
3. In areas with lots of black, compression artifacts can be particularly bad, and occasionally outright hideous.
About #3 - MPCONV2 gets a little overzealous with black, and occasionally it assigns black areas a special kind of 16x16 pixel pure-black block that is super-efficient in terms of data compression but god awful in terms of visual appeal. In these areas, I have been creative with video editing, introducing white fades or otherwise slightly brightening all the frames between visual cuts. Most of these additions you won't even notice, and it solves the problem satisfactorily. However, there is a slight chance that MPCONV2 could be hacked so as to never use these blocks, which would be sweet.
Please don't panic when you see that video, though. There are very few instances of this problem in the game. That one is by far the worst, and I've covered everything up pretty nicely.
-----
But now, the Big News. About a week ago, elmer made a fantastic tool that lets us look at data rates on a per-frame and per-second basis within each .mix file. This has led to many fascinating discoveries.
1. Encoded at 300KB/s, anywhere from 40-80% of the video content is actually sub-250KB/s depending on the specific FMV. It will only rise to the ceiling at certain peaks.
2. Even when encoded at the ceiling-less Quality=0 setting, the majority of the video content is still sub-300KB/s, and only a couple of seconds in all of the game's FMVs combined exceed 350KB/s.
3. Additionally, we also discovered in short order that the PC-FX is using a buffer when it streams the videos, and it can actually handle brief spikes over 300KB/s. I have tested this on real hardware. The size of the spikes it can handle would surprise you.
The result of all this is that we have a couple of interesting potential methods of improving the quality of our subtitled FMV.
One thing that I tried was to encode a video in two versions: one at 300KB/s and the other at Quality=0. Breaking down the two files into individual frames, I erased certain large frames from the Quality=0 version and replaced them with frames from the 300KB/s version. This produced something that looked very sigificantly improved, worked on real hardware, and was easy to make.
(Keep in mind that since the PC-FX's playback is JPEG based, every frame is stored completely. You couldn't do this with an MPEG format).
---------
HOWEVER...elmer has also come up with a whole other method which appears to work well.
Go alllll the way back to the beginning. You rip the FMV files from the game, you convert them to lossles AVI. Here's where things take a turn.
If you never apply the subtitles to the AVIs, and stop after having the video moved 16 pixels up and applying a 32 pixel black strip to the bottom, they compress into MIX much better. If you compare sizes of no-subs MIX output files, there is almost no difference between the ceilingless Quality=0 setting and a 300KB/s-ceiling setting (for the files I have reviewed so far, anyway).
Elmer found out something cool. I don't have a firm grasp on the technical details, but it seems you can effectively have the HuC6271 display two JPEGs at once. What we can do is to encode the video and the subtitles separately with MPCONV2, initially outputting individual files for each, then merging them into one so that they are both displayed simultanously. The result on screen is identical at a glance, but this method lets us encode the subtitles in a way that they are always flawless regardless of how busy the video image is.
However, the question here is whether the subtitles and the video can balance out nicely. If we have to drop the quality of the video to make room for the subs, then that might not be cool. If we have to lower to quality of the subs to the point that artifacts start appearing, then it's kind of pointless.
I have a lot of work to do to test this.
-----
So, we have two methods of improving the FMVs to choose from at the moment:
1. Comb high-quality, >300KB/s versions of MPCONV2 output for better frames that will still fit within buffer tolerances.
In short, the advantage of this method is that there's a natural balance between the video and the subtitles. Artifacts around the subtitles will only appear at peak moments when the image itself demands more of the data. The disadvantage, of course, is that any artifacts around the subtitles look bad.
2. Encode the subtitles separately from the video, then combine them.
The advantage of this method is that we can force the subtitles to always look clean. The disadvantage of it is that the clean subs might come at the cost of the quality of the video, though again, this would be only for brief moments, and would probably be very minor.
I am actually inclined to go with option #2, as long as the subs can be totally clean and the video not compromised beyond a negligable degree.
----
And last but not least, there are two completely separate additional solutions for making the FMVs look good.
1. Perform some programming voodoo to move the original video as-is up 16 pixels, and then add the subtitles separately like mentioned above. This is ideal in terms of quality (no additional lossiness to the image whatsoever), but possibly very difficult in terms of programing.
2. Write an entirely new compressor that encodes video in multiple passes and generally does things smarter and better than MPCONV2.
These things are complicated and challenging enough that we might wait and put out a "remastered" version sometime later.
----
Anyway, this is what I have been doing with myself for the past week or two. If you're wondering why Zeroigar is taking so long even though we've had the basic stuff done for so long, it's because getting it to look as good as it possibly can takes a while. I'm just grateful to elmer for going the extra mile with me.
