<[{(Channel 5 bit sample data) x 5 bit Channel Volume} x 4 bit Channel Balance] x 4 bit Global Balance> = 18 bits of volume control!
The formula is this:
left = 0x1f - (global.left*2) + 0x1f - (chan.left*2) + 0x1f - (chan.mono)
right = 0x1f - (global.right*2) + 0x1f - (chan.right*2) + 0x1f - (chan.mono)
Left and right saturate at 0x1f (they don't clip).
From there,
left_out=log_tabe[left]*current.sample
and same for the right.
They accumulate, not multiply. It works out since the channel volume steps are logarithmic. So there's only one real volume control per each channel left and right side, so you gain nothing as far as increased depth/step output if you try to mix mono and pan volume settings. Most games use mono volume for the envelope volume and channel pan as the master volume for it. Makes it convenient even if you're not gaining additional resolution in attenuation.
What the Paula sound hardware on the Amiga does is more likely: While the Amiga has 8-bit sample hardware, it also has a 6-bit (64 level) volume control. That does not add up to 14-bits, technically as there is no 64-level DAC in the Amiga. Instead, the Paula hardware suppresses (mutes) a single (8-bit) sample every certain number of clock ticks (28800 ticks per second, going by my lousy memory). If the volume is 63, all samples pass through at the specified frequency. If the volume is lower-- 62, for example-- then one out of every 64 samples is suppressed, and so on. This has the audible effect of a gradual volume fade. HOWEVER, listen to a sample on the Amiga at a very low volume and the sample sounds scratchy due to the Paula "dropping" samples to get this low volume effect.
I'd bet the PCE does it the same way, going by the way it sounds...
I've never heard that. Are you sure? The discussions I've had with a few amiga coders was that the volume control in some Amigas didn't have a linear reduction across the full range (the accuracy of the dacs wasn't meant for that exploit) and so the 14bit audio you got with the paired channel (AM mode) was really only 10bits usable. But that it varied from Amiga version to version (within models).
If the PCE did that, I would have probably notice that. If you suppress a waveform down to just 1 sample output of 32, and played it at a low rate - it wouldn't sound scratchy. It'd sound completely wrong altogether. For what it's worth, I looked at the PCE audio on the scope before it hit the amp (at its full 3.58mhz output from the chip) and I didn't see anything like that. I was specifically looking for PWM or such on the DAC output too. That and I've done a ton of recordings to the PCE from the real units over the years (some for myself and some for tests for Ryphecha and other emulation authors). I didn't notice any anomalies in the low and very low volume range. Matter of fact, when I did the paired channel 10bit DAC output tests, I used pan volume to raise and lower the volume of the sample output. I didn't notice anything weird there either. I know at some point, the lower of the paired channel will saturate to 0 level (or leak level) - but the upper channel volume is so low by then that it's not even noticeable.
I know some of Genesis and Megadrives have anomalies in the DAC output for really low volume output. A weird stepping effect (spritesmind had it documented in a thread). But since you mentioned it, I'll keep any eye out for it when I do recording tests again from the real system.
Mute any channel using any of the controls and you can still hear the waveform playing quietly... so doesn't this interference cut out the "lower bits" of the theoretical 10-bit volume envelope?
I don't see how one is related to the other. Unless you're referring to the noise. I was referring to noise via quantization at low bit depths. The artifacts of low bit depth, if you will. I wasn't referring to the 10bit DAC paired output trick, if that's what you mean. That trick works pretty good on the real hardware, but this is completely different.
I've also noticed the audio leak too. But IIRC, it was only when chan pan was set to 00 and mono was set high. I don't think I got audio leak at with $80 and $ff settings. But even if it were so, that would just make volume setting of 0 the lowest instead of 'off'. I usually turn my channels off by stopping the waveform pointer.
(edit: had to add code tags)