CC65 and the PCE

[elmer]

Does anyone know of any kind of organized attempt to get PCE support finalized for the CC65?

The toolchain already supports the HuC6280 ... so from what I can see, it just needs to have the memory descriptor and startup code done, and then any custom library routines written.

Is this an everyone-fend-for-themselves situation?

[Arkhan]

yep.

Tom (bonknuts) might have done that awhile ago or tried to.  I can't remember.

I think there's not much of a push for it because C is too slow to get anything done really, so you end up in ASM faster than you think.

[elmer]

It will certainly be a lot easier for me, personally, to just go my own way and strip the C compiler and libraries out of the CC65 source, and be left with the decent quality macro assembler and linker that I want.

In fact ... it only took about 10 minutes ... all done!

The point of going the CC65 route for assembly-langauge programmers like me, is that the CA65 assembler and LD65 linker are much more capable than PCEAS.

[EDIT]
The actual CC65 compiler looks like it's much nicer than HuC ... but I still don't personally think that C really has any place on the PCE.

I thought that some other people here might have wanted a better C compiler, though.

[Arkhan]

More capable how?

[elmer]

More capable how?

If you are asking about CC65 vs HuC, then I'll just point you to Bonknuts's blog ...
https://pcedev.wordpress.com/2014/02/22/small-c-on-the-pc-engine/

Krikzz has also said that he's using it now.

If you are asking about CA65 vs PCEAS, then I'll say that first of all, I'm more comfortable with a traditional linker-based workflow.

The entire game development industry moved to that workflow instead of the early monolithic all-in-one development systems, and the surrounding tool environment supports it better. The last all-in-one system that I used was in the late 1980's.

From my POV, I can set up an assembler/linker environment for the PCE that flows just like a C-based environment for the PC-FX (or for the PlayStation4 for that matter). That's especially useful when doing cross-platform development.

When the PCEAS documentation says that it is "perhaps" unique due to its concept of "Data Grouping" ... it isn't.

It was common in the better romable environments for the C64/Atari, and then the professional dev environments like SNASM supported it on the ST/Amiga/Genesis/SNES/PlayStation/Saturn/etc.

The bank/origin abstractions in CA65 are much more like SNASM's abstractions and so much more familiar to me than PCEAS ... and, unless I'm missing something about PCEAS, more powerful.

[EDIT]
The lack of any sensible debugging information output from HuC/PCEAS has left the entire PCE homebrew toolset in a state that would have been seen as sub-par for professionals in 1987.

Even the hacker/homebrew scene in the 1990s had better tools ...
http://otakunozoku.com/rednex-gameboy-development-system/

Carsten Sørensen & Justin Lloyd's free RGBDS, together with the nocash's No$GMB emulator, was used by a lot of professional developers to avoid the horrible official Nintendo devkit.

I believe that Mednafen's Ryphecha knows nocash, and I'm still horrified that Mednafen doesn't offer source-level debugging.

At least part of the reason could be that tools such as HuC/PCEAS just don't output the necessary information.

[dshadoff]

The lack of any sensible debugging information output from HuC/PCEAS has left the entire PCE homebrew toolset in a state that would have been seen as sub-par for professionals in 1987.

Even the hacker/homebrew scene in the 1990s had better tools ...
http://otakunozoku.com/rednex-gameboy-development-system/

Carsten Sørensen & Justin Lloyd's free RGBDS, together with the nocash's No$GMB emulator, was used by a lot of professional developers to avoid the horrible official Nintendo devkit.

I believe that Mednafen's Ryphecha knows nocash, and I'm still horrified that Mednafen doesn't offer source-level debugging.

At least part of the reason could be that tools such as HuC/PCEAS just don't output the necessary information.

Feel free to create any toolchain you see useful.  We're all very interested to see what can be done !

HuC/PCEAS were put together as a hobby project (and initially I didn't even agree that HuC was appropriate... but I still contributed because I thought it was an interesting thing to study and learn from).

Feel free to take whatever fragments you like from the assembler libraries for your own project - I wrote a large part of the HuC libraries with the original thought that HuC would be too slow, but that there also wasn't enough documentation about the system: it was my thought that people would prototype in HuC, then hand-optimize appropriate sections of the code in assembler.

Without "in-context" examples of how to get the hardware to do what you want, there would be no development and no interest whatsoever, so the libraries were written.

As for the tools themselves, David Michel (who originally grabbed both the assembler and C compiler from open-source projects) wasn't aware at the time of any others to start from, so he grabbed what was available.  Everything else at the time (mid-90's for ASM; late '90's for HuC) seemed to be 16-bit and 32-bit dependent, including (in most cases) assuming that 'byte' arithmetic could never be more efficient than (at least) 16-bit 'int', so there was a lot of excess casting and 16-bit math inefficiency going on.  Maybe other tools were available and usable, but either he didn't know about them, or it seemed like a *LOT* more work at the time.

Whatever the case, anything you can do to help improve the condition of dev tools would be very much appreciated.

[Bonknuts]

Yeah, the full control over where subroutines are placed in banks/addresses is incredibly powerful.

A linker would be nice (I used AS for 68k stuff for Genesis dev, it was nice), though I've gotten used to not having it for PCE stuffs (relocating self-modifying code being the only thing I remember wishing I had a linker for; all the addresses/labels had to resolved to relative values using equates). IIRC, for debugger support - shouldn't the symbol file that PCEAS outputs be enough to a debugger to use?

 On a side note, I read that most C compilers usually tended to use only 1 or 2 address vectors (pointers) for compile code (two if you need a source and destination). HuC treats all pointer access as long, which is one reason the why local array access is soooo slow. I guess the vector address registers of the arcade card would make a decent speed up in this respect. This is something I wouldn't mind in the extended system card, but I didn't mention it because it deals specifically with C and not ASM. That, and it would have required additional hardware from something that's specifically a compiler level issue.

 I used snasm for a while, but it didn't allow including a binary file and then reassembling on top it, which is why I switched to another 68k assembler.
 
 Which reminds me of something. The mednafen author and I worked on creating a subtitling device for PCE CD game. It was basically a 68k+bitmap device that overlaid video on top of the PCE. The 68k had shared memory with the PCE via in the open bus range of the hardware bank. The 68k also had its own ram. The idea was to initialize and enable the 68k on boot of a CD game (your own routines), copy custom data over to the 68k memory via that port (the 68k had a boot rom). Then when you wanted subtitles, you would hook the games routine to jump to an alt place in which it would copy over the arguments from the bios function call to the 68k device, to which it would look up in a data base for a script to display as an overlay. There's a special build of mednafen for this, and the idea was to make a real composite signal overlay device (a composite pass through) that would function on a real machine (no physically opening up the machine, this was an external device). Though the MCU didn't need to be 68k based for the real device, since the script data being uploaded was the important part, we decided on the 68k to prototype how the device would work (plus that 68k would manually create the bitmap overlay in realtime).

[Bernie]

I feel so dumb after reading this...so damn dumb...

[synbiosfan]

I feel so dumb after reading this...so damn dumb...

You're not alone.

I keep reading,  hoping to one day to figure it out myself.

[elmer]

I feel so dumb after reading this...so damn dumb...

Please don't ever feel dumb or intimidated just because other people have a bunch of crazy domain-specific knowledge/experience that you don't.

I'm sure that there's a hell of a lot of stuff that you know that would have me scratching my head.

Feel free to create any toolchain you see useful.  We're all very interested to see what can be done !

I'll release everything that I do at some point. That is part of the fun in doing it.

But be forewarned ... it will be tailored to my personal way of doing things, and probably won't be to everyone's tastes.

That's another part of the fun ... that programmers can modify other people's stuff to suite their own tastes.

From what I've seen, there's very little difference between good homebrew programmers and good professional developers ... it's all about the investment in, and access to, good tools.

Until Unity came around, the average small team just didn't have access to an expensive-to-create toolset.

HuC/PCEAS were put together as a hobby project (and initially I didn't even agree that HuC was appropriate... but I still contributed because I thought it was an interesting thing to study and learn from).

People like you and David Michel and Charles MacDonald created the possibility for the rest of us to play here; you have nothing but my respect.

What you guys did to create MagicKit, HuC and PCEAS was amazing ... and doing so entirely without access to the official documentation is mind boggling.

One of the biggest things that drives toolchain improvement is competition ... because each competitor comes up with ideas to outsmart their competition ... and the cycle keeps repeating.

From what I can see, HuC and PCEAS have had no real competition, and stopped evolving in the early 2000s after they'd reached the point that they did everything that the people that created them needed them to do.

I'm just stirring things up a bit with a different set of expectations.

If I don't back up my observations with some action to improve things ... then you'll have every right to just dismiss me as a whining idiot.

Feel free to take whatever fragments you like from the assembler libraries for your own project.

Thank you. 

I'll grab what I can ... but I do have to admit to like writing as much of my own stuff as I can ... I find the learning part to be fun.

I didn't really want to have to mess around inside GCC to get the PC-FX C compiler updated ... but I have to say that it's been an interesting experience (although combined with considerable grief from the never-to-be-sufficiently-damned GNU build system).

Yeah, the full control over where subroutines are placed in banks/addresses is incredibly powerful.

Definitely agree! 

IIRC, for debugger support - shouldn't the symbol file that PCEAS outputs be enough to a debugger to use?

Yep, what's there should be enough to move things up to the next step ... symbolic debugging.

But in the last decade, nobody has actually taken that step!

I suspect that it's because, for whatever reason, most people are interested in hacking old games and not in creating new ones.

It doesn't help that while symbolic debugging is a great improvement for assembly language programmers, it's nearly useless for C programmers where the compiler writes most of the assembler code.

For C you need to have much more debugging information to get source-level debugging (with structure browsing), and HuC/PCEAS definitely doesn't output that.

To be honest, I'm not sure if CC65 does either, but as an assembly language programmer, I don't really care.

I am certainly not going to write a C source-level debugger for the PCE!

But I would like to try to get GDB hooking up to Mednafen for PC-FX debugging.

I used snasm for a while, but it didn't allow including a binary file and then reassembling on top it, which is why I switched to another 68k assembler.

Yeah, it's got the regular old INCBIN, but the capability that you're talking about ... nope.

It isn't really something that I've ever heard of being needed when developing new games, but I can certainly see that it would be extremely useful when hacking existing games.

SNASM was definitely designed for (and priced for) new game development.

Which reminds me of something. The mednafen author and I worked on creating a subtitling device for PCE CD games.

That's really cool! I've always wanted to do hardware stuff, but seem to be a bit of a physical clutz with a soldering iron. I'm a master at creating dry joints.

Your memory just sent me off on an investigation about creating a TMS99000-based SBC. It's the craziest CPU architecture that I've ever seen, and I always wanted to see what it was like to program. But there are too many other interesting things to do.

Did you get as far as a prototype, or were you still in the design phase?

[EDIT]
Totally off topic, but OMG ... both the TMS99105CPU and the V9990 (the VDP that didn't make it into the MSX Turbo R) are actually still available on eBay. That would make one heck of an interesting 1992-era homebrew SBC. It's a pity that I'm too dumb to design a board that would put them together. Especially in these days when board manufacturing has gotten so affordable. 

[Aladar]

From my little experience in developing a commercial-grade game(Bubble Bobble) I can say that PCEAS is adequate.
A nice addition would be the possibility for the code to go beyond the 8kB limit(PCEAS2 v3.22(Tomaitheous release) don't work correctly).

What I really need is a symbolic debugger...

[elmer]

From my little experience in developing a commercial-grade game(Bubble Bobble) I can say that PCEAS is adequate.

From what I seen that you've posted, you're doing a wonderful job on the game! 

Lots of early commercial games were written with far worse tools than PCEAS. I'd agree that it's perfectly adequate for developing good games.

A good programmer can work within almost any environment. Some early games were done with hand assembly and pieces of paper.

But in 2015 ... we don't need to do that anymore. You get to choose for yourself whether to keep on using PCEAS as it is, work on improving it, or move on to something else.

A nice addition would be the possibility for the code to go beyond the 8kB limit(PCEAS2 v3.22(Tomaitheous release) don't work correctly).

You can manually fix the bank crossings when they occur ... and just like bonknuts pointed out, you can do a bunch of equate hacks to fix up the relocations in self-modifying code.

With more sophisticated tools, you don't need those hacks, and your code is easier to write and clearer to read. It's not about whether you can ... it's about making it more pleasant to develop.

What I really need is a symbolic debugger...

Yep. That's my big complaint ... not my personal perception of weaknesses in PCEAS.

But if I'm going to be the one that has to try to hack Mednafen to provide that, then I think that it's going to be based upon whatever CA65 outputs, or that I can get it to output with some modifications.

[elmer]

Feel free to take whatever fragments you like from the assembler libraries for your own project - I wrote a large part of the HuC libraries with the original thought that HuC would be too slow, but that there also wasn't enough documentation about the system: it was my thought that people would prototype in HuC, then hand-optimize appropriate sections of the code in assembler.

...

As for the tools themselves, David Michel (who originally grabbed both the assembler and C compiler from open-source projects) wasn't aware at the time of any others to start from, so he grabbed what was available.

Aaaarrgggghh!!! 

OK, I think that I remember seeing this before, but since I'm taking a break from the PC-FX and looking at the PCE it has hit me over the head.

There is absolutely no copyright, licensing or attribution information that I can see in the HuC or MagicKit files. The closest thing is David's comments above.

Now I understand that back in the late 1990's and early 2000's people didn't care as much about that, and that nobody even cares about it today if it's just something that you hack around with yourself ... but ...

In order to get a cleanly licensed toolchain that I can put up on github ... I can't take any of the old HuC or PCEAS library code or includes, and so will end up with something that's almost certainly totally-incompatible with any current source code.

That sucks, and will only serve to make it harder for anyone but me to benefit from what I'm doing. 

David ... is there some way that you can sort out who wrote what and put some copyright and licensing information into the files?

The includes and library source are all the files that could be moved over to CA65, so just those files would need to be looked at.

If not, then it's back to typing in the includes from the Develo docs.

[dshadoff]

Aaaarrgggghh!!! 

OK, I think that I remember seeing this before, but since I'm taking a break from the PC-FX and looking at the PCE it has hit me over the head.

There is absolutely no copyright, licensing or attribution information that I can see in the HuC or MagicKit files. The closest thing is David's comments above.

Now I understand that back in the late 1990's and early 2000's people didn't care as much about that, and that nobody even cares about it today if it's just something that you hack around with yourself ... but ...

In order to get a cleanly licensed toolchain that I can put up on github ... I can't take any of the old HuC or PCEAS library code or includes, and so will end up with something that's almost certainly totally-incompatible with any current source code.

Well, I can break it down a bit for you:

Hints usually exist in the main.c files...

Assembler:
In src/mkit/as/main.c, there is an attribution for the Assembler:
/*
 *  MagicKit assembler
 *  ----
 *  This program was originaly a 6502 assembler written by J. H. Van Ornum,
 *  it has been modified and enhanced to support the PC Engine and NES consoles.
 *
 *  This program is freeware. You are free to distribute, use and modifiy it
 *  as you wish.
 *
 *  Enjoy!
 *  ----
 *  Original 6502 version by:
 *    J. H. Van Ornum
 *
 *  PC-Engine version by:
 *    David Michel
 *    Dave Shadoff
 *
 *  NES version by:
 *    Charles Doty
 */


HuC
David did mention once which C compiler he took over - it was the only one he could find that both supported 6502 and allowed for other people to take and modify it.  David was very specific on that point.  I did see the author's name once (And I would probably recognize it), but it escapes me at the moment, and I can't seem to find it easily at the moment.

It was, of course, heavily modified to support things like banks, an optimizer, inline code, an output memory space larger than 64KB, and so on.

Hint (from src/huc/main.c):
/*   File main.c: 2.7 (84/11/28,10:14:56)
 *
 * PC Engine C Compiler
 * Made by <unknown guy>, hacked to work on Pc Engine by David Michel
 * resumed work by Zeograd
 *
 * 00/02/22 : added oldargv variable to show real exe name in usage function
 */

While this doesn't name the author, it specifies 1984/11/28 as a key release date, so that should help.  It also explains why K&R C is supported, but ANSI C is not.   

Update:
Doing a Google search for "P. L. Woods" who made numerous updates in functions such as getarg() (in src/huc/function.c), I found this:
https://github.com/begoon/smallc-85/blob/master/function.c
Pretty much verbatim for an earlier version of the code.

Looking further, I see that certain functions have the same original code as this C Compiler, published in Dr. Dobbs Journal:
http://gaby.de/ftp/pub/cpm/znode51/pcwworld/l101/user_0/zsc_1.c

Attribution:

Code: [Select]

SMALL C COMPILER  -  (Part 1)

Original version published in Dr Dobbs Journal of Computing Calisthenics and
Orthodontia, May 1980 , No. 45. by Ron Cain.

...Looks like this came from a source which was already multiply-duplicated, 30 years ago - but is closest to "smallc-85" already on github.



Libraries
Written completely from scratch.  I can tell you specifically which ones I wrote, but probably 70% of the library code was mine, especially the dual-boot targetting, overlays, ISOLINK, and mouse support.  David did the original startup (machine initialization), sprites, and scrolling.  Zeograd did the macros and some other pieces (hard to remember).

We got a lot of information from Develo books (before any of us started studying Japanese, this was hard to decipher), and pretty much everything else from reverse-engineering.

David ... is there some way that you can sort out who wrote what and put some copyright and licensing information into the files?

If you're looking for me to annotate the source code of PCEAS and HUC, that's a huge amount of work, which I can't undertake.  At best, I can try to find snapshots of earlier versions of these so you can see by diff what was added.  As I mentioned earlier, the PCE assembler libraries and macros are all from scratch, written by myself, Zeograd, and David Michel.

The includes and library source are all the files that could be moved over to CA65, so just those files would need to be looked at.

Oh, if I understand correctly, then we're fine as it is.

If not, then it's back to typing in the includes from the Develo docs.

Don't do that... it hurts.

[elmer]

Attribution:

Code: [Select]

SMALL C COMPILER  -  (Part 1)

Original version published in Dr Dobbs Journal of Computing Calisthenics and
Orthodontia, May 1980 , No. 45. by Ron Cain.


Yep, Ron Cain's Small C has gotten everywhere in the last 35 years!

I can tell you specifically which ones I wrote, but probably 70% of the library code was mine, especially the dual-boot targetting, overlays, ISOLINK, and mouse support.  David did the original startup (machine initialization), sprites, and scrolling.  Zeograd did the macros and some other pieces (hard to remember).

That's my problem ... you've stated that it's the work of 3 different authors over some unknown time period with unknown licensing restrictions.

You 3 guys need to agree on what license you want to release the libraries and include files under.

I can't, in good conscience, just make something up and put it on the files.

By modern practices, EVERY file needs a copyright author, copyright date and license information.

Some countries apparently don't have the concept of "public domain", so you can't even just wash your hands and be done with it.

The least restrictive license that I know of is the Boost license, which basically comes out as ...
[uldecimal][li]We wrote this.[/li][li]Don't pretend that you did the bits that we wrote.[/li][li]Do whatever you like with it.[/li][/ul]... but put in a nice legal way.

So, if all three of you guys agree to use that one, then it might go something like ...

Code: [Select]

Copyright David Shadoff, David Michel and <whatever-zeograd's-real-name-is> 1995-2005.

Distributed under the Boost Software License, Version 1.0.
 (See accompanying file LICENSE_1_0.txt or copy at
  http://www.boost.org/LICENSE_1_0.txt)
Now, I'm happy to do the actual editing to add something like that to the library and include files ... but you've got to publicly state what the names (real ones), dates (years will do) and the license are ... and that you've got the other guy's agreement on this. Or they can post here to confirm it.
Or there can be a new release somewhere the appropriate information added.

It might seem silly for stuff that you guys just did as a hobby, but this is important legal stuff here.

It's the only way that I can think of to sanitize those files for distribution.

Sorry, I know that it's a pain, and I sympathize ... I've had to do it with both recent code that I've written and released, and also with old code that I released recently.

FYI, the last release of HuC on zeograd's site was 2005 ... so that would be the 2nd date.

If not, then it's back to typing in the includes from the Develo docs.

Don't do that... it hurts.

Well ... let's just say that I might have a better option than that.