Everdrive SDK / SD card IO

[megatron-uk]

MooZ is working on the low level write functions I believe. If he can do something like his read implementation then hooking it in to a fwrite() call should be fairly straightforward as the hardware reads and writes single sectors at a time.

I need to do some speed tests shortly - I'm at a stage where I can do some filesystem-level read tests now.

[megatron-uk]

Committed the latest code to github. Has support for fopen() and fclose() - errors are stored in 'everdrive_errors' global var:

Code: [Select]

char filename[256];
strncpy(filename, "/path/to/a/file.txt", 19);
f = fopen(filename);
if (f != 0){
    put_string("opened file ptr", 0, 1);
    put_number(f, 2, 16, 1);
    fclose(f);
} else {
    put_string("error opening file", 0, 1);
    put_number(everdrive_error, 3, 19, 1);
}
Next to do is fread() and fgetc() and properly test fopen() with directories that span multiple clusters (my test setup has a rather small directory that fits inside a single cluster - one sub folder and two files).

[megatron-uk]

Anyone got any tips for timing function calls? I'm doing some basic rough and ready approximations of the transfer speed I'm getting with the Turbo Everdrive (64-100kb/sec by my estimations), but I'm not sure my use of clock_tt() is particulary accurate (or indeed useful with longer running tests, ie >512kb).

[megatron-uk]

It is indeed very interesting - as BonkNuts mentioned in another thread, the CD ram (64k) and SuperCD ram (192k) are also still available while the Everdrive is inserted (albeit under certain conditions) so we seem to have quite a bit of flexibility.

The ultimate goal for myself is indeed to have a utility that can read/write BRAM to/from the SD card - write support may be a little longer - but I really don't think I'm too far away from being able to read save files from the SD card (created in an emulator perhaps) and loading it into BRAM on the system. Saving records back out will be the icing on the cake!

[megatron-uk]

Ok, I played around with the clock_xx() functions a bit more and have got some results that are repeatable, so I'm a bit happier now. Bear in mind that these figures are only accurate to 1/6 of a second.

Test: 512byte sector from SD card
Time: 0 s, 0 ticks -  less than one vsync interval or clock 'tick'.

Test: 64kbyte read from SD card
Time: 0 sec 37 ticks

Test: 128kbyte read from SD card
Time: 1 sec, 15 ticks

Test: 256kbyte read from SD card
Time: 2 sec, 30 ticks

Test: 512kbyte read from SD card
Time: 4s, 59 ticks (lets say 5 sec)

I work that out to be a transfer rate of approximately 115kbytes/second. That's really not bad at all. I would think that with some optimisation we could get pretty damn close to the speed of the CDROM drive, with, of course, a hell of a lot lower latency in access time.

Notes:
[ul][li]All tests where done after a call to fopen() - so the time to seek to a file and find it's directory entry were not considered (will be dependent on directory hierarchy).[/li][li]The test was fairly dumb - just sequential sector reads, I'm not even paying any notice of where the file ends yet.[/li][li]I'm recalculating LBA address from the file cluster number on each successive 512byte read, my 32bit math multiplication function, well, isn't multiplication at all, it's a massive huge hack in the form of an add, so there are definite savings to be made there.[/li][li]A cluster <-> LBA sector lookup for the most recently accessed file entries would be worthwhile looking at too - it depends on how often you reopen the same file or indeed how slow that is (next test!).[/li][li]There will be some overhead in retrieving clock_tt(), clock_ss() etc, at the end of each test, but the speed seems to be consistent between the different transfer tests so I would imagine it's a very small, consistent error number.[/li][/ul]

[Lochlan]

Have you considered using GNU Make for your project's build system?  Rather than having a bunch of shell scripts you could wrangle everything into one Makefile with multiple targets.  It would also be nice to have access to make clean!  I don't know if you like GNU Make, but I use it to build all my programming projects, commercial and personal.

[megatron-uk]

Yeah, I probably should do. Once I get the fix to the 32bit math functions in place and fread() working correctly again I'll take a look at it.

[Punch]

I work that out to be a transfer rate of approximately 115kbytes/second. That's really not bad at all. I would think that with some optimisation we could get pretty damn close to the speed of the CDROM drive, with, of course, a hell of a lot lower latency in access time.

Now I only wonder if a CDROM emulator via HuCard door is possible...

[megatron-uk]

Latest commit to github has fixed the math32 mul_int32_int8 function, removed some superfluous debug code in fat-files and generally cleaned stuff up quite a bit.

The fread() function now works correctly (correct cluster and sector address is used!), but is not yet tied into any logic for reading multiple sectors, incrementing file position pointers after a read etc. i.e. it will read the first sector of a file each time you call it.

All those provisos being true, this is still a major milestone in the project.

We can read files from the SD card from our running PC-Engine games and utilities!

I attache the output from a simple test routine (examples/test-files.c in github) that scans the FAT for a file called "/text/dracula.txt" using fopen(), displays directory entry info for the file and then makes a call to fread() to read the first 512 bytes and display them on the screen (I'm not formatting anything here, just demonstrating the ability to read and display data from a file on the filesystem).

One more thing - I would really appreciate feedback from anyone else who tries the code out; specifically the examples/test-speed.c programme... I'm getting some awfully quick results (at least twice as fast as I reported previously) and I don't know if I should trust them yet.

[megatron-uk]

First several hundred bytes of the eBook 'Dracula' from Project Gutenburg:

[Bonknuts]

Awesome work! I'll do some tests later this week, if you need.

[megatron-uk]

That would be really helpful - just as long as I don't find that this code doesn't work on anything but the two micro-sd cards I've been testing it with!

[megatron-uk]

Well my application and interview for a new job went well (got an offer and accepted it) so I'm going to try to find some time this weekend (after taking the wife out for dinner for valentines, of course!) to get the fread() implementation finished (so that sequential sectors of a file can be read) and fseek() started (so that position within a file can be changed).

[pceslayer]

any progress?

[Lochlan]

any progress?

If you looked in the Cyber Knight thread (next to this one), where megatron is also active, you'll note that he is busy with a move.

And IDK if he feels this way, but IMO it's pretty rude to ask for status updates like this unless it's been like several months.  You'll get your free software when you get it.  I definitely understand the anticipation, though, I'm looking forward to this release as well!