Establishing a ROM Format

Jun 16, 2014 • 17:59

I’m in the middle of making a PVB emulator (hey, no one objected to putting PVB’s name on it!), and I’m making it flexible: it’ll support things like ROM mappers and use of the cartridge expansion area, things that no commercial Virtual Boy game does. In order to do it right, however, we’re going to need some more information than is available in a straight ROM dump from a commercial game.

So we’re about to write history. I’d like to invite everyone to pitch in with discussions regarding a new, unofficial ROM file format for Virtual Boy games.

As it stands, all Virtual Boy ROMs are just the data from the ROM chip, verbatim, byte-for-byte and nothing more. And that works for pretty much every Virtual Boy program ever made: all commercial games, all homebrew, all… I guess that’s about it. But let’s say we do some hardware development and implement some custom audio hardware in the cartridge, or a rendering coprocessor, or simply make support for games larger than 16MB.

Such programs wouldn’t straight-up be able to run in an emulator unless the emulator knew what capabilities the cartridge has. That’s what this thread is all about.

Proposal

Virtual Boy ROM images do contain some information beginning at 0xFFFFFDE0 (the 544th byte from the end of the file):

0x00    4 bytes    Game Title, Shift-JIS
0x04    5 bytes    (Reserved, zero)
0x09    2 bytes    Maker code, ASCII
0x0B    4 bytes    Game code, ASCII
0x0F    1 byte     Game version, 1.

This is information about the program, but not the cartridge. An additional, unofficial header should be present at the beginning of the ROM file: it can be detected because the size of the file will no longer be a power of 2 bytes.

I propose the following data and format for a new, unofficial Virtual Boy ROM header:

0x00    2 bytes    Identifier: "VB" as ASCII (0x4256 little-endian)
0x02    2 bytes    Cartridge Type
0x04    2 bytes    Header Version Major
0x06    2 bytes    Header Version Minor
0x08    4 bytes    Checksum
0x0C    4 bytes    Header Size
0x10    ...        (Header data according to size and version)

Identifier

This 2-byte magic number can be checked to determine whether the data actually represents a Virtual Boy ROM. If it’s ‘BM’ or something then you’re looking at a Windows Bitmap, dummy!

Cartridge Type

A numeric identifier indicating the physical capabilities of the cartridge. I suggest the following two values to represent the commercial games:

0x0000    Stock ROM Only
0x0001    Stock ROM+RAM

Header Version

The header’s data version. Different versions contain different data beginning at 0x10. Version numbers are notated as major.minor, and the initial version will be 1.0.

Checksum

A CRC32 of the entire ROM file, except for the bytes occupied by this header. It can be ignored in the event some modding is going on or whatever, but it should in good faith represent the actual data checksum.

Header Size

The remaining number of header bytes present beginning at 0x10. Additional data may appear after the defined data range for custom info used by level editors or other applications.

ROM Header v1.0

For ROM Headers of version 1.0, the format will be as follows:

0x10    4 bytes    ROM Size (in bytes)
0x14    4 bytes    RAM Size (in bytes)

ROM Size

The size, in bytes, of the ROM data. This field effectively expresses the remaining size of the file.

This field is a literal 32-bit value in order to support mappers. A mapper (indicated by Cartridge Type) might swap out only data in the range of 0x07100000 to 0x071FFFFF, for instance, so there’s no power-of-2 requirement here.

RAM Size

A metadata field representing the number of available bytes of cartridge RAM. For stock cartridges, this means the size of the internal RAM chip and affords proper mirroring of the cartridge RAM address space. For other cartridge types, this may indicate the number of swappable pages, etc.

Viewing 8 replies - 1 through 8 (of 8 total)

8 Replies

Jun 17, 2014 • 00:00

Tauwasser

Hi, I like your idea to allow for as-of-yet un-emulated stuff.

However, what you’re proposing is essentially butchering the ROMs respectively adding an iNES-type of header.

Representations change, knowledge changes, requirements change etc.

IMHO, one should go the way byuu went about Higan, i.e. add metadata. It takes a little preparation:
[list]
[*]have XML files of all PCBs you want to support at a reasonable level of abstraction
[*]have XML files that describe an actual game cartridge, i.e. bind “cartridge” to PCB and all its relevant data.
[/list]

What PCB descriptions might look like:


	
		VDD
		A20..A0
		#ROM_CE
		#OE
	


	
		VDD
		A20..A0
		#ROM_CE
		#OE
	
	
		battery.v
		A12..A0
		#RAM_CS
		mm1245.cs
		#OE
		#WE
	
	
		VDD
		battery.v
		150nF
	
	
	
		#RAM_CS

What a game description might look like:


	
		VUE-VMTJ-0.vb

Basically, have an abstraction in whatever detail you want of the PCB. Then have a game file that maps files, can turn options on or off.
For instance, battery might not be fitted or not. Some game might have VUE-AA3A’s resistor fitted and mm1245 not fitted, so they would write


	
		VUE-VMTJ-0_hacked.vb
		ram.bin
		
		<

As you saw earlier, instances have a fixed set of signals they provide. GEC is implicitly on all PCBs. It provides A0..A22, D0..D15, #OE etc.
MM1245 provides cs, csb, and tr.b. It wants a capacitor value in tc and it wants to know which voltages go into it.
Battery provides its voltage v.

A custom mapper could provide ROM addresses:


	
		VDD
		mapper.ra22..mapper.ra21 A20..A0
		#ROM_CE
		#OE
	
	
		 A22..A15 A7..A0
		#OE
		#RAM_CS
		#UWR

So now my ROM has two address bits more, RA22 and RA21, which are provided by mapper. Mapper itself needs some address lines to determined if it’s turned on or not etc.

Orient yourself by what’s actually physically possible and then find a level of abstraction you’re comfortable with. For instance, you could just nix the whole SRAM backup IC and just always have SRAM work. Then you couldn’t emulate the SRAM battery running dry or a brown-out of the Virtual Boy supply voltage (batteries in controller).
You would then have to determine if you’d be comfortable with not emulating this behavior or not.

But please, don’t invent some custom ROM header format that will get spread for no good reason and become out-dated once the first real-world mapper application is developed and being emulated.

cYa,

Tauwasser

Jun 17, 2014 • 00:30

Guy Perfect

Tauwasser wrote:
However, what you’re proposing is essentially butchering the ROMs respectively adding an iNES-type of header.

That’s exactly correct and exactly the intention. Doing it that way pairs the ROM with its respective metadata, and avoids the issue of attempting to keep a comprehensive database of Virtual Boy software somewhere. The meta data simply must be included with the ROM data in order to be useful.

Even what Higan does with its games database doesn’t solve the problem I see when someone distributes or receives a Virtual Boy program, but the emulator doesn’t necessarily know what kind of hardware it was intended to work with. That’s why I strongly feel that the metadata needs to be in the same file with the ROM.

I do a lot of experimenting, we’ve got a fair amount of homebrew, and someday we might even do some manufacturing. It’s simply not practical to try to keep a running bank of everything all the time.

Tauwasser wrote:
Representations change, knowledge changes, requirements change etc.

My proposal fully supports arbitrary representations and affords the additions of new features at a later date without breaking compatibility. I understand the dangers of making assumptions.

Tauwasser wrote:
But please, don’t invent some custom ROM header format that will get spread for no good reason and become out-dated once the first real-world mapper application is developed and being emulated.

That’s why I made this thread; to keep that from happening. (-:

Though again, my proposal isn’t something that can become outdated because there’s no rigid format that makes any assumptions about the software contained within the ROM file.

Jun 17, 2014 • 15:45

RunnerPack

Why not split the difference? We could have a human-readable (though not necessarily XML, ick ;-)) descriptor file which is kept alongside the ROM, rather than being attached to it. You could even take a cue from Mednafen and include a hash of the ROM in the descriptor’s filename, which would help prevent mix-ups.

Jun 19, 2014 • 23:55

Tauwasser

Guy Perfect wrote:

Tauwasser wrote:
However, what you’re proposing is essentially butchering the ROMs respectively adding an iNES-type of header.

That’s exactly correct and exactly the intention.

You are aware of the fact that everybody loathes iNES headers?

Guy Perfect wrote:
Doing it that way pairs the ROM with its respective metadata, and avoids the issue of attempting to keep a comprehensive database of Virtual Boy software somewhere. The meta data simply must be included with the ROM data in order to be useful.

Keeping a database is not what my approach is about. While feasible for VUE, most Nintendo game systems have the necessary data encoded somewhere in their header. When this is wrong, too bad => user has to decide what’s right.

If you’re keen on distributing everything on one file, then create your preferred file format(s) and zip them together with the ROM. Problem solved. No need to distribute sullied ROM images.

Guy Perfect wrote:
Even what Higan does with its games database doesn’t solve the problem I see when someone distributes or receives a Virtual Boy program, but the emulator doesn’t necessarily know what kind of hardware it was intended to work with. That’s why I strongly feel that the metadata needs to be in the same file with the ROM.

The need to know what hardware a game is supposed to run on does not implicate the need to have one file. Guess what, the VUE doesn’t know anything about the game that is inserted into it either.
Instead, the information is there implicitly by the PCB design of the game. You need to make that data explicit, i.e. by writing it down in a markup format, and distribute it.

Guy Perfect wrote:
I do a lot of experimenting, we’ve got a fair amount of homebrew, and someday we might even do some manufacturing. It’s simply not practical to try to keep a running bank of everything all the time.

Well, you’re a developer then. I see two primary use-cases here.

[list][*]casual gamer: a database of VUE game hashes will suffice, games will work
[*]developer: you know your stuff anyway and get the benefit of telling the emulator about the board you’re actually experimenting on[/list]

One case doesn’t need special knowledge and “it will just work”, the other has so much knowledge that choosing the right board file should be easy and a one-time job.

Guy Perfect wrote:
My proposal fully supports arbitrary representations and affords the additions of new features at a later date without breaking compatibility. I understand the dangers of making assumptions.

Your first assumption is that your format would work… Again, look at the iNES mess when all the mapper numbers were used up or used to mean different things at different times :-/

Basically, you should think about why you would want to do this if you end up touching the original ROM images. Because it’s 2014 and there just is no excuse for not generating metadata that can be distributed alongside whatever homebrew you cook up.

cYa,

Tauwasser

Jun 20, 2014 • 02:07

blitter

Just chiming in here– IMO this is solving a problem that doesn’t exist. Currently there are no games (or flash cart hardware for that matter) that make use of the expansion RAM, and until there are, we’re putting the cart before the horse by designing a file format that we don’t even know how it will be used. Like Tauwasser hinted at, VB games already know what hardware is used just by virtue of what the code does– if the code talks to SRAM, then it uses the SRAM; if the code talks to the link port, then it uses the link port; etc.

No need to overengineer a special format describing “mappers” or other capabilities when such things are merely hypothetical right now. Just stick with the straight ROM images.

Jun 20, 2014 • 04:00

DogP

blitter wrote:
Just chiming in here– IMO this is solving a problem that doesn’t exist. Currently there are no games (or flash cart hardware for that matter) that make use of the expansion RAM, and until there are, we’re putting the cart before the horse by designing a file format that we don’t even know how it will be used. Like Tauwasser hinted at, VB games already know what hardware is used just by virtue of what the code does– if the code talks to SRAM, then it uses the SRAM; if the code talks to the link port, then it uses the link port; etc.

No need to overengineer a special format describing “mappers” or other capabilities when such things are merely hypothetical right now. Just stick with the straight ROM images.

I agree 100%. If you want to come up with a new format and use it in your emulator… feel free to. But please, let it assume that it’s a standard cart if no header exists, because I’m certainly not going to be adding any headers to my own dumps, since they ARE all standard carts with no need for a header.

And if I was forced to have any sort of opinion on an attached header… it’d be to add it to the end of the ROM. That way the offsets are normal when opened in a hex editor (with some crap at the end). That’d also make it easy for the emu to ignore the header, since it could just always load the largest power of 2 <= the file size. But really, if a header needed to exist, a detached header seems much more reasonable, since it's very possible that a cart that has some crazy expansion stuff will also need multiple ROMs (MCU firmware, FPGA bitfile, NVRAM initialization, etc). If you try jamming all that into some ugly single binary file... you'll need a TOC, or section delimiters, or... DogP

Jun 25, 2014 • 14:48

Guy Perfect

Tauwasser wrote:
If you’re keen on distributing everything on one file, then create your preferred file format(s) and zip them together with the ROM. Problem solved.

This is the idea I like the best. MAME does it this way, and as MAME can testify, a given game can have a lot of little ROMs depending on the system hardware. That should be more than sufficient in all cases, and a manifest can be supplied when/if necessary.

Jun 25, 2014 • 15:08

DogP

Yes, the zip file for MAME ROMs is nice… but the one major change I’d make is to allow the zip to be any name. That’s annoying how MAME ROMs have to be named whatever cryptic name the devs gave the game. I wish they would have dropped a file inside the zip that told MAME which driver(s) to use, and other specifics… rather than hard-coding it by ROM name.

DogP

Viewing 8 replies - 1 through 8 (of 8 total)

Write a reply

News

Database

Tools

Development

Community

You are not logged in

Establishing a ROM Format