David Haywood's Homepage
MAME work and other stuff

Ready Steady Deco

February 13, 2012 Haze Categories: General News. 19 Comments on Ready Steady Deco

Charles MacDonald has recently been posting information about the DECO Cassette system in order to get some more tapes dumped because there hasn’t really been any activity on them since way back when the initial batch was added.

This is good news because these things (being magnetic media and custom dongles) are extremely fragile, and the remaining ones REALLY need sourcing and dumping before it’s too late. Of course, all that work isn’t much good if there’s nothing to show for it, but luckily there is. Here is ‘The Grand Sumo’ one of the previously undumped DECO Cassettes. In the end it was a simple slot-in to the driver (Type 4 dongle, BIOS Type A) so I can’t really take credit for anything, but Charles’ fantastic work, and the contributors who have helped buy the game and equipment needed deserve a shout out here.


The Grand Sumo The Grand Sumo

AFAIK the Dumping Union *don’t* currently have access to the other rare games, and if they show up they’re bound to be pricey, so please consider donating to them if you want to help source these rare and fragile pieces of history.

One of the most important things to note about Charles’ work is it means the dumping is now a NON-DESTRUCTIVE process. The dongles are dumped without even opening them, so it’s risk free compared to older methods. This is good news for people who own the cassettes if they’re considering loaning them out.

Credit this time round goes to Charles MacDonald, Dr. Spankenstein, Kevin Eshbach, T. Huff, SteveS, E. Page-Hanify, Hikari, ArcadeDude, F. Bukor, N. Francfort, jmurjr, arcade-history.com, ThumB, Hurray Banana, Paratech, Xiaou2, Cornishdavey, A. Costin, M. Ponweiser, Tormod, Rambo, Smitdogg, The Dumping Union

Go to article.. »

7-Zip-a-Dee-Doo-Dah

February 11, 2012 Haze Categories: General News. 36 Comments on 7-Zip-a-Dee-Doo-Dah

As promised below I’ve put a bit of time into hooking up 7-zip support in MAME. This is a first pass, so there is still room for improvements (both to code quality, and potentially bug fixes)

The way I see it zip / 7z offer the following pros / cons

.zip
– Aging 32-bit format
– Can only hold 2gb of data (we’re hitting that limit in some cases)
– Only supports UTF-8 filenames
– Single, dated algorithm
+ low memory overhead

.7z
+ Modern 64-bit format
+ No practical limitation to filesizes
+ UTF16 support (debatable advantage)
+ newer more advanced algorithms
– higher memory overhead if ‘solid’ blocks are used

Zlib (.zip support) might be tried and tested, but it’s showing it’s age, it’s a format of the previous decade when FAT32 was the standard, and nobody had large files. .7z is newer and doesn’t have many of the inherent issues of an older format, although does still need some testing. Staying ahead of the game is always a good idea tho, better to be modern and be prepared.

Download the diff here. This has been sent to R.Belmont for portability checking and official inclusion. Right now this can be used for roms, artwork, samples, cheats etc. -romident and romcmp.exe aren’t hooked up to it yet, but that will come with time. As I said, this is first pass, and a demonstration it works.

I’ve tested this with a wide range of settings in the Windows 7-zip binary and haven’t managed to make a file which fails to decompress yet. Both 32-bit and 64-bit Windows MAME builds have been tested and as long as you’re sensible about block sizes and compression used speed / memory overhead is comparable to standard .zip (ie don’t use PPmd with huge block sizes on huge ROMs because that would just be stupid and can take a good 3 minutes to decompress kof2003.zip even with the official tools ;-)

I’m increasingly being sent .7z files when asked to look at things to work on in MAME as opposed to zips in days gone by. I’d attribute a lot of this to the decline of WinZip (they practically nuked their own business model) as well as the horrendous build-in Zip support Windows has, forcing people to look for alternatives. This means having .7z support makes MAME work easier as I don’t have to decompress / recompress everything just to check it out. Weigh that up with the limitations of the format and moving forward seems like a good idea. I’m also often sent .rar files (because people appear to have flocked to either 7-zip or WinRAR in equal numbers) but I need to do further investigation on the suitability of decompression code for those, there are a few strings attached because it’s not a ‘free’ format.

Of course, the original .zip support remains, .7z is just an option. There is no reason to rip out the long standing .zip support, merely supplement it with something more modern.

The other benefit of course is that the LMZA and LMZA2 algorithms will be in the MAME code for leverage elsewhere, and would be ideal for use in CHDs and such. Not done anything on that front tho because it’s not a priority, and the FLAC work is still pending cleanups and being officially turned on.

Go to article.. »

Donkey Kong Does Audio

February 6, 2012 Haze Categories: General News. 29 Comments on Donkey Kong Does Audio

Ok, this update has nothing to do with Donkey Kong at all, Donkey Kong doesn’t even use samples anymore, but nevertheless it is about Monkeys.

Actually, maybe it isn’t about Monkeys at all either, but it is about the lossless audio compress format ‘Monkey’s Audio’ or, those .APE files you sometimes find.

Monkey’s Audio (MA) is one of the other big players in the lossless audio compression arena along with FLAC, and WavPack.

The main advantages of MA is that it offers better compression rates than FLAC, which for large files can be quite significant. The disadvantage is it’s slower, quite significantly slower. It’s a symmetric algorithm which basically means decompression takes the same amount of time as compression.

0.145 saw the FLAC code being enabled in MAME for use with samples (not yet CHDs), and as a test I’ve done a bit of porting and brought MA over with the same sample decoding functionality. It took a bit of work keeping the compiler happy (wasn’t quite 64-bit safe), and I’ve opted to compile the non-windows codepaths on Windows to ensure that it’s as portable as possible, but it works.

As expected, it’s quite a bit slower than FLAC, and while it’s not quite as fast as it could be due to me disabling some x86 ASM / MMX code the sample implementation still gives a pretty strong indication that it wouldn’t really be fast enough for decoding of CHD data where we need data to decompress with minimal overhead. I know a few people had been wondering / asking about this, which is one reason I looked into it. I might do a hookup later to test, but IMHO it’s not worth it.

You can download the source diff to add MA support to samples here (I don’t claim it to be pretty!)

Also for reference the following Gorf samplesets (note, some files are not recorded, so MAME will still show ignorable NOT FOUND errors on those rather than silently failing as it would prior to 0.145, I’ve also resampled one of the clips to 8-bit to show 8-bit decompression is working)

As APE (860,649 byte zip)
As FLAC (905,243 byte zip)
As WAV (1,110,709 byte zip)

I wouldn’t hold your breath on this being officially included, although I’ll submit it anyway. More of a fun little experiment than anything else.

Go to article.. »

15 Years is a Long Time

February 3, 2012 Haze Categories: General News. 7 Comments on 15 Years is a Long Time

Depending on who you ask MAME was born either on the 5th Feb 1997 with the release of Mame 0.1 or slightly before that in the form of Nicola’s MultiPac releases which were the basis of that release (even today you can see some remnants of how they grew to become MAME, for example the graphic decoding structures)

The official anniversary is generally considered to be the 5th, so, this coming Sunday and the date has come to correspond with the release of a new full version, 0.145 in this case.

I am slightly concerned at how tight things are running as far as bug fixes go. Aaron’s recent batch of changes have left a number of Saturn / STV games broken*, clickabale artwork broken, the tilemap viewer in indexed modes broken, the cheat system broken, and possibly one or two other outstanding bugs caused by the tag changes which have yet to be shaken out. Kale’s recent changes also mean that many games in metro.c are still broken and a couple in suprnova.c as well, then you’ve got some longer standing regressions too such as some of the Taito GNET games, broken PSX hardware quads and a couple of other (looking further back there are still the performance regressions from some memory system updates..)

I’ve been doing my bit to clean up / fix some of the recent issues, as well as some which have been lingering from the Data East rewrite I did a bit ago, while converting the remaining drivers over to use the generic handling but I think for the highlighted issues it’s going to take a big push from the people concerned (Aaron, Kale etc.) either today or on Saturday if they’re to be fixed in time for a proper stable anniversary release. I have a feeling official support for the FLAC CHDs won’t make such a release due to the priority of other stuff being higher, although I have just submitted code to allow sound samples to be in either format which will hopefully make the build.

It’s good to try and make anniversary builds special, throw in a few surprises. The 10th anniversary was especially cool (5 years ago!) with a good number of interesting things, and a special modern build of MAME 0.1 to boot. I guess it’s also worth mentioning that PC emulation in MESS recently improved to the level whereby you can actually run the original MAME 0.1 binary inside of MESS (albeit slowly!) which is pretty neat. If there are any surprises this year they’ve been kept well under wraps, I did want to sneak one in myself but the contact I was working with on one has been unavailable for a month, so progress has temporarily stalled there, and the other possible one I still haven’t managed to get the green light for inclusion, always next year, or the year after.

Either way, 15 years is an impressive achievement and while the project isn’t pulling up trees at the same rate it was you do have to wonder where we’d be without it, how many other projects and current games / ports would be different without it providing a reference point for all to use. It’s also been a great eye opener to arcade culture around the world, as unpopular as things like Mahjong are with the non-Japanese crowd the sheer number of titles listed is testament to the success of the genre in Japan, likewise how many people would know Korea had it’s own arcade industry, producing games which were mostly ripped from more popular ones? How many people have had their eyes opened to companies like Cave by review sites written by people using emulators? How many people have picked up programming skills, and an understanding of obscure assembly languages by trying to contribute to MAME? As an educational tool it’s been immense. There aren’t many projects which survive 15 years where there is still just as much can be done today as there was back then. MAME is almost as old today as some of the first hardware/games it emulated were when 0.1 was released, think about that one for a while.

So… even if 0.145 does come out and ends up being a bit buggy you can also look on the bright side, at least you’re still seeing regular releases 15 years from when the project started, and everybody has access to the latest code via the GIT repository these days so you can build yourself a fixed build as soon as any fixes go in.

* It’s possible the Saturn breakage (Radiant Silvergun specifically) occurred earlier, but I believe it was traced back to that. Likewise the cheat issue might be fixable by changing all existing cheat files, but I’m not sure that’s desirable behavior.

Go to article.. »

Sliver Place

January 18, 2012 Haze Categories: General News. 1 Comment on Sliver Place

Integrating FLAC isn’t the only thing I’ve been working on lately. After the initial work on FLAC (prior to the CHD bit) I decided to integrate some proper JPEG decoding libraries into MAME.

Why? Well, there is one game, which has been running in MAME for a while which actually uses JPEG decoder chips on the PCB. This game is the rather obscure ‘Sliver’. As a game, it’s not even remotely interesting, or good, but from a hardware perspective the pair of Zoran chips used to decode the JPEG images stored in the ROMs make it an interesting case.

Until recently MAME has simply loaded an extra ‘fake’ rom with the JPEG images pre-decoded, and used a look-up table in the driver to convert the addresses requested by the game for the JPEGS into addresses where our pre-decoded bitmaps were stored. Obviously this isn’t how the hardware worked ;-)

By integrating the JPEG library I’ve managed to remove the fake ROM, and can now point the decoder directly at the real JPEG data in roms as and when it’s requested by the game.


Sliver Sliver

Note, you can see ugly JPEG artifacting on the game screen, they obviously didn’t compress the images with very high quality settings ;-)

From an end-users perspective, the game still runs and plays exactly how it did before, but from a MAME perspective the emulation is now better, and we have a JPEG library to play with if we want it, maybe as an option when creating screen-shots at high resolutions where PNG becomes excessively large? Either way, it’s there and could be put to further use.

There is one other game which uses the same JPEG decoder chips on the board Magic the Gathering: Armageddon (Progettoemma link due to MAWS being down after it was used to hack MameWorld) Currently that game doesn’t work, but Phil Bennett was last seen getting some 3D out of it, so having a JPEG decoder should help him get the correct textures, which are encoded as JPEGs in the ROM.

As with the FLAC stuff, thanks also goes to R.Belmont for ensuring it compiles / works on Linux and Mac environments.

Go to article.. »

Taking the FLAC

January 18, 2012 Haze Categories: General News. 70 Comments on Taking the FLAC

One of the criticisms often made of MAME / MESS’s CHD format is that it doesn’t actually provide very efficient compression, especially when it comes to CD AUDIO data. I’ve had a number of people ask me if I can look into improving this, especially when you consider that in with the current format a complete Saturn set is almost 1TB, with a large portion of that being AUDIO.

The reason it’s inefficient is because it’s using zlib’s inflate algorithm for the blocks, blocks which are rather small to ensure that data is decompressed quickly. While this is fine for DATA (it’s the same thing that ZIP files use) it’s absolutely hopeless for AUDIO.

There are dedicated audio lossless compressors out there, FLAC is a popular one.

I’ve spent the last 4-5 days solid integrating support for this into the MAME / MESS tree, and extending the CHD format to not only support it’s native blocks (hunks), but also reference to embedded streams via ‘virtual hunks’ which point at a stream, and allow the actual FLAC codec to do the seeking and decoding work for this.

By doing this I can achieve a good level of compression with FLAC, far better than trying to split it into CHD hunks due to the lower overhead, and improved ability of the compression algorithms to predict how data best compresses. I also still get good speed decoding, as the FLAC format is designed to be quick to seek, and has built in seektable support of it’s own which I’m levering.

I have to say FLAC is an absolute joy to work with, the API does everything you can expect, the documentation is great, and it’s very good at letting you know if something is wrong. (the only issue I had with the documentation / API was with the seektables, whereby calling things in the wrong order / wrong time during encoding could cause data to be overwritten without throwing an error)

I’ve also added support to the MAME SAMPLE interface to playback files from FLAC sources, this should allow the recently dumped tape loops to be compressed much better than they are now (they’re uncompressed PCM .wav files)

The other possibilities for this are endless, -wavwrite could also output FLAC data if support was added, MESS could potentially load cassette based software from FLAC images. It’s an incredibly useful codec to have around.

I’ve uploaded my first pass of this code Here (link offline for the time being, there is definitely still an error). This should be considered ALPHA SOFTWARE and I won’t be held responsible if you end up destroying your CHDs with it. I’m currently in the process of batch converting many images and haven’t found a broken case yet, but still, it’s in testing. While I’m happy with the current format extensions and CHD format created it could change in a final version, you have been warned.

This code has been submitted to R.Belmont, who is currently making some portability fixes. FLAC is designed to be portable, so this shouldn’t be too much of a problem, so fingers crossed it can be sorted out soon.

Usage is simple, I’ve added an additional -createcdflac commandline option which will use the FLAC routines when compressing AUDIO. If you already know how to use CHDMAN then it’s simple enough.

Have fun :-)

Go to article.. »

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close