[font=Arial][size=24px]Foreword

With the VB20 event–or Virtualfest or Virtual Half-Decade or whatever we’re gonna call it–coming up a few months from now, I’ve decided it was high time to roll up my sleeves, squeeze out my very best effort, and see what I can do to knock the ball out of the park. It’s Guy Perfect overdrive from here on out, and I’m going to do everything in my power to make sure the gettin’s good. (-:

This thread exists as a newsletter of sorts, chronicling my progress and dumping all manner of demos into your lap between now and the main event. I’ll have something or other to present each and every month, though I can’t promise I’ll always update on the 1st. But hey, let’s get this off on the right foot!
__________

[size=24px]Into the Mainframe

My original project idea for last year’s Virtual Fall event was a game. A new game. The design was coming along quite swimmingly, but then I realized in no uncertain terms that I am not an artist. And that was a bit of a problem, since I kinda needed some art for my game. So I couldn’t make that game. That was when F-Zero entered the picture, then exited the picture due to technical difficulties, and then a third project that I didn’t finish designing in time. And that’s why I didn’t have a Virtual Fall submission!

Things are a bit different this round. A co-worker of mine has an astounding graphical ability and he’s agreed to help me with the artwork for my game. So I’m gonna make the game this time, using stick figures if I have to until the super-awesome graphics are ready. And the desktop wallpapers, maybe some posters… We’ll see. I’m seriously considering an original soundtrack album too.

I don’t want to spill all the beans about exactly what the game is and what it’s about, since that would ruin the surprise. There will be plenty of time for hype later on. What I can say for now is that even though the literal gameplay might place you in mountains or forests or cities or whatever, the greater lore of the game universe is actually a highly technical one, inspired by the Virtual Boy itself. There are many abstract concepts from the tech and IT industries being incorporated into the theme and placed in a lush world of critters and scenery, and the way everything meshes together… well, I for one find it fascinating. I hope you guys will too. (-:

[size=24px]Fun For All

Oh, and did I mention all of the assets are being made available to the public? That’s right: the program source code, the concept art, all source graphics and all tools developed for the project are being made available for download and royalty-free modification when the Virtual Boy celebrates its big two-oh.

It’s being developed with devkitV810, which is dasi’s little pride and joy, though I haven’t seen him stop by these parts lately. Maybe he’ll wander in sometime in the next ten months. I’d like to get the compiler and libraries out to the world before the VB20 event.
__________

So with that all out of the way, let’s get this show started. As I said, I’ll be making an announcement and presentation every month, so stay tuned!
[/font]

Since things are coming along, I wanted to share some technical stuffs.

Both Utopia Sound and Utopia Music require the application to allocate object memory, meaning you can declare a byte array on the stack if you don’t want your program to be using some kind of fancy malloc/free setup. They don’t make any assumptions regarding where the memory comes from. When you’re done with it, just deallocate it and that’s the end of that.

Utopia Sound’s Architecture

Utopia Sound is based on two simple object types: Mixer and Sound. The Mixer maintains a collection of Sounds. By implementation, it’s technically a linked list, but the bytes available for instantiating Sounds are allocated once by the application and given to the API. New sounds can be configured and played by the Mixer until memory for new sounds runs out. The back-end algorithm that drives memory management is extremely simple and runs in O(1) time with just a handful of bus operations. The overhead introduced by Utopia Sound is negligible.

Sounds are conceptual little sound channels in and of themselves. They’re driven by an application-defined callback function, meaning there’s no convoluted sound effect or music note format that the library tries to force on you. From the data side, you can actually precompile Sound handler functions and incorporate them into your program as binary resources (or share them with other developers, etc.). Sounds are assigned to a hardware channel, and have a priority value. The callback function simply assigns VSU-compliant register values to the Sound object, and the callback’s return value can signal to the Mixer that the Sound has finished playing.

Each audio frame, as given by the application (usually by VIP or timer interrupts), the Mixer will process all of its currently active Sounds. If a Sound ends, it will be removed from the linked list and placed back into the pool of available bytes. The Mixer will automatically determine which Sound (if any) currently has ownership of a given VSU channel. At the end of the audio frame (usually the first thing after the CPU wakes from halt status following an interrupt), the application invokes a “render” command to the Mixer, which then updates the VSU registers with the values of the Sounds determined to be in ownership of each channel.

Pause and resume commands are also available on the Mixer.

Utopia Music’s Architecture

Utopia Music builds upon Utopia Sound. Specifically, it supplies a flexible Sound handler function that takes care of all of the operations needed for playing music notes. But Utopia Music isn’t a mere list of music notes…

The two highest-order object types in Utopia Music are the Event List and the Event. Playback begins on a top-level Event List known as the Root Event List, and the Events included therein represent the entire track. Each Event has a timestamp, a type field, and whatever additional information is needed according to its type. The following Event types are part of the current spec:

* Event List Call – An Event List can spawn child Event Lists, which are executed in parallel. That is to say, all active Event Lists within a Utopia Music context are processed simultaneously. Event List Calls have a duration field, which will cause the child Event List to repeat from the beginning until the duration expires.

* Note – The elementary music note. Each Note has a duration and a VSU channel on which it plays, as well as dynamic envelopes for each of volume, pitch and panning. If only one value is desired, simply define only one element in the respective envelope (the composer application simplifies this distinction). Otherwise, a note can, for instance, begin to modulate over time to mimic the style of a real-life string instrument.

* Tempo Change – For the Root Event List only, this is a simple two-value gradient for tempo. It starts at the starting value, then gradually changes linearly to the ending value over the time specified by its duration. Tempo Change events present in child Event Lists have no effect.

* Voice Change – Used to reconfigure the VSU channels. This Event type specifies the VSU channel to modify and the VSU wave bank to switch to.

* Sample Update – Used to reconfigure VSU wave memory. This Event will only succeed while no sounds are being played, as per the hardware limitation. It specifies a VSU wave to update, and the index of a wave in the music file to load from.

* End – Signifies the end of the Event List. If an Event List was called with a duration, then reaching one of these events will cause the Event List to reset and start playing again from its beginning. This is great for something like a drum loop or a chiptune arpeggio. When an Event List ends, all of its Notes, and all of its child Event Lists (and all of their Notes, recursively), end. If the Root Event List hits an End event, the music stops then and there.

* Operation – A simple CPU-style operation. Each music context has an application-defined number of bytes associated with it for the purpose of scratch memory within the music program itself. An Operation has a left operand, an operation ID, and a right operand, and produces a result. All three values can come and go through music memory. While the operations do not themselves have any effect on the audio being produced, they can be used to direct the flow of the soundtrack via conditional execution using Goto (see below)

* Goto – Moves the current position within an Event List to some arbitrary location within that Event List. The branch is conditional: Goto will not occur if the result of the most recent Operation (see above) was zero. On the one hand, conditional execution can be used to, say, play the same loop exactly 3 times before exiting. A clever program can allow the music to change dramtically depending on game circumstances (time is running out, the character dives underwater, etc), by writing to the music context’s memory from the game program.

Each Event List schedules its Events on a time scale, which can be either Beats (which changes with the track’s tempo) or Realtime (which ignores tempo). Beats should be used in most cases, but Realtime is essential for chiptune-style arpeggios, which need to play at a particular rate due to the way the human brain perceives them. Individual property envelopes of Notes can also operate on either Beats or Realtime.

Event Lists and the Utopia Music context have volume and panning controls that influence the final intensity of sounds being played. Event Lists have dynamic envelopes just like Notes, while the main context only includes master values to be controlled by the application.

But What of MIDI?

The question came up before regarding MIDI and whether it can be converted to Utopia Music in lieu of not having to learn another composer. While MIDI can be converted to UM to an extent, it’s not really a smooth transition because, as you can see above, it’s apples to oranges.

At the current time, I have no plans to include any conversion utility in the composer, but don’t let that stop you from convincing me it’s a good idea. (-:

I’m so excited to try the Music Utopia. It would be SO cool to be able to compose chiptunes, for the VB no less!

Guy,
If only we could get in touch with Dasi and get the devkit rocking and rolling. The initial hurdle I faced when getting in to VB home-brew was the hodgepodge of tools available. Thank you for your efforts in creating and modifying tools so that everyone else can home-brew better! One day I know we will have a full suite of tools that work together where it might even be somewhat easy to create home-brew VB games.

I’m glad you guys are looking forward to it. I’m working on it every day and the scent of done-ness is growing stronger and stronger. (-:

DogP lent his electronics expertise once again to solve an interesting little hardware anomaly I ran into. See, the VSU outputs a digital signal where the minimum sample value is 0 and the maximum, if all six channels are outputting their maximum samples at full volume, happens to be 684. Since no negative samples come out of the unit, the audio streams look something like this:

“Surely,” I thought, “this isn’t actually what gets sent from the unit. It’d fry the speaker!” And sure enough, it didn’t look like it gets sent verbatim to the speakers. I recorded some audio output from the Virtual Boy and noticed the resulting wave form was… well, it kinda curved a bit. Still effectively the same audio, but somehow or other it automatically flexed itself to balance around +0V. So I asked DogP what was going on.

Since I’m somewhat less than a noob in terms of electronics, I had no idea it was a simple matter of something called capacitive coupling, wherein a capacitor and resistor are used to block any constant DC voltage from a signal before passing it onto another circuit. The long and short of it is that it’s an implementation of the simplest-possible high pass filter with a very low cutoff frequency, and the effective result is that it removes the DC offset of a stream.

Fortunately, since this is just a first-order filter where the only wiggle room is the cutoff frequency itself, I was able to implement it in the Java VSU emulator without issue and now my output is effectively identical to the clips I was capturing from the hardware:

So score another point for accuracy in emulation. (-:

Attachments:

1. 

   vsuanalog_zps4a61d074.png
   

2. 

   vsudigital_zps1f489452.png
   

Turns out I accidentally nerd sniped myself this week. Productivity dwindled as a result. On the bright side, it provided great clarity, and most importantly, it allowed me to come to a decision on how to proceed. (-:

Allow me to provide some background…

Event Lists and Envelopes operate on one of two time scales: realtime or “beats”, which is based on the master tempo. Realtime is simple: it’s just some number of milliseconds. But beats posed an interesting problem because at any given time, the master tempo can change, thereby changing the relationship of milliseconds-to-beats when processing other beats-based events… Okay, so it’s kind of hard to explain. Let’s try a for-instance.

Generally speaking, when writing music, you’ll be using the beats time scale because that’s the whole point of including the tempo mechanic. When Mario’s timer reaches 99 units remaining, the music picks up speed to express a sense of urgency. That’s accomplished with an increase in tempo. Music notes are almost always going to be built around beats: their timestamps and durations need to be synchronized with all the other notes according to the tempo.

Individual Envelopes on a Note can use either time scale, and in many cases you may opt to use both. What’s an envelope? It’s a sort of graph over time that modifies a property of a note (or Event List, or the master Tempo), including volume, pitch and panning:

* If you’re going for a piano sound, for example, there will be a very brief period of high volume followed by a prolonged period of lower volume that gradually tapers out. That’s a good example of when you’d use a realtime-based Envelope. If the attack period takes too long, it won’t sound right, and you don’t want that to be influenced by the tempo of the track (such as if the music slows down).

* If, on the other hand, you want to express some artistic flair in line with the rest of the composition, you may need to adjust volume to make, as an example, the note of a distorted guitar gradually fade out just as another part of the music begins. This would be a situation where you would use a beats-based Envelope.

* Perhaps the same note needs both an attack/decay component and a song-synchonized fade component. If that’s the case, then you can use both time scales on the same Note.

… And that’s the background.

While processing Event Lists each audio frame, starting positions and durations of Event Lists, Notes or Envelopes needs to be relative to how much of the audio frame has actually been processed in the scheduler. For example, if two thirds of an audio frame “elapse” before a Note is played, then the duration of that note needs to be shortened by the remaining one third of the time in the audio frame that haven’t been processed yet. Otherwise, the Note would start too late and end too late.

The first solution was awfully simple: just calculate at the beginning of the audio frame how many milliseconds and beats are actually occupied by that frame and process each Event List, Note and Envelope accordingly.

But then it hit me: while processing Event Lists and Envelopes, the tempo can actually be changed in the middle of frame processing. That would affect how many beats-per-millisecond are occupied by the remainder of the audio frame. What’s the most accurate way to handle that?

Well, it gets kind of complicated after that. I came up with a dozen ways it could hypothetically work, and all of them took a depressingly large number of operations to work out. So I made a call and will be going forward from here:

The first version of the tracker will not reflect changes to tempo during the processing of a single audio frame: the changes will take effect at the beginning of the next frame. This allows the implementation to use my original idea: just calculate the time elapsed up front and use it through the whole frame processing.

I really don’t think that’ll be an issue because audio frames might be something like 80 times a second. Do you really need your tempo to be able to change more than 80 times a second? If it did, would you even be able to tell? I sure as heck couldn’t; the video frames themselves only happen 50 times a second.

Yeowch, when you’re laying in bed for days to rest an inflamed neck muscle and it still hurts, that’s no good. And then you get up and go to work and you get to use that muscle all day long to hold your own dang head up. Needless to say, this week could have been more comfortable!

Remember when I mentioned that I have the audio project so figured-out that I won’t be making further changes at this point? Recall that being a thing what was once said?

Guy Perfect wrote:
[…] I feel quite firmly that I’ve got the specifics down to the point that I won’t really be making them any better through further revisions, so I’m ready to make an announcement…

Okay, so that’s still technically true. The specifics of the audio architecture, the data format and all the many musical features haven’t changed. I’ve just been second-guessing myself on the implementation.

See, I don’t like brute force algorithms as a rule. On large data sets, they’re very slow. So when I started implementing these audio libraries, I knew I’d be looking at an array of objects, where any given one of them might be active or might be available for use. But instead of scanning through the array from the start, checking each object to see whether it was available, I implemented a sort of automated memory manager that operated in O(1) time to fetch the next available object. And it was beautiful.

Then, when implementing music notes and event lists for the music project, I noticed that the link information to maintain object status like that was taking up a lot of bytes per object. And there’s some pretty esoteric code driving the whole thing. And then I had to implement something very similar on top of that so event lists could terminate their child lists and notes when they themselves ended, and the memory usage started to bloat. I kept telling myself, “Look, there’s only going to be like 5 to 10 notes playing at a time, so it’s not like the system is going to run out of memory.”

Of course, I started listening to myself. If we’re talking about collections of 5 to 10 elements, then what would otherwise have been a speed boost granted by the memory management routines no longer saves any time because 10 elements doesn’t take time to begin with. So all I’m left with in its place is some confusing code that operates on a lot of bytes in memory, and frankly, I think that’s a bit overkill.

I’m currently analyzing the architecture to see what kind of speed and memory impact a brute force approach would have. So far, I’m thinking it’s going to be both a lot more streamlined, easier to work with (the source is going public, after all), and in the end not be any slower than the first way I was doing it.

Yet another rewrite is around the corner, but at least it’s only a small part of the greater whole. (-:

Have you given any more thought to a midi converter ? or converters from other audio formats ?

It seems (not sure) that your engine will be for composers that are programers working on virtual boy games 🙂

That’s a specific set a criteria needed. If there was a converter for another audio format…programmers could get midi’s or wavs online, use your music tools to get rid of pops and static (from the source) and add it to their games.

But, i’m not a coder..so I might have misread the whole idea 🙂

-Eric