It’s finally happening for real this time! Working behind the scenes with some community members, I decided to take an honest swing at an in-house PVB emulator at a full-time capacity. This will be my actual day job. So for the first time in history, I’m going to put my skills to work and produce some actual results. (-:
This community has always been amazing, and I have a good feeling about the emulator initiative going forward. The official project page can be found here:
http://perfectkiosk.net/
__________
So why should you care? Aren’t there Virtual Boy emulators out there already? Let’s see what the project page has to say…
The initiative has four primary objectives:
Nintendo 3DS Emulator
I’m sure most of us thought about Virtual Boy when we first heard about the Nintendo 3DS. A custom emulator specifically designed to run on 3DS can take full advantage of the system’s power and capabilities, giving us true stereo Virtual Boy on the go.Reverse Engineering
Any time we want to learn about how a game works or make changes to it, we need the right tools to dig in and take a look around. A cross-platform desktop application is just the ticket for providing the user with the right kit to expose all the deepest secrets buried within Virtual Boy software.TAS
The tool-assisted speedrun is a cornerstone of video game culture, and like reverse engineering, we need the right tools at our disposal in order to get the job done right. The aforementioned desktop application pulls double duty by bringing TAS-focused utilities to the party.Documentation
Comprehensive documentation of the Virtual Boy’s internals will come from emulator research, and it’s a perfect way for homebrew authors and modders to learn about the system’s features. Those who remember the Sacred Tech Scroll document I wrote can look forward to a brand new version of it.
For further reading of what exactly is planned, refer to the technical document trail on the project page.
__________
How exactly are we gonna pull this off? Glad you asked…
I have enough money in savings to begin work immediately (2018-04-18), but within a few weeks, I’m going to need to rely on financial support from the gaming community. We’ll be reaching out to other sites to gather support, since this undertaking spans a broader scope than what solely applies to Planet Virtual Boy.
A conservative estimate of my monthly expenses comes out to around $500 USD, and with enough interest, I’ll be able to keep this up full-time. By “full-time”, what I mean is 40 productive hours every single week, with status reports and a detailed outline of the development requirements. Planet Virtual Boy has its own developer working for it to make this dream come true. (-:
Donations are managed through a Patreon page:
https://patreon.com/GuyPerfect
__________
For now, this thread will be the place to be notified of progress and for general discussion of the initiative. Thanks again for being such a great community, and for giving me a chance to do what I really love. <3
Special thanks to Fire-WSP, STEREO BOY and KR155E for their assistance in getting the ball rolling.
--Guy Perfect
- This topic was modified 5 years, 8 months ago by Guy Perfect.
Although the outward functionality appears to be relatively the same, reworking the application UI to accommodate linked pairs of emulation windows was a lot of intricate work. But it’s done now!
Attached to this post is a new build: (See following post)
New Features
• The main application window has been redesigned
• The communication port is now implemented
Code Changes
• Unmapped reads in the [font=Courier New]0x02000000-0x02FFFFFF[/font] range will now be zeroes
• The localization driver was updated to support substitutions
• A new class MainWindow was created for representing the application interface
• Multiple windows can now be instantiated and linked
• Global input commands are now implemented and are configurable
• The Config class and its new helper classes are in a new “config” package
Bug Fixes
• Writes to DPSTTS were forwarding to DPCTRL, which I don’t think is correct
• Simultaneous Left/Right and Up/Down are disabled by default, correcting a crash in Wario Land
__________
To use the new communication feature:
• Create an additional emulator window with File → New Window
• Associate communication peers through Emulation → Link
• Use the emulator windows as you normally would
When only one emulator window exists, the Link sub-menu will not be available. When multiple windows exist, the Link sub-window will display a list of all current windows, allowing the user to select one to associate as a communication peer. If the window is already linked, a Disconnect option is also available in the Link sub-menu.
Communication Workshop has been attached to this post to help with testing.
I did a lot of experimentation with boosting performance when linking simulations, but the unfortunate fact of the matter is that the only way to keep linked instances synchronized is to process them one cycle at a time. I thought maybe checking if both simulations were in halt status I could boost performance by skipping to the next interrupt, but communications don’t always raise interrupts, and there’s no way to predict ahead of time when the software is going to trigger a communication event…
… So long story short, linking in the current Java implementation is slow. Hopefully the eventual native implementation can handle linking at full speed.
Each emulation window animates frames in its own thread as a consequence of how timers work. However, linked frames will only use the animation thread from whichever window initiated simulation processing. I’m pretty sure I got all the logic correct, but since thread locks are rather intermingled under a variety of circumstances, it’s possible there might be some combination of circumstances that could cause the application to deadlock.
The following describes the general contract of the linking behavior:
• Unlinked windows operate in their own threads
• When paused windows are linked, the one that invokes the Run command controls both
• When running windows are linked, the one that invokes the Link command controls both
• When a running and paused window are linked, the paused window is activated and the one that invokes the Link command controls both
• When running linked windows are disconnected, the “slave” window resumes control of itself
• Closing a linked window invokes the Disconnect command
• The following commands will only apply to the linked window that invokes them: Reset, Frame Advance, Step Over, and any user-defined breakpoints
Keep an eye out for any reproducible problems that might occur with thread management, and please report any problems in a reply. I did my best to make sure it works correctly, but this is a bit of an atypical method of threading windows and I don’t want to say with absolute certainty that it can’t break.
__________
Besides the link feature, the main window was redesigned and all of the old features remain implemented. However, the only new feature that was implemented is the Game/Debug Mode setup, with additional menu options to be implemented later. One thing to keep in mind with regards to window modes:
• In Game Mode, emulation will always begin when a ROM is loaded
• In Debug Mode, emulation will always be paused when a ROM is loaded
This should be most useful to the most users and might wind up in the Settings window depending on what I decide on the matter.
Ctrl+3 still cycles anaglyph colors, but it is a temporary command that will be removed once the Video Settings interface is implemented, which will allow the user to configure whatever colors they wish.
The debugger interfaces still do not contain any safeguards while the simulation is running, and should not be used unless the simulation is paused. This is also temporary, as I didn’t want to delay this build any longer than necessary.
Step, Step Over and Frame Advance can now be used while the simulation is running. All three will cause emulation to pause after they complete.
- This reply was modified 5 years, 9 months ago by Guy Perfect.
Turns out I’d forgotten to put the new “config” package into the .jar file, so the previous upload did not work. The one attached to this post should be fine.
Also, after typing out why I had to process linked simulations one cycle at a time, it occurred to me that I could check for communications while both simulations were in halt status even if the communication doesn’t raise an interrupt. This build takes that into account and sees vastly improved performance over the broken one that you never got a chance to use anyway because of the aforementioned .jar mistake…
Attachments:
A new build is attached to this post:
New Features
• Audio is now implemented, disabled by default, toggled with Ctrl+A
• Added disassembler comments for VSU registers
Code Changes
• Rebuilt the Console class with new threading, timing and audio code
Bug Fixes
• Items in the Link menu will now retain the check mark when selected
• Triggered breakpoints will no longer steal focus from other main windows
• Corrected the address that presents BKCOL disassembler comments
• Disassembler comment detection for I/O ports now correctly masks address bits
The key attraction this time around is audio support, and the Sacred Tech Scroll has been updated as usual. There’s very little left for the document before the formal v1.0 release, and the emulator application just needs a few menu items implemented before it’s quote-unquote “ready” for its own version 1.0.
If you give the audio a go, you’ll probably notice pretty quick a bit of a glaring problem: it’s a bit too much for Java to handle unless you’ve got a hulking monster of a computer. On my own box, most games stutter pretty badly, spending more time between frames than presenting them, which is why audio is disabled by default. This isn’t a problem that’s going to go away without a native implementation driving the workload. The JVM profiler does its best to optimize the emulation core, but there’s just too much going on for it to keep up.
Audio does work correctly, though, and dare I say perfectly. Attached to this post is a .ogg file that is a log of audio output from the emulator during a short Wario Land session. If you open it up in an audio editor, you’ll see the actual samples produced by the emulator, including the simulated analog filter that centers the output around zero. All of the samples are unedited, exactly as they came out of the emulator. The only thing I did to the clip was trim off some silence at the beginning.
Also attached to this post is an IPS patch (use it with something like Lunar IPS) that will modify Wario Land slightly in order to use the [font=Courier New]HALT[/font] instruction while waiting for VIP activity. The disassembler and breakpoints came in real handy for this! Performance is improved relative to the unmodified game, but it’s still not quite fast enough on my box. That native implementation is going to be the key to getting performance back up to where it needs to be, but that’s still a ways out yet.
Lastly, a copy of VSU Workshop is attached to this post for testing purposes.
All of the internals regarding the frame animator were completely redone in this build, meaning a completely different set of potential bugs now applies. If you could play around with the window commands and linked window communication to check for bugs, it would be appreciated.
Attachments:
With the entire emulation core implemented, with the experience of producing the emulator of my dreams, and with a sigh of relief, I’m finally freed up to do other things. I won’t be abandoning this project–far from it–but as of this latest update, it is no longer my primary spare-time focus. I’ll still work on it from time to time when other things don’t have my attention, but since interest in this project is very low, there are other things I want to be doing that I’ve been putting off.
The remaining window menus will be implemented in the coming weeks, but I won’t be sticking too strongly to my Wednesday update schedule like I have been for the past couple of months. If someone is keenly interested in something that can’t be clicked on, I’d be happy to prioritize it to make it available, but if it’s just a matter of “it’s not done yet”, then I’ll get to it when I get to it. (-:
As for what I will be doing, there’s 1) a long-neglected project with some other PVB members I’d like to get the ball rolling on, 2) something I want to do with Wario Land and 3) something related to NES that I’d like to keep a complete secret until the time comes.
For those who have stuck with me through all of this, thank you for your patience and thank you for your financial support. Planet Virtual Boy’s emulator is becoming something great and that means the world to me.
Good work as usual but I have to say Bye Bye 3DS VB Emulator.
Reading the last post this will never happen anymore.
Shame.
And you entirely missing the point. There is and was interest in the Emulator but as I said from the beginning, the people today do not care so much about the next PC emulator or debugger. That is only us here who care about such stuff mostly. And if people care for emulators they want it to work with stuff like retroarch as libretro compatible emulator.
The Patreon page was not going too bad for a while but now it is almost to zero after such a long time of nothing. And if somebody would have listen to me from the beginning, patreon would still go strong. But without any 3DS stuff it is no wonder that is not working.
Sorry, I am very disappointed!
So okay, who is next in line for doing finally the right thing???
Well that audio for what can be made out is clean clear and impressive. I’d hate to think what machine you’d need to get this to run right with the audio on. I’ve got a mid-range i7 from 4 years back and an nvidia 980 with 8GB of ram on that and 16gb on the system and it is chugging to say the least.
Also are some games implemented and others not on this? I tried to load up Red Alarm, Golf, and one other and it never started them up while others did (Vertical Force, Wario, Tennis.)
And thirdly the menu is grayed out I noticed for configuring the emulator settings for audio, visual, inputs.
VmprHntrD wrote:
I’d hate to think what machine you’d need to get this to run right with the audio on.
The JVM is able to get short burts out–longer than the 1/50 second rendering interval–so I know it’s not a limitation of CPU power at the lowest level. The native implementation written in C should shine a better light on this, since it doesn’t have any of the sandboxing overhead that the managed Java code has to deal with.
VmprHntrD wrote:
Also are some games implemented and others not on this?
The emulation core in its current state isn’t 100% compatible with the commercial library. Red Alarm in particular is mostly supported, but stopped booting after I adjusted the number of CPU cycles taken by a store operation. I don’t know what the game is doing that it would even be aware of how long a store takes, but getting the game up and running will require some investigation in the debugger. It’s a great example of why the CPU and breakpoint features are so important to have!
If anyone is technically inclined and would be able to help look into Red Alarm specifically, that would be greatly appreciated.
VmprHntrD wrote:
And thirdly the menu is grayed out I noticed for configuring the emulator settings for audio, visual, inputs.
Anything greyed out (except for the Link menu when only one window is active) isn’t implemented yet, so that’s not an error. If you want something specific to help with testing, let me know and I’ll make a point to get it set up for you.
Fire-WSP wrote:
And you entirely missing the point. There is and was interest in the Emulator but as I said from the beginning, the people today do not care so much about the next PC emulator or debugger. That is only us here who care about such stuff mostly.
Every part of the project is important, which is a much broader scope than running the core on 3DS. The documentation and Java implementation account for some 80% of the project goals, and they’ve taken a fair amount of time and effort to get to their current state. The CPU and breakpoint debuggers in particular have been instrumental for research and bug-hunting, so it’s not like any development time has been spent on frivolous things.
The remaining 20% is still on the way, but it’s abundantly clear that I’d be doing it mostly for myself. The invitation still stands that if someone needs something implemented I’d be happy to work on it, and the source code for every build released has been bundled inside of every .jar file.
Fire-WSP wrote:
[…] I have to say Bye Bye 3DS VB Emulator.
Reading the last post this will never happen anymore.
Shame.[…]
So okay, who is next in line for doing finally the right thing???
Overlooking the part where I said exactly the opposite, is this really the tone you want to convey?
Rude comments weren’t they?
Anyway, nothing I’m doing in any particular bit, just recalled an earlier build had stuff like that enabled at the least so you could setup your own button configuration on the keyboard so that’s why I asked.
It’s a nice try at an emulator definitely, guess it’s just java being java perhaps holding things back some due to how it works.
Guy Perfect wrote:
With the entire emulation core implemented, with the experience of producing the emulator of my dreams, and with a sigh of relief, I’m finally freed up to do other things. I won’t be abandoning this project–far from it–but as of this latest update, it is no longer my primary spare-time focus. I’ll still work on it from time to time when other things don’t have my attention, but since interest in this project is very low, there are other things I want to be doing that I’ve been putting off.The remaining window menus will be implemented in the coming weeks, but I won’t be sticking too strongly to my Wednesday update schedule like I have been for the past couple of months. If someone is keenly interested in something that can’t be clicked on, I’d be happy to prioritize it to make it available, but if it’s just a matter of “it’s not done yet”, then I’ll get to it when I get to it. (-:
As for what I will be doing, there’s 1) a long-neglected project with some other PVB members I’d like to get the ball rolling on, 2) something I want to do with Wario Land and 3) something related to NES that I’d like to keep a complete secret until the time comes.
For those who have stuck with me through all of this, thank you for your patience and thank you for your financial support. Planet Virtual Boy’s emulator is becoming something great and that means the world to me.
This is great news about your other projects! Looking forward to all of them!! Thank you for all that you do.
Finally got to play around with this last night. This is a GODSEND for VB devs. You’re the bomb Guy.
Do you plan on getting this up on Github? I think that would be a great place to host builds/report issues/make pull requests.
VmprHntrD wrote:
It’s a nice try at an emulator definitely, guess it’s just java being java perhaps holding things back some due to how it works.
Fortunately, I was thinking ahead when I wrote the Java implementation of the emulator core. I did it in such a way that converting it to C would require minimal modification, so for the most part it’ll just be a copy/paste job and changing some data types… and turning all of the comments into [font=Courier New]/* C style */[/font]… I even plan to make functions with names like [font=Courier New]Integer_compareUnsigned()[/font] and [font=Courier New]Float_intBitsToFloat()[/font] just to keep the two code bases that much more parallel.
The JNI module that will expose the C implementation to the emulator application only needs to implement the methods declared in the VUE interface, which itself isn’t a huge deal. It’ll be a lot of busy work getting the ~236 KiB of Java source code moved over to C, but it’s nothing that requires anything resembling expertise.
Plus, it’ll give me a chance to review every line of code just in case I think of something along the way that could improve it.
trailboss wrote:
Do you plan on getting this up on Github? I think that would be a great place to host builds/report issues/make pull requests.
My opinion of Github aside, I haven’t been bothering with version control so far since I’m the only one working on the project. And even if one or two other people do get involved (which doesn’t seem likely), I’ll still probably not elect to set up a repository for it. And the source files are all in those .jar files, so it’s not a matter of making them available.
In short, using version control at this time would just be giving myself more work without adding anything of value. (-:
Guy Perfect wrote:
I haven’t been bothering with version control so far since I’m the only one working on the project.
Ha, I know how this story ends just from experience. Jokes aside, a place to formally submit bugs would be nice
Good job on the forward thinking (Java to C ease of conversion, and not wasting your time either since the source is in the jars.)
I tried to figure this stuff out in college 20 years back and well I had such a horrible teacher I was assigned to that most the class failed and it turned me off it entirely. Never learn from an angry little asian guy who thinks there’s only one way to code a result, his way, or you fail, as that’s how that went. Only D I had since like elementary school. 😛
If I could, I’d love to do the work. I think what you have there are some really solid bones and with the source in those JARs perhaps someone can take a crack at it. Mednafen SUCKS and their VB implementation is sketchy as it is all hidden behind the most anti-user friendly program out there. I think there would be interest as I said as people have been picking at it like with the VBjin project and others. You have the advantage of having a far better base to work from that does things correctly.
Perhaps put out some feelers, see if someone bites.
trailboss wrote:
Ha, I know how this story ends just from experience. Jokes aside, a place to formally submit bugs would be nice
Hasn’t failed me yet. Just look at this great emulator we have already!
Bug reports go in this thread. Don’t worry, I’ll see them! You have bugs to report, don’t you?
Raverrevolution wrote:
How do I load this up in my 3DS?
You don’t, just yet. The most recent development finished up the hardware functions in a Java implementation, which is designed to run on a workstation. The next phase (after filling out those greyed-out menus) is to port the implementation to C so it can be built as a native library, then use that on 3DS.
I’d have been working on 3DS before now, but Nintendo for some unknown reason refuses to let the silly thing die. I don’t want to risk getting a homebrew solution set up just to have a firmware update blow it out of the water. And as of the time of this post, Nintendo released a game for 3DS (Kirby’s Extra Epic Yarn) a mere 24 days ago, meaning they’re not ready to take their eyes off of it just yet.
Let’s hope Kirby is the end of 3DS. I’m really itching to sink my teeth in.
I spent the last week working on my Wario Land project, which has had a fantastic start, but this isn’t the thread for that. The reason I’m posting right now is because tonight, out of curiosity, I decided to do a little research into the JNI to see what I would need to do in order to get the native emulation core set up. I figured it would be some involved process with headers and libraries every which way, but that’s not the case. In fact, working with the JNI is easy. Really easy.
It’s so easy, in fact, that I’ve decided to turn my immediate attention toward getting that native implementation set up posthaste. There’s a little bit of added work in getting the code moved over from Java, but the vast majority is done as the Java implementation in and of itself, so it shouldn’t take more than a handful of days. I’ll keep you posted.
Whoooo boy, that was… that was an experience.
I spent literally every waking hour the last two and a half days working on setting up the native module stuff, and it has been very successful. However, as you can see by the lack of anything to download attached to this post, it hasn’t been “ready for prime time” successful. Very, very nearly, but not quite.
When I finally got everything put together and flipped the audio on… the exact same stuttering and slow-down occurred that was present in the Java implementation. This of course made me suspicious, as having identical performance in the native module is preposterous. But my earlier troubleshooting involved disabling audio rendering in the Java code in exchange for some supplied zeroes, and the speed went back up. I reasoned the audio processing itself was to blame…
After the curiosity with the native audio, I switched back over to the Java implementation and merely disabled sending the rendered audio to the speakers. That also ran full-speed. In fact, it stayed full-speed while logging audio to a file, despite being the Java code. So it’s not the processing overhead at all! My best guess is that the earlier test “worked” because the initial contents of the buffer was zeroes and I was writing zeroes into it. Java may have found a way to make that work.
As far as what this means for getting good, full-speed audio output in Java, that one’s got me a bit worried. VSU Workshop manages to run all six channels full-speed in the Java code, and the native implementation has more than enough muscle to get the job done, so exactly what Java is doing or how to make it do what I want is a mystery.
Truth be told, Java Sound isn’t designed to be used as a timing source like the emulator requires. It doesn’t actually provide any means to determine how much audio has finished playing or any mechanism for signalling the application when a buffer has finished. What I’ve been doing so far is a not-exactly-documented use of SourceDataLine.available(), and it’s obviously causing problems. That was my best effort, though, and I’m inclined to think that perhaps Java Sound isn’t going to be a good option.
__________
In other news, porting the Java code over to C was pretty much as smooth as I was expecting it to be, with two exceptions: the VIP rendering code and pretty much everything to do with breakpoints.
The VIP code is a couple of monstrous functions with many variables that were designed to produce the images as efficiently as possible, which they do rather well for the most part. Porting those to C proved to be a bit troublesome, as I did a lot of sign extensions with Java’s shift operators (behavior that is guaranteed in Java but not necessarily in C) and a few other Java-centric design choices. After converting to C with the associated adjustments, there are some bugs in the rendering code that I’m not happy with. I’ll either have to try again or write new rendering code to make sure it works in both languages.
The breakpoints library is another matter altogether. Unlike the emulation core, I didn’t take care to design breakpoints in a way that would easily adapt to C. The native version of breakpoints that I came up with has some drastic differences and what code did make it over is ugly as heck after the needed modifications, and I’m overall disappointed with the whole thing. They function well enough to get games running (at full speed, naturally), but beyond that I want to toss them in the garbage.
There’s a mounting pile of other considerations that would require extensive refactoring to incorporate new things I’ve learned along the way. At this point the prospect of a total project do-over seems very appealing, but I haven’t decided what I want to do going forward.
- This reply was modified 5 years, 8 months ago by Guy Perfect.
Thank the stars, that’s music to my ears. tl;dw: Nintendo mentioned during an investors meeting that they have nothing coming down the pipe for 3DS in 2019. It’s about frickin’ time.
Eight years for a handheld is a hell of a run. The GBA lasted 9 years, the original Gameboy was 9 (1989) alone 14 including the Color (1989-2003). Interestingly the DS line also lasted 9 years 2004-13.
The GBC being the extension system, if you look at the main line of units, they all seemingly have lasted with the overlap to more or less 8-9 years. That’s a good long run for a single piece of hardware with minor changes/revisions.