Skip to content

Add Metal Renderer and native macOS/iOS support for TIC-80 - #2973

Open
Wang-Yue wants to merge 1 commit into
nesbox:mainfrom
Wang-Yue:pr/apple-metal-backend
Open

Add Metal Renderer and native macOS/iOS support for TIC-80#2973
Wang-Yue wants to merge 1 commit into
nesbox:mainfrom
Wang-Yue:pr/apple-metal-backend

Conversation

@Wang-Yue

Copy link
Copy Markdown
Contributor

This commit introduces native macOS and iOS support using Metal, AVAudioEngine, GCController.

This commit introduces native macOS and iOS support using Metal, AVAudioEngine, GCController, and Swift, along with several follow-up optimizations and fixes:
1. Cache the previous screen state to avoid redundant rendering commands when the screen state is unchanged.
2. Refactor viewport handling in MetalRenderer and enhance text input filtering in TICMetalView (excluding macOS function/arrow keys).
3. Fix CMake configuration errors on macOS by enabling Objective-C compilation support at the root level and removing redundant declarations in apple.cmake.
@joshgoebel

Copy link
Copy Markdown
Collaborator

Can you elaborate a bit more on what this supports... does it support zoom factors, background border, etc? Haven't had a change to compile it yet for a look-see.

What is the big advantage over SDL (on Mac OS)? Are you hoping to revive the iOS port?

Comment thread src/studio/system.h
rightChannel?[writeIdx + i] = sampleR
}
} else {
for i in 0..<samplesToCopy {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think we could use memset here, yes?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mentioned in the above comment that this is not the main bottleneck

Comment on lines +51 to +57
for i in 0..<samplesToCopy {
let sampleL = Float(samplesBuffer[(self.bufferReadIdx + i) * 2]) / 32768.0
let sampleR = Float(samplesBuffer[(self.bufferReadIdx + i) * 2 + 1]) / 32768.0

leftChannel[writeIdx + i] = sampleL
rightChannel?[writeIdx + i] = sampleR
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI suggestion:

I’d also consider replacing the per-sample copy loop (lines 51–57) with an interleaved→planar bulk conversion (e.g., vDSP) later,

Do you understand the suggestion? I did imagine there would be a more optimize way to do this transfer, and sounds like there is.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am specialized in audio programming (see for instance my previous project https://github.com/Wang-Yue/cdsp). vDSP is only tremendously better when doing bulk conversion for thousands of samples. In this paticular case if we pull every frame, we only get 44100/60 = 735 samples. vDSP performs badly when you have small chunk size, as there are cost calling vDSP functions --- they are not inline functions. Also the code here and below is not the major concern unless we

  1. solve the main bottleneck, which is the software renderer.
  2. add api to start/stop audio, as currently we are still draining battery when there's nothing playing.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. I'll take your word for it. :-)

as currently we are still draining battery when there's nothing playing.

Sounds like something worth investigating one day in a future PR maybe. (after other things as you mention)

Comment thread src/system/apple/AudioEngine.swift
Comment thread src/system/apple/GamepadManager.swift
Comment thread src/system/apple/MetalRenderer.swift
Comment thread README.md
Comment thread README.md
Comment thread src/system/apple/main.swift
Comment thread src/system/apple/main.swift
}

func windowWillClose(_ notification: Notification) {
NSApp.terminate(nil)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a final notice or something we could interact with? Remember sometimes we might have editor changes we need to save.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. SDL backend also don't check this. I just tried, no final notice or anything.
I also don't think TIC-80 expose functions that let us check if editor have changes.

@joshgoebel

Copy link
Copy Markdown
Collaborator

I did my best to review this a bit. Looks pretty awesome though overall (the functionality - getting this back in core)...

@joshgoebel

Copy link
Copy Markdown
Collaborator

how does loading/saving cartridges on IOS work? Do you use Files top drop source/carts into a folder TIC-80 can access or?

@Wang-Yue

Copy link
Copy Markdown
Contributor Author

I did my best to review this a bit. Looks pretty awesome though overall (the functionality - getting this back in core)...

Thank you for the detailed review!

how does loading/saving cartridges on IOS work? Do you use Files top drop source/carts into a folder TIC-80 can access or?

as mentioned above, there are still many functionalities I have not implemented/tested, and while I was playing on iPads myself, I already found a few bugs. mac app feels far more ready than iOS, btw. so I am constantly fixing them in my main branch. Once the PR is accepted I will send those incremental fixes to here. I also believe when the users test this on their apple device they will notice bugs I didn't notice, as they may have a different usage pattern compared to mine.

So I can address bugs/missing features in later PRs for incremental fixes whenever I have time. Or other interested party may contribute their PRs. It's impossible for me to get to close to 100% ready in the first PR. The good thing is we now have the full app running on iPhone and iPad, and playing games work end to end. so iterative improvements become possible now.

@Wang-Yue

Wang-Yue commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Can you elaborate a bit more on what this supports... does it support zoom factors, background border, etc? Haven't had a change to compile it yet for a look-see.

zoom --- yes.
background border --- yes.

On a mac I think we support almost everything SDL supports. iOS is less ready than Mac. iOS window cannot be set with fixed aspect ratio, so I do have commits to draw the background border as well (Wang-Yue@c7027c2) when it's rendered in a window. I can send those incremental improvements later when this PR is merged.

What is the big advantage over SDL (on Mac OS)?

Higher performance for both audio and graphics. Cleaner callbacks.

Here's TIC-80 running the supernova game (60fps 3D car race) profile:

SDL:
Screenshot 2026-07-28 at 12 02 28

Metal:
Screenshot 2026-07-28 at 12 03 18

You can see in the SDL case only 70% of the run time is leaving for TIC-80's studio_tick (user program and software rendering), while for the metal case, it's > 85%. If you do the math, you'll find Metal + AVAudioEngine is half of the overhead compared to SDL (30/15*85/70 = 2.4x faster).

For close to static screens, things are even more extraordinary. This is TIC-80's music editor when music is not playing:

Screenshot 2026-07-28 at 12 20 48

You see SDL is very busy and takes 55% of the run time

Screenshot 2026-07-28 at 12 26 57

with the help of the draw cache, our new backend spend most time doing _platform_memcmp, then find all of them hit the cache, and all the rendering functions become no-op. CPU is almost idle in this case. This is super crucial for mobile and low powered devices.

I also want to add that the biggest _platform_memcmp on the right (around 23% of the running time) can be eliminated once TIC-80 decide to accept my draw cache patch. This PR is written with even draw cache not being accepted in mind, so we still compare final buffer every single frame when shipping rendered buffer to GPU. With Draw Cache PR accepted, we will know without memcmp if the buffer is dirty or not, and fully eliminate the 23% memcmp time, saving even more CPU cycles.

Are you hoping to revive the iOS port?

Yes. Before working on this port I spent a long time to fix the SDL port for iOS, but even after a lot of work, it's still almost not usable --- Existing user interface is very buggy, and it is running very slow. At this stage, I believe starting from scratch is much easier than fixing that port. So I did it.

@joshgoebel

Copy link
Copy Markdown
Collaborator

@nesbox I'm not sure how you want to merge/release this, but this looks pretty awesome to me. Obviously would be room for you to make an official iOS release in the app store in the future as well if you were so interested... this looks like a pretty solid foundation for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants