Metal is coming. Cupertino has decided to bring more joy and happiness to the world by deprecating the widely used open standard OpenGL, thus making everyone's code obsolete. There are ominous pronouncements that OpenGL is due to die, most likely in the next major MacOS version after Catalina, to be replaced by their own proprietary standard Metal. As all the graphics in mgl is written with OpenGL using Cocoa frameworks (a variant of C called Objective-C), we will need to rewrite the backend of mgl to make it compatible.
The good news is that since mgl is written using simple, atomic functions with future OS API compatibility in mind, this transition is not such a monumental task. In fact, as of January 2020, I (jlg) have an alpha version of the mgl code running that is able to do the most important basic functions such as clearing the screen (mglClearScreen), drawing points (mglPoints), drawing lines (mglFixationCross, mglLines), drawing quads (mglQuads) and textures (mglCreateTexture, mglBltTexture).
As this will be the third major rewrite of the mgl code, it will be version 3.0 and will be written primarily in Swift. The first version was written using the 32-bit Carbon API and the second version was written using the 64-bit Cocoa Frameworks.
There will be advantages to the new 3.0 metal compliant version of mgl.
The alpha mgl 3.0 can be retrieved using git by cloning and then switching to the metal branch
git clone https://github.com/justingardner/mgl.git cd mgl git checkout metal
If this worked correctly, then you should see a new directory called metal within the mgl repository:
grumini:/Users/justin/mgl> ls COPYING metal/ readme.md task/ Contents.m mgllib/ readme.txt utils/
I've tested this on the latest version of Catalina, not sure if it will work on Mojave or older OS. You probably will need a current version of XCode because Cupertino wouldn't allow compilation (at least for my setup) w/out updating everything.
First go to matlab and start up mglMetalTest. This will run mglSocketOpen to setup a socket server, which will wait for the mglMetal application to make a connection. Don't hit enter yet, as you need to start the mglMetal application.
>> mglMetalTest (mglSocketOpen) Opened socket testsocket with socketDescriptor: 298 Hit ENTER to ping:
From XCode, open up the mglMetal application which is in mgl/mglMetal/mglMetal.xcodeproj
Then click on the sideways triangle, play button, in XCode at the top left to build and start up the mglMetal application. Xcode should look like the following, with the console showing that a socket connection has been opened and that mglRenderer is processing OS events:
You should also see an mglMetal window that has opened. This is where stimuli will be displayed. Note that it may show up as a full screen window somewhere offscreen (still working on that!). If it does you won't see a window, but if you swipe right with three fingers you will get to a gray screen (or if you do command-tab, you will see the mgl icon as one of the running applications).
Now, go back to Matlab and hit enter. This should ping the mglMetal application and return something like this:
Hit ENTER to ping: (mglSocketWrite) Waiting for a new connection (mglSocketWrite) New connection made: 301 (mglSocketWrite) Using connectionDescriptor 301 (mglSocketWrite) Wrote 2 of 2 bytes (Len: 1 (1 x 1), dataSize: 2) (mglSocketWrite) Using connectionDescriptor 301 (mglSocketWrite) Wrote 2 of 2 bytes (Len: 1 (1 x 1), dataSize: 2) (mglSocketWrite) Using connectionDescriptor 301 (mglSocketWrite) Wrote 2 of 2 bytes (Len: 1 (1 x 1), dataSize: 2) (mglSocketDataWaiting) Using connectionDescriptor 301
That means that the connection is working. If that is not what happens, you might want to stop the mglMetal application and hit ctrl-c on the Matlab side and start again (i.e. first run mglMetalTest on Matlab, then rebuild/run the mglMetal application in XCode and hit enter to ping).
If all is well and you got a successful ping, then the communication is setup and you should be in business. You will get a prompt in Matlab like this:
Hit ENTER to clear screen:
After hitting enter the mglMetal display should turn red:
On Matlab then you get the prompt:
Hit ENTER to test lines:
And the screen should look like this after you hit ENTER:
Then in Matlab:
Hit ENTER to test quads:
And a display like this:
It should print out some profiling information, about how long it takes to send and respond to the mglQuad command (first one below) and how long it takes to display. The first one should be way under a ms as it just the communication delay, and the second one should be under one frame refesh (16.7ms for 60Hz).
(mglMetalTest:mglProfile) Profile time for mglProfileOn is: 0.080889 ms (mglMetalTest:mglProfile) Profile time for mglQuad is: 0.694815 ms
You can now make it flicker
Hit ENTER to flicker:
Which should just make the checkerboard flicker.
Next we test coordinate xforms (which will be used for setting display coordinates to visual angles in the future). You should get two prompts which will first display three dots offset to the left, and then in the center:
Hit ENTER to test coordinate xform: Hit ENTER to test coordinate xform:
Now, for some (slightly) more fun stuff. Next should display a vertical gabor. Note the profile time tells you how long mglCreate and mglBlt texture take.
Hit ENTER to test texture: (mglMetalTest:mglProfile) Profile time for mglProfileOn is: 0.117438 ms (mglSocketWrite) Using connectionDescriptor 302 (mglSocketWrite) Wrote 1048576 of 1048576 bytes (Len: 262144 (1 x 262144), dataSize: 4) (mglMetalTest:mglProfile) Profile time for mglCreateTexture is: 3.228833 ms (mglMetalTest:mglProfile) Profile time for mglBltTexture is: 0.453883 ms
Next it should go full screen, show you the grating drifting to the left and then go back to a windowed context:
Hit ENTER to test blt (drifting): (mglMetalTest) Median frame time: 0.0167 (mglMetalTest) Max frame time: 0.0221 (mglMetalTest) Number of frames over 0.0167: 164/271 (mglMetalTest) Number of frames 5% over 0.0167: 17/271 (mglMetalTest) Number of frames 10% over 0.0167: 1/271 (mglMetalTest) Number of frames 20% over 0.0167: 1/271
Next it will do something similar, but show you drifting gratings at different locations and orientations.
Hit ENTER to test blt (multiple rotating and drifting): (mglMetalTest) Median frame time: 0.0167 (mglMetalTest) Max frame time: 0.0252 (mglMetalTest) Number of frames over 0.0167: 163/271 (mglMetalTest) Number of frames 5% over 0.0167: 51/271 (mglMetalTest) Number of frames 10% over 0.0167: 1/271 (mglMetalTest) Number of frames 20% over 0.0167: 1/271
Next, a test of dots
Hit ENTER to test dots: (mglMetalTest:mglProfile) Profile time for mglProfileOn is: 0.074804 ms (mglMetalTest:mglProfile) Profile time for mglPoints2 is: 0.422125 ms
Then a test of moving dots, this will also go full screen and then should go back to a windowed context.
Hit ENTER to test dots: (mglMetalTest) Median frame time: 0.0167 (mglMetalTest) Max frame time: 0.0924 (mglMetalTest) Number of frames over 0.0167: 554/971 (mglMetalTest) Number of frames 5% over 0.0167: 137/971 (mglMetalTest) Number of frames 10% over 0.0167: 24/971 (mglMetalTest) Number of frames 20% over 0.0167: 5/971
Note that mglMetalTest with no arguments, starts up a new socket which the mglMetal application would have to reconnect with, so if you want to test the sequence again, without restarting mglMetal, you can run it like this:
The following table is as of January 2020. The framework for all the key functionality that needs to be ported over to Metal is in place.
|Function name||Implementation status||Notes|
|mglOpen||Working alpha||Code for this is in mglMetalTest, not yet pulled out into a clean function with all the same functionality as the mglOpen function, but should be trivial.|
|mglFlush||Working alpha||Works by busy-waiting on the flush within mglMetal - seems like this could be rethought to free up time on the matlab side|
|mglClose||Not yet implemented||Trivial. But, need to decide if there should be a close state in which mglMetal is minimized|
|mglVisualAngleCoordinates||Not yet implemented||Trivial. Already have the shader vertex code setup to accept a transformation matrix from matlab|
|mglSetGammaTable||No need to update|
|mglGetGammaTable||No need to update|
|Keyboard / mouse functions||No need to update|
|National Instruments digital / analog I/O||NO need to update|
|Task code||No need to update|