Bug 569 - Multisampling doesn't work with PBuffer on OSX Lion and AMD CPU
Summary: Multisampling doesn't work with PBuffer on OSX Lion and AMD CPU
Status: RESOLVED FIXED
Alias: None
Product: Jogl
Classification: JogAmp
Component: macosx (show other bugs)
Version: 2
Hardware: pc_x86_64 macosx
: --- normal
Assignee: Sven Gothel
URL:
Depends on: 599
Blocks: 617
  Show dependency treegraph
 
Reported: 2012-03-23 23:08 CET by ac
Modified: 2014-04-15 22:05 CEST (History)
4 users (show)

See Also:
Type: ---
SCM Refs:
Workaround: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description ac 2012-03-23 23:08:07 CET
Multisampled surfaces cannot be created on OSX Lion even if the hardware supports more than 1 sample buffers.

Tested with MacOSX 10.7.3 on a MacBook Pro, AMD Radeon HD 6490M vcard, drivers up to date. Latest jogl built from trunk.

Querying the maximum number of samples with:

glGetIntegerv(GL_MAX_SAMPLES...); 

returns 4. However, if the surface is created for example with 2 buffers:

profile = GLProfile.getDefault(); 
capabilities = new GLCapabilities(profile); 
capabilities.setSampleBuffers(true); 
capabilities.setNumSamples(2); 
    
canvas = new GLCanvas(capabilities); 
canvas.setBounds(0, 0, width, height); 

simple visual inspection indicates that multisampling is not working, furthermore querying the number of samples after the surface has been created with:

glGetIntegerv(GL.GL_SAMPLES...); 

returns zero.

The following links point to full applications showing the issue:

http://pastebin.com/fYezrMA1  (AWT) 
http://pastebin.com/3HSFKt95 (NEWT)
Comment 1 ac 2012-03-27 22:23:21 CEST
This issue doesn't affect the same machine when switching to the integrated Intel HD 3000 graphics, so I guess is due to some problem with the OpenGL drivers for AMD under Lion.
Comment 2 ac 2012-04-22 23:03:31 CEST
Hello Sven, I was planning to start looking at the code to try to came up with a solution/patch for this bug. But maybe you are already working on it. Let me know.
Comment 3 Sven Gothel 2012-05-01 09:23:26 CEST
AC, pls do so .. I don't have a machine to reproduce the issue. Thank you.
Comment 4 ac 2012-05-27 21:58:34 CEST
hi, I finally got some time to work on this. I will let you know how it goes.
Comment 5 ac 2012-05-28 04:36:11 CEST
I have not been able to determine a possible cause for this issue yet. All I was able to do so far is to check that the requested sampling level is passed correctly to the native function initWithFormat in MacOSXWindowsSystemInterface.m, but when is queried right afterwards, it turns to be zero. Consequently, anti-aliasing is disabled. 

If I run my test program with full debug output enabled, this is what I get for the AMD Radeon HD 6490M:

NS create OSX>=lion true
NS create backendType: NSOPENGL
NS create backingLayerHost: JAWT-Window[windowHandle 0x7fde82c82320, surfaceHandle 0x7fde82c7f800, bounds [ 0 / 0  300 x 300 ], insets [ l 0, r 0 - t 0, b 0 - 0x0], shallUseOffscreenLayer false, isOffscreenLayerSurface true, pos 0/0, size 300x300, visible true, lockedExt false,
	config AWTGraphicsConfiguration[AWTGraphicsScreen[AWTGraphicsDevice[type AWT, connection \Display0, unitID 0, awtDevice CGraphicsDevice[screen=0], handle 0x0], idx 0],
	chosen    GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pixmap],
	requested GLCaps[on-scr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw]],
	apple.awt.CGraphicsConfig@4a4e79f1[dev=CGraphicsDevice[screen=0],pixfmt=0],
	encapsulated MacOSXCGLGraphicsConfiguration[DefaultGraphicsScreen[MacOSXGraphicsDevice[type MacOSX, connection decon, unitID 0, handle 0x0], idx 0],
	chosen    GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pbuffer [r2t 0, r2tr 0, float 0]],
	requested GLCaps[on-scr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw]]]],
	awtComponent AWT-GLCanvas[Realized true,
	jogamp.opengl.macosx.cgl.MacOSXPbufferCGLDrawable,
	Factory   jogamp.opengl.macosx.cgl.awt.MacOSXAWTCGLDrawableFactory@36f0b7f8,
	handle    0x7fde82c7f800,
	Drawable size 300x300,
	AWT pos 0/0, size 300x300,
	visible true,
	AWTGraphicsConfiguration[AWTGraphicsScreen[AWTGraphicsDevice[type AWT, connection \Display0, unitID 0, awtDevice CGraphicsDevice[screen=0], handle 0x0], idx 0],
	chosen    GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pixmap],
	requested GLCaps[on-scr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw]],
	apple.awt.CGraphicsConfig@4a4e79f1[dev=CGraphicsDevice[screen=0],pixfmt=0],
	encapsulated MacOSXCGLGraphicsConfiguration[DefaultGraphicsScreen[MacOSXGraphicsDevice[type MacOSX, connection decon, unitID 0, handle 0x0], idx 0],
	chosen    GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pbuffer [r2t 0, r2tr 0, float 0]],
	requested GLCaps[on-scr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw]]]]],
	surfaceLock <39ed1b0b, 7d8a8ce2>[count 1, qsz 0, owner <AWT-EventQueue-0>]]
NS create share: 0
NS create chosenCaps: GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pbuffer [r2t 0, r2tr 0, float 0]]
NS create pixelFormat: 0x7fde82c83a00
NS create drawable native-handle: 0x7fde82c7f800
NS create drawable NSView-handle: 0x0
NS create screen refresh-rate: 0 hz, Infinity micros
NS create pixelformat2GLCaps: GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/0, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pbuffer [r2t 0, r2tr 0, float 0]]
NS create fixedCaps: GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/0, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pbuffer [r2t 0, r2tr 0, float 0]]
NS create nsOpenGLLayer 0x7fde87369ef0, texSize 300x300, MacOSXPbufferCGLDrawable[Realized true,
	Factory   jogamp.opengl.macosx.cgl.awt.MacOSXAWTCGLDrawableFactory@36f0b7f8,
	handle    0x7fde82c7f800,
	Window    JAWT-Window[windowHandle 0x7fde82c82320, surfaceHandle 0x7fde82c7f800, bounds [ 0 / 0  300 x 300 ], insets [ l 0, r 0 - t 0, b 0 - 0x0], shallUseOffscreenLayer false, isOffscreenLayerSurface true, pos 0/0, size 300x300, visible true, lockedExt false,
	config AWTGraphicsConfiguration[AWTGraphicsScreen[AWTGraphicsDevice[type AWT, connection \Display0, unitID 0, awtDevice CGraphicsDevice[screen=0], handle 0x0], idx 0],
	chosen    GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pixmap],
	requested GLCaps[on-scr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw]],
	apple.awt.CGraphicsConfig@4a4e79f1[dev=CGraphicsDevice[screen=0],pixfmt=0],
	encapsulated MacOSXCGLGraphicsConfiguration[DefaultGraphicsScreen[MacOSXGraphicsDevice[type MacOSX, connection decon, unitID 0, handle 0x0], idx 0],
	chosen    GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/0, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pbuffer [r2t 0, r2tr 0, float 0]],
	requested GLCaps[on-scr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw]]]],
	awtComponent AWT-GLCanvas[Realized true,
	jogamp.opengl.macosx.cgl.MacOSXPbufferCGLDrawable,
	Factory   jogamp.opengl.macosx.cgl.awt.MacOSXAWTCGLDrawableFactory@36f0b7f8,
	handle    0x7fde82c7f800,
	Drawable size 300x300,
	AWT pos 0/0, size 300x300,
	visible true,
	AWTGraphicsConfiguration[AWTGraphicsScreen[AWTGraphicsDevice[type AWT, connection \Display0, unitID 0, awtDevice CGraphicsDevice[screen=0], handle 0x0], idx 0],
	chosen    GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pixmap],
	requested GLCaps[on-scr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw]],
	apple.awt.CGraphicsConfig@4a4e79f1[dev=CGraphicsDevice[screen=0],pixfmt=0],
	encapsulated MacOSXCGLGraphicsConfiguration[DefaultGraphicsScreen[MacOSXGraphicsDevice[type MacOSX, connection decon, unitID 0, handle 0x0], idx 0],
	chosen    GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/0, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pbuffer [r2t 0, r2tr 0, float 0]],
	requested GLCaps[on-scr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw]]]]],
	surfaceLock <39ed1b0b, 7d8a8ce2>[count 1, qsz 0, owner <AWT-EventQueue-0>]]]
NS setSwapInterval: 2147483647 micros

As you can see, the chosenCaps include "24/8/4, sample-ext default", but fixedCaps after creation of the context just says "24/8/0, dbl".

The debug output for the same program, but using the Intel HD Graphics 3000, is:

NS create OSX>=lion true
NS create backendType: NSOPENGL
NS create backingLayerHost: JAWT-Window[windowHandle 0x7fc332809030, surfaceHandle 0x7fc331c4b4c0, bounds [ 0 / 0  300 x 300 ], insets [ l 0, r 0 - t 0, b 0 - 0x0], shallUseOffscreenLayer false, isOffscreenLayerSurface true, pos 0/0, size 300x300, visible true, lockedExt false,
	config AWTGraphicsConfiguration[AWTGraphicsScreen[AWTGraphicsDevice[type AWT, connection \Display0, unitID 0, awtDevice CGraphicsDevice[screen=0], handle 0x0], idx 0],
	chosen    GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pixmap],
	requested GLCaps[on-scr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw]],
	apple.awt.CGraphicsConfig@348bdcd2[dev=CGraphicsDevice[screen=0],pixfmt=0],
	encapsulated MacOSXCGLGraphicsConfiguration[DefaultGraphicsScreen[MacOSXGraphicsDevice[type MacOSX, connection decon, unitID 0, handle 0x0], idx 0],
	chosen    GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pbuffer [r2t 0, r2tr 0, float 0]],
	requested GLCaps[on-scr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw]]]],
	awtComponent AWT-GLCanvas[Realized true,
	jogamp.opengl.macosx.cgl.MacOSXPbufferCGLDrawable,
	Factory   jogamp.opengl.macosx.cgl.awt.MacOSXAWTCGLDrawableFactory@421fbfd6,
	handle    0x7fc331c4b4c0,
	Drawable size 300x300,
	AWT pos 0/0, size 300x300,
	visible true,
	AWTGraphicsConfiguration[AWTGraphicsScreen[AWTGraphicsDevice[type AWT, connection \Display0, unitID 0, awtDevice CGraphicsDevice[screen=0], handle 0x0], idx 0],
	chosen    GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pixmap],
	requested GLCaps[on-scr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw]],
	apple.awt.CGraphicsConfig@348bdcd2[dev=CGraphicsDevice[screen=0],pixfmt=0],
	encapsulated MacOSXCGLGraphicsConfiguration[DefaultGraphicsScreen[MacOSXGraphicsDevice[type MacOSX, connection decon, unitID 0, handle 0x0], idx 0],
	chosen    GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pbuffer [r2t 0, r2tr 0, float 0]],
	requested GLCaps[on-scr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw]]]]],
	surfaceLock <132789d2, 8c943d1>[count 1, qsz 0, owner <AWT-EventQueue-0>]]
NS create share: 0
NS create chosenCaps: GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pbuffer [r2t 0, r2tr 0, float 0]]
NS create pixelFormat: 0x7fc331c49b30
NS create drawable native-handle: 0x7fc331c4b4c0
NS create drawable NSView-handle: 0x0
NS create screen refresh-rate: 0 hz, Infinity micros
NS create pixelformat2GLCaps: GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pbuffer [r2t 0, r2tr 0, float 0]]
NS create fixedCaps: GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pbuffer [r2t 0, r2tr 0, float 0]]
NS create nsOpenGLLayer 0x7fc331c596d0, texSize 300x300, MacOSXPbufferCGLDrawable[Realized true,
	Factory   jogamp.opengl.macosx.cgl.awt.MacOSXAWTCGLDrawableFactory@421fbfd6,
	handle    0x7fc331c4b4c0,
	Window    JAWT-Window[windowHandle 0x7fc332809030, surfaceHandle 0x7fc331c4b4c0, bounds [ 0 / 0  300 x 300 ], insets [ l 0, r 0 - t 0, b 0 - 0x0], shallUseOffscreenLayer false, isOffscreenLayerSurface true, pos 0/0, size 300x300, visible true, lockedExt false,
	config AWTGraphicsConfiguration[AWTGraphicsScreen[AWTGraphicsDevice[type AWT, connection \Display0, unitID 0, awtDevice CGraphicsDevice[screen=0], handle 0x0], idx 0],
	chosen    GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pixmap],
	requested GLCaps[on-scr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw]],
	apple.awt.CGraphicsConfig@348bdcd2[dev=CGraphicsDevice[screen=0],pixfmt=0],
	encapsulated MacOSXCGLGraphicsConfiguration[DefaultGraphicsScreen[MacOSXGraphicsDevice[type MacOSX, connection decon, unitID 0, handle 0x0], idx 0],
	chosen    GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pbuffer [r2t 0, r2tr 0, float 0]],
	requested GLCaps[on-scr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw]]]],
	awtComponent AWT-GLCanvas[Realized true,
	jogamp.opengl.macosx.cgl.MacOSXPbufferCGLDrawable,
	Factory   jogamp.opengl.macosx.cgl.awt.MacOSXAWTCGLDrawableFactory@421fbfd6,
	handle    0x7fc331c4b4c0,
	Drawable size 300x300,
	AWT pos 0/0, size 300x300,
	visible true,
	AWTGraphicsConfiguration[AWTGraphicsScreen[AWTGraphicsDevice[type AWT, connection \Display0, unitID 0, awtDevice CGraphicsDevice[screen=0], handle 0x0], idx 0],
	chosen    GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pixmap],
	requested GLCaps[on-scr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw]],
	apple.awt.CGraphicsConfig@348bdcd2[dev=CGraphicsDevice[screen=0],pixfmt=0],
	encapsulated MacOSXCGLGraphicsConfiguration[DefaultGraphicsScreen[MacOSXGraphicsDevice[type MacOSX, connection decon, unitID 0, handle 0x0], idx 0],
	chosen    GLCaps[offscr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw], pbuffer [r2t 0, r2tr 0, float 0]],
	requested GLCaps[on-scr, rgba 0x8/8/8/8, opaque, accum-rgba 0/0/0/0, dp/st/ms: 24/8/4, sample-ext default, dbl, mono  , hw, GLProfile[GL2/GL2.hw]]]]],
	surfaceLock <132789d2, 8c943d1>[count 1, qsz 0, owner <AWT-EventQueue-0>]]]
NS setSwapInterval: 2147483647 micros

where both the chosenCaps and fixedCaps say "24/8/4, sample-ext default", indicating that 4x multisampling has been correctly initialized.

I don't think this is a driver problem, since I tested an equivalent code with 4X antialiasing on the Radeon using LWJGL, and the multisampled surface was created ok.

Do you have any leads as to where to look next?
Comment 6 ac 2012-05-29 04:32:52 CEST
After some more tinkering, I found a few more pieces of info about this issue (but still not a solution). If the GLCanvas is added to the Frame directly, multisampling on the Radeon card is enabled. The problem appears when the GLCanvas is added to an Applet, which in turn is added to the Frame. So I have the following cases:

1) Intel HD300, Frame <- GLCanvas, multisampling OK
2) Intel HD300, Frame <- Applet <- GLCanvas, multisampling OK
3) Radeon 6490, Frame <- GLCanvas, multisampling OK
4) Radeon 6490, Frame <- Applet <- GLCanvas, NO multisampling 

This could to be a nasty combination of driver-specific quirks that are maybe incompatible with some initialization methods in JOGL when a component containing the native surface is an instance of Applet.

I looked for explicit references to the Applet class in the JOGL code, and I found some, precisely in MacOSXJAWTWindow and JAWTWindow, where there is a isApplet() method apparently only used to ensure the use an offscreen layer when retrieving the JAWT object.
Comment 7 ac 2012-05-29 15:02:06 CEST
Another bit of debugging info: if I force to use onscreen layer (by setting tryOffscreenLayer = false, in JAWUtil.getJAWT()), the surface does have multisampling when embedding the GLCanvas in the Applet object, in the case of the discrete Radeon card. But nothing is displayed on the window (although the rendering loop seems to be going on). So the incompatibility seems to be more specifically located in the offscreen layer mode. I will look further look at it. Of course, if you Sven have any suggestion or comments, they will be greatly appreciated, as we are trying to get the new release of Processing out fairly soon and this issue is a stopper on Mac.
Comment 8 ac 2012-05-29 17:53:38 CEST
ok, I might be getting closer...

I run a detailed debug session on the LWJGL version of my test app, and discovered that it throws an exception when trying to create a pixel format with pbuffer support (the exception happens only when switching to the discrete Radeon card, with the integrated Intel there is no exception), but catches it and defaults back to a pixel format with same attributes with the exception of pbuffer support.

On the JOGL side, I noticed that the PixelFormat2GLCapabilities method enables/disables pbuffer support depending on the corresponding value stored in ivalues. However, if I call capabilities.setPBuffer(false) from my main program, the pbuffer option still gets set to true...

So I just forced GLCapabities.setPBuffer() to set pbuffer to false regardless of the argument, and voila!, now the surface is created with multisampling support. However, the main window shows nothing again...
Comment 9 ac 2012-05-29 18:40:21 CEST
... I just noticed that in the implementation of createGLDrawable() in GLDrawableFactoryImpl you override the onscreen caps settings, so that an offscreen layer surface is always attempted to be created first. 

I commented out the first if branch so either an onscreen or offscreen surface is created depending on the value returned by chosenCaps.isOnscreen(). What happens now is the following:

1) I disable pbuffers and enable onscreen from the main app:
capabilities.setPBuffer(false);
capabilities.setOnscreen(true);

then I get a multisampled surface, but no image on the window.

2)  I enable pbuffers and disable onscreen from the main app:
capabilities.setPBuffer(true);
capabilities.setOnscreen(false);

then I get normal rendering output, but without multisampling.

The fact that you override the onscreen/pbuffer caps settings from the user makes me think that offscreen PBuffers is currently the only working code path on OSX (please, correct me if I am wrong). However, it seems that at least for discrete AMD cards on OSX Lion, PBuffers are not compatible with multisampling, even if the hardware supports multisampling. 

And then I just found this thread on the mozilla bugzilla:

https://bugzilla.mozilla.org/show_bug.cgi?id=688844

where people say that PBuffers are deprecated in 10.7. This turns to be officially confirmed by Apple here:

http://developer.apple.com/library/mac/#releasenotes/Cocoa/AppKit.html

(look for the "Deprecation of Pixel Buffers (PBuffers)" section). This probably explains why PBuffers don't seem to work well on Lion, and the only realistic solution would be to replace PBuffers by FBOs. How difficult do you think this would be?
Comment 10 Sven Gothel 2012-06-03 23:53:44 CEST
Hi AC,

sorry for being offline for such a long time, kudos to your triage.

Indeed, it looks like that AMD's pbuffer offscreen surface doesn't support MSAA.

Note that both "OSX >= 10.6.8 Browser Applets" and OpenJDK7
only support the offscreen JAWT CALayer integration on OSX.
Hence you don't see anything w/ forced onscreen.

How expensive will it be to replace PBuffer w/ FBO ?
Well, it should be somehow straightforward, however we may need to figure out
how to inject the FBO setup and it's selection transparently so the user won't notice.
But in case the user already tingles w/ FBO in her code .. I am afraid this won't be possible.
Note: The GL FBO setup commands would disable our offscreen FBO,
           if the user likes to switch back to 'normal' rendering - if using FBO herself.

PBuffer allows us to transparently use offscreen rendering
while the user does not need to care about.

Maybe we can think about a 'transparent' FBO usage w/o sideeffect .. and find a solution together?

~Sven
Comment 11 ac 2012-06-04 15:08:32 CEST
Since we needed to get a new version of Processing out by the end of last week with full anti-aliasing support in OSX 10.7+, I wrote a hack directly in Java that uses a muti-sampled FBO to take care of the issue, at least temporarily.

I imagine that the native code in JOGL that could replace the PBuffer implementation with FBOs would be analogous to the one I wrote in Processing. The main problem I see happening, as you say, is when the user binds her own FBO, and then restores the "screen buffer" with gl.glBindFramebuffer(GL.GL_FRAMEBUFFER, 0), which would break offscreen rendering since the underlying multi-sampled FBO should be bound back instead. In Processing this is not an issue, since I have an FBO stack that takes care of this problem. 

As a side note, I did some debugging using a LWJGL-version of the code, for comparison purposes. I didn't do an exhaustive analysis, but I got the impression that LWJGL handles the anti-aliasing issue somehow. I noticed that when using the AMD card on OSX 10.7, LWJGL first does a native call to NSOpenGLPixelFormat-initWithAttributes with the NSOpenGLPFAPixelBuffer attribute, however this call results in an exception. It then it creates a pixel format without pbuffer support, which works. The rendering surface does have anti-aliasing though, but I don't know the rest of the implementation details.
Comment 12 Sven Gothel 2012-06-17 17:15:37 CEST
(In reply to comment #11)
> Since we needed to get a new version of Processing out by the end of last week
> with full anti-aliasing support in OSX 10.7+, I wrote a hack directly in Java
> that uses a muti-sampled FBO to take care of the issue, at least temporarily.

Would be nice if you can point me to your code
in case it contains a few specialties - so I can benefit from it.
Thx in advance.

> 
> I imagine that the native code in JOGL that could replace the PBuffer
> implementation with FBOs would be analogous to the one I wrote in Processing.
> The main problem I see happening, as you say, is when the user binds her own
> FBO, and then restores the "screen buffer" with
> gl.glBindFramebuffer(GL.GL_FRAMEBUFFER, 0), which would break offscreen
> rendering since the underlying multi-sampled FBO should be bound back instead.
> In Processing this is not an issue, since I have an FBO stack that takes care
> of this problem. 

Well, we could override gl.glBindFramebuffer(..) in our GL* implementation 
and hardwire it to the offscreen framebuffer.
Maybe that would be transparent for all and allows getting rid of pbuffer
even for other platforms, if we desire.

> 
> As a side note, I did some debugging using a LWJGL-version of the code, for
> comparison purposes. I didn't do an exhaustive analysis, but I got the
> impression that LWJGL handles the anti-aliasing issue somehow. I noticed that
> when using the AMD card on OSX 10.7, LWJGL first does a native call to
> NSOpenGLPixelFormat-initWithAttributes with the NSOpenGLPFAPixelBuffer
> attribute, however this call results in an exception. It then it creates a
> pixel format without pbuffer support, which works. The rendering surface does
> have anti-aliasing though, but I don't know the rest of the implementation
> details.

Using an 'onscreen' pixelformat for the pbuffer one looks like 
dealing w/ a bug in the [driver] implementation.
I prefer to refrain from such hacks and highly favor the FBO solution ofc.
Comment 13 ac 2012-06-19 19:45:28 CEST
this is the file in Processing that contains the FBO hack to deal with the lack of antialiasing on OSX 10.7

http://code.google.com/p/processing/source/browse/trunk/processing/java/libraries/opengl/src/processing/opengl/PGL.java

In particular, take a look at the following methods;

1) Called once during surface initialization:
initPrimarySurface()  

2) Called at the start of each frame:
updatePrimary() 
beginOnscreenDraw() 

3) Called at the end of each frame:
endOnscreenDraw()
Comment 14 Sven Gothel 2012-06-30 14:57:39 CEST
See Bug 599
Comment 15 Sven Gothel 2012-07-11 17:45:34 CEST
FBO: FBODrawable basically working on X11, enhancing FBObject allowing the multisample and blit/readPixel thing. Then need to hook it up w/ the offscreen-layered surface.
Comment 16 offbeata 2012-07-31 16:08:58 CEST
A solution to this problem would be greatly appreciated. Thank you very much for working on it!
Comment 17 Sven Gothel 2012-08-02 09:28:01 CEST
(In reply to comment #16)
> A solution to this problem would be greatly appreciated. Thank you very much
> for working on it!

See Bug 599, will be completed after SIGGRAPH.
Comment 18 Sven Gothel 2012-09-15 17:06:54 CEST
see Bug 599 for details ..
Comment 19 Sven Gothel 2012-09-19 07:31:25 CEST
For OSX stabilization see Bug 617.
Comment 20 Wade Colson 2014-04-15 22:05:14 CEST
*** Bug 260998 has been marked as a duplicate of this bug. ***
Seen from the domain http://volichat.com
Page where seen: http://volichat.com/singles-chat-rooms
Marked for reference. Resolved as fixed @bugzilla.