Question about setting alpha of a mesh uniformly: an open problem for you to solve

keszegh · June 2017

So i'm trying to fade off a mesh but setting it's alpha sets the alpha of each constituting triangle separately, thus overlapping parts fade differently from non-overlapping parts. this is not good for me, i'd prefer uniform fade off.
see attached figure for full alpha, and faded off alpha, the intersection of the two strokes is more opaque, that's what i don't want.
is there any solution? rendering to rendertarget and then setting alpha for this would work but i'm working with thousands of meshes, and so this would slow down the app too much. i was hoping that maybe you can think of some other solution, using shaders perhaps?

thanks for your ideas.

Yan · June 2017

This might be done using mesh only, just worked on. But unfortunately I would sell this in store

Did you expected this fx ? https://www.dropbox.com/s/ev989ha9pdw5lec/mesh.avi?dl=0

keszegh · June 2017

@Yan,
i don't understand completely your comment.
coincidentally i have also an app called fragmenter that works similarly as your video shows, see my signature for more info about it if you are interested.
and indeed my question is related to some features related to my drawing app.

but what i want to achieve is something else:

given a mesh that is. e.g. the union of two triangles that overlap. i want to set opacity 50% for this mesh so that all pixels become 50% transparent. the problem is that setting setAlpha(50) will make each triangle 50% transparent and at the end the overlapping part will be less transparent, just like on the 2nd picture i attached to my first comment.

keszegh · June 2017

@hgy29, as a shader-expert, i'm sure you have an opinion whether it is possible what i want to achieve. thanks

Yan · June 2017

@hgy29, as a shader-expert, i'm sure you have an opinion whether it is possible what i want to achieve. thanks

Now I understand your question clearly. So 1st approach is to use shader this would be - the easiest shader in your live, because it will be 1 string shader.

Second approach is to draw shapes on RenderTarget and use setAlpha

hgy29 · June 2017

@keszegh, since you asked here is what I can think of:
A) As @Yan suggested: render your mesh without alpha to a rendertarget than render that render target with alpha.

Similar to A, draw your mesh to stencil, then draw stencil to screen. Unfortunately stencil isn't exposed to lua
C) Maybe we can do it in a single pass with stencil test and render at the same time
D) Use Z-buffer and draw your mesh in 'font to back' order: first triangle is drawn in front and last in background

I think C would be the best technique, do you reckon we should add lua bindings for OpenGL stencil management ?

Yan · June 2017

@keszegh, since you asked here is what I can think of:
A) As @Yan suggested: render your mesh without alpha to a rendertarget than render that render target with alpha.
Similar to A, draw your mesh to stencil, then draw stencil to screen. Unfortunately stencil isn't exposed to lua
C) Maybe we can do it in a single pass with stencil test and render at the same time
D) Use Z-buffer and draw your mesh in 'font to back' order: first triangle is drawn in front and last in background

I think C would be the best technique, do you reckon we should add lua bindings for OpenGL stencil management ?

Hi Friend, Tell me please the advantages of stencil before a simple shader

hgy29 · June 2017

hi @yan, I don't think it can be done with a simple shader, but I may be wrong! if you know how to do it then please share here

With stencil you could theoretically draw as usual, without a shader, and just let OpenGL keep or discard your fragment based on stencil content, while updating the stencil at the same time.

keszegh · June 2017

thanks for you comments,

@Yan, please let us know how you would do it with a shader.

meanwhile and in case the shader way won't work, @hgy29,
A) for 10000 rendertargets will surely be too slow,
B,C) sounds good, i'm not familiar with opengl stencil, so i's not clear what and how it does (having a look at a wikipedia page i was not instantly wiser), but if it can do this, then i'm for it, surely
D) i again don't understand this completely, as giving different depths for the triangles would still make this undesired effect when they become transparent, so you probably mean something more tricky.

in any case what to do to achieve a solution like C)?

Yan · June 2017

Well the shader approach meant you will draw the figure on a separate sprite - this is similar to setAlpha on a Sprite, kinda wierd way ... dont listen to me xD

So if you afraid for 10000 rendertargets will surely be too slow you can have a one big render - call this main and draw a small renders with applied alpha tro this main canvas

keszegh · June 2017

@Yan,
"you can have a one big render - call this main and draw a small renders with applied alpha tro this main canvas "

that will have exactly the same problem with intersecting triangles i think as i currently have.

hgy29 · June 2017

For C option to work gideros would have to do this:

glEnable(GL_STENCIL_TEST);
//Clear stencil buffer
glClearStencil(0);
glClear(GL_STENCIL_BUFFER_BIT);
//Stencil test will pass if current stencil value is 0, and fail otherwise
glStencilFuncSeparate(GL_FRONT_AND_BACK.GL_NOTEQUAL,1,1);
//If fragment passes all tests, increment the stencil value (and display)
glStencilOpSeparate(GL_FRONT_AND_BACK.GL_KEEP,GL_KEEP,GL_INCR);
 
draw();
 
glDsiable(GL_STENCIL_TEST);

This means that once a fragment is drawn, it will mark the stencil and all subsequents attempts to draw something at the same place will fail stencil test, and nothing will be drawn twice at the same location. Unfortunately, gideros doesn't expose stencil API from lua yet.

D option is in fact quite similar, but tries to use depth test instead of stencil test. When you draw something in 3D, you typically don't want objects in the background to be displayed in front of those in foreground. To achieve this, OpenGL tracks Z value of fragments drawn and discard fragments which are behind something already drawn.
If you can add a Z value to your mesh triangles (value shouldn't matter since you use orthographic projection) and arrange for faces to be drawn from front to back, then OpenGL will discard pixels drawn twice since they would necessarily be behind the first drawn pixels.

keszegh · June 2017

-stencil solution seems good, could you expose this in lua?

"If you can add a Z value to your mesh triangles (value shouldn't matter since you use orthographic projection) and arrange for faces to be drawn from front to back, then OpenGL will discard pixels drawn twice since they would necessarily be behind the first drawn pixels."
-are you sure? even if the top one has 50% alpha, the lower triangles will be discarded?

-if for both solutions gideros needs to be updated too than i guess the stencil solution is cleaner (as if my mesh has 1000 triangles, then i need to set different z-value for each of them, which seems to be sort of untidy solution), moreover i'm experimenting with 3d polygonal brushes, which would not work together with the depth-check solution

hgy29 · June 2017

-are you sure? even if the top one has 50% alpha, the lower triangles will be discarded?

Yes, 100% sure, read this: https://www.khronos.org/opengl/wiki/Transparency_Sorting

And yes, I will expose stencil ops to lua, in a future version, probably with Sprite object. However using depth test is possible with current gideros with 3D meshes, but I agree: this is hackish.

keszegh · June 2017

@hgy29, thanks for the pointer,
this limitation of opengl transparency turns out to be helpful, if i get time to do it, i will test the depth solution, until there are stencils. what i'm curious about though that if there is no z-value, then why transparency works fine for '2d sprites', is it done with different algorithms? or opengl recognizes if two objects are in the same plane and then alpha is done as expected?

also, about the stencil solution, will be there a way to make gideros/opengl process the triangles backwards?
because if not, then i need to add the triangles in reverse order to the mesh, which means that whenever user draws another segment of triangles, then i need to rewrite the whole indexarray to put the new triangles at the beginning - this is not too efficient.

keszegh · June 2017

@hgy29, i made a quick check with depth, the problem is that to make them smooth , i add a narrow border to my strokes which is gradually fading out to the outer border. this worked perfectly so far, but with different depths it is rendered weirdly. see attachment. i will try further, but it seems stencil will be the best possibility.

keszegh · June 2017

i realised that stencil solution will have the same problem with smoothing.
the question is how else to do this or perhaps how else i can do the smoothing so that it works with stencil.

hgy29 · June 2017

Wouldn't it work if you could draw your smooth edges after everything else ? If you split your mesh in two parts: content and edges, then you can render all contents first and all edges last. problem should then disappear

keszegh · June 2017

@hgy29, if e.g. the two lines are of different color (which in my app easily happens) than i'd need to smooth the top one over the bottom one, which won't happen if i draw smooth edges at the end with stencil-test.

actually i realised that it does happen occasionally that the user draws so much on a frame that i reach the vertexarray limit and i need to divide the drawing into more meshes. in this case stencil etc nothing seems to work, am i right?
so at the end i need to start to experiment with rendering to rendertargets, ideally i can reuse the same rendertarget as @Yan proposed, so at least memory-wise it won't be a problem, performance-wise i have worries, but i will need to test.

hgy29 · June 2017

Yes, now I understand that your problem is more intricate than I first thought, I was assuming your shapes are monochrome. If they are not then only a two pass rendering will work.

Aside from this, splitting your shape in several meshes wouldn't have been an issue, since stencil tests can be done on a group of sprites. I added stencil control in gideros anyway, it may be handy for some other case.

keszegh · June 2017

stencil control on a group of sprites sounds good. these will come handy at some point, i'm sure.

keszegh · June 2017

btw, is there no way to antialias meshes natively somehow?

hgy29 · June 2017

The only native ways I am aware of are through global scene antialiasing: FSAA, MSAA and others. @SinisterSoft and I discussed this already and we plan to add an option to make them available to gideros someday...

keszegh · June 2017

@hgy29, those sound good, only alternative so far i can guess is to put a blur shader on the whole scene rendered to a rendertarget, but that evidently will blur the lines and not antialias them, far from optimal.

keszegh · September 2019

@hgy29 can this problem be solved also with stencils? note that i need to maintain also the opacity values as i use fading out at the boundaries of the lines to simulate anti-alias.
well, i guess not, reading back my message about "actually i realised that it does happen occasionally that the user draws so much on a frame that i reach the vertexarray limit and i need to divide the drawing into more meshes. in this case stencil etc nothing seems to work, am i right? "

hgy29 · September 2019

Actually it looks like I suggested using stencil already for this. Re-reading the whole thread you'll find that your last question has already been answered!

keszegh · September 2019

perhaps, i got a bit confused, so you think that even with stencil operations in hand "only a two pass rendering will work", that is, i need to use rendertarget?

hgy29 · September 2019

Not sure, you had an issue linked to the depth test, so maybe stencil would work differently. The technique to use would be same as for the alpha thing I commented earlier

keszegh · September 2019

perhaps there is a right combination of stencil settings.
in any case, having a pixel not be able to be written after it is written once won't work for me.
just to repeat: i want to achieve the same as drawing a complicated thing and then rendering it to a rendertarget and decreasing the alpha of the result (and putting only this on screen). so there is a way to do it using rendertarget, just if doing the same is possible with stencil then i guess that would have considerably less resource penalty.

hgy29 · September 2019

I don't think it is possible in your case

keszegh · September 2019

thanks for contemplating it though.

Howdy, Stranger!

Categories

In this Discussion

Top Posters

Question about setting alpha of a mesh uniformly: an open problem for you to solve

Comments

Howdy, Stranger!

Quick Links

Categories

In this Discussion

Top Posters

Question about setting alpha of a mesh uniformly: an open problem for you to solve

Comments