Quick Links: Download Gideros Studio | Gideros Documentation | Gideros Development Center | Gideros community chat | DONATE
Question about setting alpha of a mesh uniformly: an open problem for you to solve — Gideros Forum

Question about setting alpha of a mesh uniformly: an open problem for you to solve

keszeghkeszegh Member
edited June 2017 in General questions
So i'm trying to fade off a mesh but setting it's alpha sets the alpha of each constituting triangle separately, thus overlapping parts fade differently from non-overlapping parts. this is not good for me, i'd prefer uniform fade off.
see attached figure for full alpha, and faded off alpha, the intersection of the two strokes is more opaque, that's what i don't want.
is there any solution? rendering to rendertarget and then setting alpha for this would work but i'm working with thousands of meshes, and so this would slow down the app too much. i was hoping that maybe you can think of some other solution, using shaders perhaps?

thanks for your ideas.

Comments

  • YanYan Member
    edited June 2017
    This might be done using mesh only, just worked on. But unfortunately I would sell this in store

    Did you expected this fx ? https://www.dropbox.com/s/ev989ha9pdw5lec/mesh.avi?dl=0
    vk.com/yan_alex
  • @Yan,
    i don't understand completely your comment.
    coincidentally i have also an app called fragmenter that works similarly as your video shows, see my signature for more info about it if you are interested.
    and indeed my question is related to some features related to my drawing app.

    but what i want to achieve is something else:

    given a mesh that is. e.g. the union of two triangles that overlap. i want to set opacity 50% for this mesh so that all pixels become 50% transparent. the problem is that setting setAlpha(50) will make each triangle 50% transparent and at the end the overlapping part will be less transparent, just like on the 2nd picture i attached to my first comment.
  • @hgy29, as a shader-expert, i'm sure you have an opinion whether it is possible what i want to achieve. thanks
  • YanYan Member
    @hgy29, as a shader-expert, i'm sure you have an opinion whether it is possible what i want to achieve. thanks
    Now I understand your question clearly. So 1st approach is to use shader this would be - the easiest shader in your live, because it will be 1 string shader.

    Second approach is to draw shapes on RenderTarget and use setAlpha
    vk.com/yan_alex
  • hgy29hgy29 Maintainer
    edited June 2017
    @keszegh, since you asked here is what I can think of:
    A) As @Yan suggested: render your mesh without alpha to a rendertarget than render that render target with alpha.
    B) Similar to A, draw your mesh to stencil, then draw stencil to screen. Unfortunately stencil isn't exposed to lua
    C) Maybe we can do it in a single pass with stencil test and render at the same time
    D) Use Z-buffer and draw your mesh in 'font to back' order: first triangle is drawn in front and last in background

    I think C would be the best technique, do you reckon we should add lua bindings for OpenGL stencil management ?
  • YanYan Member
    @keszegh, since you asked here is what I can think of:
    A) As @Yan suggested: render your mesh without alpha to a rendertarget than render that render target with alpha.
    B) Similar to A, draw your mesh to stencil, then draw stencil to screen. Unfortunately stencil isn't exposed to lua
    C) Maybe we can do it in a single pass with stencil test and render at the same time
    D) Use Z-buffer and draw your mesh in 'font to back' order: first triangle is drawn in front and last in background

    I think C would be the best technique, do you reckon we should add lua bindings for OpenGL stencil management ?
    Hi Friend, Tell me please the advantages of stencil before a simple shader
    vk.com/yan_alex
  • hgy29hgy29 Maintainer
    hi @yan, I don't think it can be done with a simple shader, but I may be wrong! if you know how to do it then please share here :)

    With stencil you could theoretically draw as usual, without a shader, and just let OpenGL keep or discard your fragment based on stencil content, while updating the stencil at the same time.
  • keszeghkeszegh Member
    edited June 2017
    thanks for you comments,

    @Yan, please let us know how you would do it with a shader.

    meanwhile and in case the shader way won't work, @hgy29,
    A) for 10000 rendertargets will surely be too slow,
    B,C) sounds good, i'm not familiar with opengl stencil, so i's not clear what and how it does (having a look at a wikipedia page i was not instantly wiser), but if it can do this, then i'm for it, surely
    D) i again don't understand this completely, as giving different depths for the triangles would still make this undesired effect when they become transparent, so you probably mean something more tricky.

    in any case what to do to achieve a solution like C)?
  • YanYan Member
    Well the shader approach meant you will draw the figure on a separate sprite - this is similar to setAlpha on a Sprite, kinda wierd way ... dont listen to me xD

    So if you afraid for 10000 rendertargets will surely be too slow you can have a one big render - call this main and draw a small renders with applied alpha tro this main canvas
    vk.com/yan_alex
  • @Yan,
    "you can have a one big render - call this main and draw a small renders with applied alpha tro this main canvas "

    that will have exactly the same problem with intersecting triangles i think as i currently have.
  • hgy29hgy29 Maintainer
    For C option to work gideros would have to do this:
    glEnable(GL_STENCIL_TEST);
    //Clear stencil buffer
    glClearStencil(0);
    glClear(GL_STENCIL_BUFFER_BIT);
    //Stencil test will pass if current stencil value is 0, and fail otherwise
    glStencilFuncSeparate(GL_FRONT_AND_BACK.GL_NOTEQUAL,1,1);
    //If fragment passes all tests, increment the stencil value (and display)
    glStencilOpSeparate(GL_FRONT_AND_BACK.GL_KEEP,GL_KEEP,GL_INCR);
     
    draw();
     
    glDsiable(GL_STENCIL_TEST);
    This means that once a fragment is drawn, it will mark the stencil and all subsequents attempts to draw something at the same place will fail stencil test, and nothing will be drawn twice at the same location. Unfortunately, gideros doesn't expose stencil API from lua yet.

    D option is in fact quite similar, but tries to use depth test instead of stencil test. When you draw something in 3D, you typically don't want objects in the background to be displayed in front of those in foreground. To achieve this, OpenGL tracks Z value of fragments drawn and discard fragments which are behind something already drawn.
    If you can add a Z value to your mesh triangles (value shouldn't matter since you use orthographic projection) and arrange for faces to be drawn from front to back, then OpenGL will discard pixels drawn twice since they would necessarily be behind the first drawn pixels.


  • -stencil solution seems good, could you expose this in lua?

    "If you can add a Z value to your mesh triangles (value shouldn't matter since you use orthographic projection) and arrange for faces to be drawn from front to back, then OpenGL will discard pixels drawn twice since they would necessarily be behind the first drawn pixels."
    -are you sure? even if the top one has 50% alpha, the lower triangles will be discarded?

    -if for both solutions gideros needs to be updated too than i guess the stencil solution is cleaner (as if my mesh has 1000 triangles, then i need to set different z-value for each of them, which seems to be sort of untidy solution), moreover i'm experimenting with 3d polygonal brushes, which would not work together with the depth-check solution
  • hgy29hgy29 Maintainer

    -are you sure? even if the top one has 50% alpha, the lower triangles will be discarded?
    Yes, 100% sure, read this: https://www.khronos.org/opengl/wiki/Transparency_Sorting

    And yes, I will expose stencil ops to lua, in a future version, probably with Sprite object. However using depth test is possible with current gideros with 3D meshes, but I agree: this is hackish.

  • @hgy29, thanks for the pointer,
    this limitation of opengl transparency turns out to be helpful, if i get time to do it, i will test the depth solution, until there are stencils. what i'm curious about though that if there is no z-value, then why transparency works fine for '2d sprites', is it done with different algorithms? or opengl recognizes if two objects are in the same plane and then alpha is done as expected?

    also, about the stencil solution, will be there a way to make gideros/opengl process the triangles backwards?
    because if not, then i need to add the triangles in reverse order to the mesh, which means that whenever user draws another segment of triangles, then i need to rewrite the whole indexarray to put the new triangles at the beginning - this is not too efficient.
  • @hgy29, i made a quick check with depth, the problem is that to make them smooth , i add a narrow border to my strokes which is gradually fading out to the outer border. this worked perfectly so far, but with different depths it is rendered weirdly. see attachment. i will try further, but it seems stencil will be the best possibility.
    opacity50 depth.png
    524 x 480 - 6K
  • i realised that stencil solution will have the same problem with smoothing.
    the question is how else to do this or perhaps how else i can do the smoothing so that it works with stencil.
  • hgy29hgy29 Maintainer
    Wouldn't it work if you could draw your smooth edges after everything else ? If you split your mesh in two parts: content and edges, then you can render all contents first and all edges last. problem should then disappear
  • keszeghkeszegh Member
    edited June 2017
    @hgy29, if e.g. the two lines are of different color (which in my app easily happens) than i'd need to smooth the top one over the bottom one, which won't happen if i draw smooth edges at the end with stencil-test.

    actually i realised that it does happen occasionally that the user draws so much on a frame that i reach the vertexarray limit and i need to divide the drawing into more meshes. in this case stencil etc nothing seems to work, am i right?
    so at the end i need to start to experiment with rendering to rendertargets, ideally i can reuse the same rendertarget as @Yan proposed, so at least memory-wise it won't be a problem, performance-wise i have worries, but i will need to test.
  • hgy29hgy29 Maintainer
    Yes, now I understand that your problem is more intricate than I first thought, I was assuming your shapes are monochrome. If they are not then only a two pass rendering will work.

    Aside from this, splitting your shape in several meshes wouldn't have been an issue, since stencil tests can be done on a group of sprites. I added stencil control in gideros anyway, it may be handy for some other case.

    Likes: antix, keszegh

    +1 -1 (+2 / -0 )Share on Facebook
  • stencil control on a group of sprites sounds good. these will come handy at some point, i'm sure.
  • btw, is there no way to antialias meshes natively somehow?
  • hgy29hgy29 Maintainer
    The only native ways I am aware of are through global scene antialiasing: FSAA, MSAA and others. @SinisterSoft and I discussed this already and we plan to add an option to make them available to gideros someday...
    +1 -1 (+2 / -0 )Share on Facebook
  • @hgy29, those sound good, only alternative so far i can guess is to put a blur shader on the whole scene rendered to a rendertarget, but that evidently will blur the lines and not antialias them, far from optimal.
  • keszeghkeszegh Member
    edited September 2019
    @hgy29 can this problem be solved also with stencils? note that i need to maintain also the opacity values as i use fading out at the boundaries of the lines to simulate anti-alias.
    well, i guess not, reading back my message about "actually i realised that it does happen occasionally that the user draws so much on a frame that i reach the vertexarray limit and i need to divide the drawing into more meshes. in this case stencil etc nothing seems to work, am i right? "
  • hgy29hgy29 Maintainer
    Actually it looks like I suggested using stencil already for this. Re-reading the whole thread you'll find that your last question has already been answered!
  • perhaps, i got a bit confused, so you think that even with stencil operations in hand "only a two pass rendering will work", that is, i need to use rendertarget?
  • hgy29hgy29 Maintainer
    Not sure, you had an issue linked to the depth test, so maybe stencil would work differently. The technique to use would be same as for the alpha thing I commented earlier
  • perhaps there is a right combination of stencil settings.
    in any case, having a pixel not be able to be written after it is written once won't work for me.
    just to repeat: i want to achieve the same as drawing a complicated thing and then rendering it to a rendertarget and decreasing the alpha of the result (and putting only this on screen). so there is a way to do it using rendertarget, just if doing the same is possible with stencil then i guess that would have considerably less resource penalty.
  • hgy29hgy29 Maintainer
    I don't think it is possible in your case
Sign In or Register to comment.