pixelate shader behaves strange on android

keszegh

so i've the following shader to pixelate an image which works fine on desktop but on my android tablet it stretches the (horizontally) middle ~1/4th portion of the image, see screenshot:

#ifdef GL_ES precision mediump float; #endif uniform lowp vec4 fColor; uniform lowp sampler2D fTexture; uniform mediump vec4 fTextureInfo; uniform mediump float fResolution; uniform mediump vec4 fTexelSize; varying mediump vec2 fTexCoord;   void main() { if (fResolution==64.0) {gl_FragColor=texture2D(fTexture,fTexCoord); return;}   float fRes=fResolution*2.0-1.0; mediump vec2 coord;   coord.y=(fTexCoord.y/fTextureInfo.y-0.5); coord.x=(fTexCoord.x/fTextureInfo.x-0.5)/fTextureInfo.z*fTextureInfo.x*fTextureInfo.w/fTextureInfo.y;   mediump vec2 coordrounded; coordrounded.y= (floor(coord.y*fRes+mod(fRes,2.0)*0.5)/ fRes+0.5)*fTextureInfo.y; coordrounded.x= (floor(coord.x*fRes+mod(fRes,2.0)*0.5)/ fRes*fTextureInfo.z/fTextureInfo.x/fTextureInfo.w*fTextureInfo.y+0.5)*fTextureInfo.x;   lowp vec4 col=texture2D(fTexture,coordrounded);   gl_FragColor = vec4(col); }

can you tell me what's the issue?

keszegh

to easier understand what's those computations, the aim is that fResolution sets the number of pixels vertically (more precisely the number of pixels i want to be fRes=fResolution*2.0-1.0 for whatever reason which does not matter here). hopefully this is irrelevant to the issue though.

keszegh

also in another shader i've a time component:
uniform mediump float fTime;
defined like this in lua:
{name="fTime",type=Shader.CFLOAT,vertex=false},

which works on desktop but is not updated on android. any remedies?

MoKaLux

in my little experience so far with shaders (when I was using another engine), I noticed it never worked as expected on android. Maybe due to the difference of power processing.

rrraptor

Well, you can use RenderTarget instead of shader

keszegh

not every shader can be emulated with a rendertarget. even a pixelate shader would be much slower that way i think, while speed is essential for me.

hgy29

The issue is usually that GL ES devices don't have enough floating point precision, at less less than desktop. I am not sure why you divide/multiply your coordinates by fTextureInfo parameters and only in X direction, but you may lose precision in the process. You can try to make your texture coordinates variable a 'highp' float to see if it works, but that won't be supported on all android devices.

keszegh

@hgy29 , i will try what you suggest. the computation i do seems to be complicated, maybe it's unnecessary, just that's what i could come up with. my aim was that the pixels should be square-shaped, the height of the texture should be split into any given odd number (the fRes value) of pixels, and one pixel should be in the center of the image.

all in all if precision is the issue it's still surprising to produce an image which is fine except for the middle part.

also, precision does not explain the other issue with time variant not being updated, any ideas about that one? thanks

hgy29

keszegh said:

also, precision does not explain the other issue with time variant not being updated, any ideas about that one? thanks

I find it surprising that it is updated on desktop when declared just as

{name="fTime",type=Shader.CFLOAT,vertex=false},

. It misses a sys=Shader.SYS_TIMER field to tell Gideros to update it with the time value, unless you do that yourself in code ?

keszegh

well, i don't do anything else. i just copied the shader from somewhere and modified it a bit, and was happy that it worked.
so you say that it should correctly be:
{name="fTime",type=Shader.CFLOAT,sys=Shader.SYS_TIMER,vertex=false},
and then maybe it would work on android as well?
thanks

UPDATE: well, it did not seem to work, on android still nothing happens, on desktop it's still ok.

hgy29

Well I mean it shouldn't have worked on Windows. If it did then some lua code was probably updating the fTime constant. Now why the same code would work on Windows and not on Android, I don't know beside that well known precision stuff.

keszegh

just for your information here is the shader that uses time and works on windows for some reason:

#ifdef GL_ES
precision mediump float;
#endif
uniform lowp vec4 fColor;
uniform lowp sampler2D fTexture;
uniform mediump vec4 fTextureInfo;
//uniform mediump float fStrength;
uniform mediump float fResolution;
uniform mediump float fTime;
uniform mediump vec4 fTexelSize;
varying mediump vec2 fTexCoord;

//uniform float iTime=60.0; // shader playback time (in seconds)


//2D (returns 0 - 1)
float random2d(vec2 n) {
return fract(sin(dot(n, vec2(12.9898, 4.1414))) * 43758.5453);
}

float randomRange (in vec2 seed, in float min, in float max) {
return min + random2d(seed) * (max - min);
}

// return 1 if v inside 1d range
float insideRange(float v, float bottom, float top) {
return step(bottom, v) - step(top, v);
}



void main()
{
//inputs
//float AMT = 1.0-fResolution/128.0; //0 - 1 glitch amount
//float AMT = 0.25-fResolution/512.0; //0 - 1 glitch amount
float AMT = 0.25-fResolution/256.0; //0 - 1 glitch amount
float SPEED = 0.5; //0 - 1 speed

if (fResolution==64.0) {gl_FragColor=texture2D(fTexture,fTexCoord); return;}

float time = floor(fTime * SPEED * 60.0);

//vec2 uv = fTexCoord.xy / fResolution.xy;
vec2 uv=fTexCoord.xy;
//copy orig

vec4 outCol = texture2D(fTexture, uv).rgba;

//randomly offset slices horizontally
float maxOffset = AMT/2.0;
for (float i = 0.0; i < 10.0 * AMT; i += 1.0) {
float sliceY = random2d(vec2(time , 2345.0 + float(i)));
float sliceH = random2d(vec2(time , 9035.0 + float(i))) * 0.25;
float hOffset = randomRange(vec2(time , 9625.0 + float(i)), -maxOffset, maxOffset);
vec2 uvOff = uv;
uvOff.x += hOffset;
if (insideRange(uv.y, sliceY, fract(sliceY+sliceH)) == 1.0 ){
outCol = texture2D(fTexture, uvOff).rgba;
}
}

//do slight offset on one entire channel
float maxColOffset = AMT/6.0;
float rnd = random2d(vec2(time , 9545.0));
vec2 colOffset = vec2(randomRange(vec2(time , 9545.0),-maxColOffset,maxColOffset),
randomRange(vec2(time , 7205.0),-maxColOffset,maxColOffset));
if (rnd < 0.33){
outCol.r = texture2D(fTexture, uv + colOffset).r;

}else if (rnd < 0.66){
outCol.g = texture2D(fTexture, uv + colOffset).g;

} else{
outCol.b = texture2D(fTexture, uv + colOffset).b;
}

//gl_FragColor = vec4(outCol,1.0);
outCol.a = texture2D(fTexture, uv + colOffset).a;
gl_FragColor = vec4(outCol);
}

defined like this in lua:

shader_glitch=Shader.new("vshader_basic","fshader_glitch",0,
{
{name="vMatrix",type=Shader.CMATRIX,sys=Shader.SYS_WVP,vertex=true},
{name="fColor",type=Shader.CFLOAT4,sys=Shader.SYS_COLOR,vertex=false},
{name="fTextureInfo",type=Shader.CFLOAT4,sys=Shader.SYS_TEXTUREINFO,vertex=false},
{name="fTexture",type=Shader.CTEXTURE,vertex=false},
{name="fTexelSize",type=Shader.CFLOAT4,vertex=false},
{name="fResolution",type=Shader.CFLOAT,vertex=false},
{name="fTime",type=Shader.CFLOAT,vertex=false},
{name="fDir",type=Shader.CFLOAT2,vertex=false},
},
{
{name="vVertex",type=Shader.DFLOAT,mult=3,slot=0,offset=0},
{name="vColor",type=Shader.DUBYTE,mult=4,slot=1,offset=0},
{name="vTexCoord",type=Shader.DFLOAT,mult=2,slot=2,offset=0},
});

UPDATE: ok, i've found it, that's how i update it on onEnterFrame (i completely forgot):
shader_glitch:setConstant("fTime", Shader.CFLOAT, 1, os.timer())

keszegh

still i don't understand why it is not updating on android, again must be a precision issue?

keszegh

thanks btw, now i finally know that this setConstant line is not needed if i change the def with the SYS_TIMER as you suggested. now i just have to find and correct the precision issue which makes it not work on android. or let me know if you can spot the problem in the glsl i shared.

keszegh

@hgy29, i've changed my coord variables to highp and pixelate does work now.
i still don't know what should i change in the glitch shader to make it work.

hgy29

I don't know what you should change, because I don't understand what you do with the coordinates and why. But a mediump mantissa is at least 10 bit, and a highp is at least 16. Since it works with highp and not mediump, then you must be needing more than 10 mantissa bits at some point in your math. If your texture is 1024x1024 then all ten bits are used and there is no room sub texel coordinates.

keszegh

@hgy29 , so in the shaders which had errors i've changed the beginning to
#ifdef GL_ES
precision highp float;
#endif

which made everything work fine on my two android machines.
in addition in the glitch shader there was an unnecessary rounding, changing
float time = floor(fTime * SPEED * 60.0);
to
time = floor(fTime * SPEED * 60.0);
made it function correctly.

i'm really grateful for your help (as it happens quite often).

rrraptor

Let it be here
I have same fps with and without pixelate effect.

PixelTarget = Core.class(Sprite)   function PixelTarget:init(pixelate, w, h) assert(pixelate ~= 0, "Pixelate cant be zero!") self.pixelate = pixelate self.w = w or application:getContentWidth() self.h = h or application:getContentHeight()     self.canvas = RenderTarget.new(self.w / self.pixelate, self.h / self.pixelate) self.btm = Bitmap.new(self.canvas) self.btm:setScale(self.pixelate) self:addChild(self.btm) end   function PixelTarget:update(sprite, x, y) self.canvas:clear(0,0) sprite:setScale(1 / self.pixelate) self.canvas:draw(sprite, x, y) sprite:setScale(1) end ----------------------------------------------------------------- MAX_OBJ @ 3000   local W = application:getContentWidth() local H = application:getContentHeight()   local rt = PixelTarget.new(4, W, H) stage:addChild(rt) local scene = Sprite.new()   local t = {} for i = 1, MAX_OBJ do local px = Pixel.new(math.random(0xffffff), 0.5 + math.random() * 0.5, math.random(32, 64), math.random(32, 64)) local x = math.random() * W local y = math.random() * H px.initX = x px.initY = y px.rotationSpeed = math.random(-100, 100) px.radX = math.random(100) px.radY = math.random(100) px:setPosition(x, y) px:setAnchorPoint(.5,.5) scene:addChild(px) t[i] = px end   --stage:addChild(scene) -- uncomment this to see result without fake shader <img class="emoji" src="https://forum.giderosmobile.com/resources/emoji/smiley.png" title="=)" alt="=)" height="20" />   local fps_tf = TextField.new(nil, "FPS: -", "Aq|") fps_tf:setPosition(20, 20) fps_tf:setScale(2) stage:addChild(fps_tf)   local timer = 0 local fps_timer = 0 stage:addEventListener("enterFrame", function(e) local dt = e.deltaTime   timer += dt fps_timer += dt   if (fps_timer > 1) then fps_timer = 0 fps_tf:setText("FPS: ".. 1 // dt) end   for i, px in ipairs(t) do px:setRotation(px:getRotation() + px.rotationSpeed * dt) px:setPosition(px.initX + math.cos(timer) * px.radX, px.initY + math.sin(timer) * px.radY) end rt:update(scene) -- comment this to see result without fake shader <img class="emoji" src="https://forum.giderosmobile.com/resources/emoji/smiley.png" title="=)" alt="=)" height="20" /> end)

keszegh

@rrraptor , thanks, i forgot about this method of doing pixelation effect. it is effective indeed. the main issue is that one has no control about the position of pixels, in particular i need that one pixel is in the center of the bmp/texture, which seems to be hard to achieve with your method.
also i use different shaders as 'special fx' in my animation app and it's convenient to just simply change the shader when i need a new fx.