Android Dev Intro - Stencil Test

1. Enable Stencil Test and Depth Test

 // Enable the depth and stencil tests
 GLES20.glEnable( GLES20.GL_DEPTH_TEST );
 GLES20.glEnable( GLES20.GL_STENCIL_TEST );

2.  set compare function 

void WINAPI glStencilFunc(GLenum func, GLint  ref, GLuint mask);

Parameters

func

The test function. The following eight tokens are valid.

Value	Meaning
GL_NEVER	Always fails.
GL_LESS	Passes if (ref & mask) < (stencil & mask).
GL_LEQUAL	Passes if (ref & mask) ≤ (stencil & mask).
GL_GREATER	Passes if (ref & mask) > (stencil & mask).
GL_GEQUAL	Passes if (ref & mask) ≥ (stencil & mask).
GL_EQUAL	Passes if (ref & mask) = (stencil & mask).
GL_NOTEQUAL	Passes if (ref & mask) ≠ (stencil & mask).
GL_ALWAYS	Always passes.

ref

The reference value for the stencil test. The ref parameter is clamped to the range [0, 2ⁿ 1], where n is the number of bitplanes in the stencil buffer.

mask

A mask that is ANDed with both the reference value and the stored stencil value when the test is done.

Return value

This function does not return a value.

Error codes

The following error codes can be retrieved by the glGetError function.

Name	Meaning
GL_INVALID_ENUM	func was not one of the eight accepted values.
GL_INVALID_OPERATION	The function was called between a call to glBegin and the corresponding call to glEnd.

Remarks

Stenciling, like z-buffering, enables and disables drawing on a per-pixel basis. You draw into the stencil planes using OpenGL drawing primitives, then render geometry and images, using the stencil planes to mask out portions of the screen. Stenciling is typically used in multipass rendering algorithms to achieve special effects, such as decals, outlining, and constructive solid geometry rendering.

The stencil test conditionally eliminates a pixel based on the outcome of a comparison between the reference value and the value in the stencil buffer. The test is enabled by glEnable and glDisable with argument GL_STENCIL_TEST. Actions taken based on the outcome of the stencil test are specified with glStencilOp.

The func parameter is a symbolic constant that determines the stencil comparison function. It accepts one of the eight values shown above. The ref parameter is an integer reference value that is used in the stencil comparison. It is clamped to the range [0, 2ⁿ 1], where n is the number of bitplanes in the stencil buffer. The maskparameter is bitwise ANDed with both the reference value and the stored stencil value, with the ANDed values participating in the comparison.

If stencil represents the value stored in the corresponding stencil buffer location, the preceding list shows the effect of each comparison function that can be specified by func. Only if the comparison succeeds is the pixel passed through to the next stage in the rasterization process (see glStencilOp). All tests treat stencilvalues as unsigned integers in the range [0, 2ⁿ 1], where n is the number of bitplanes in the stencil buffer.

Initially, the stencil test is disabled. If there is no stencil buffer, no stencil modification can occur and it is as if the stencil test always passes.

GLES20.glStencilFunc( GLES20.GL_LESS, 0x7, 0x3 );

3.  set Stencil Operation

void WINAPI glStencilOp(GLenum fail,  GLenum zfail,  GLenum zpass);

Parameters

fail

The action to take when the stencil test fails. The following six symbolic constants are accepted.

Value	Meaning
GL_KEEP	Keeps the current value.
GL_ZERO	Sets the stencil buffer value to zero.
GL_REPLACE	Sets the stencil buffer value to ref, as specified by glStencilFunc.
GL_INCR	Increments the current stencil buffer value. Clamps to the maximum representable unsigned value.
GL_DECR	Decrements the current stencil buffer value. Clamps to zero.
GL_INVERT	Bitwise inverts the current stencil buffer value.

 

zfail

Stencil action when the stencil test passes, but the depth test fails. Accepts the same symbolic constants as fail.

zpass

Stencil action when both the stencil test and the depth test pass, or when the stencil test passes and either there is no depth buffer or depth testing is not enabled. Accepts the same symbolic constants as fail.

Return value

This function does not return a value.

Error codes

The following error codes can be retrieved by the glGetError function.

Name	Meaning
GL_INVALID_ENUM	fail, zfail, or zpass was any value other than the six defined constant values.
GL_INVALID_OPERATION	The function was called between a call to glBegin and the corresponding call to glEnd.

Remarks

Stenciling, like z-buffering, enables and disables drawing on a per-pixel basis. You draw into the stencil planes using OpenGL drawing primitives, then render geometry and images, using the stencil planes to mask out portions of the screen. Stenciling is typically used in multipass rendering algorithms to achieve special effects, such as decals, outlining, and constructive solid geometry rendering.

The stencil test conditionally eliminates a pixel based on the outcome of a comparison between the value in the stencil buffer and a reference value. The test is enabled with glEnable and glDisable calls with argument GL_STENCIL_TEST, and controlled with glStencilFunc.

The glStencilOp function takes three arguments that indicate what happens to the stored stencil value while stenciling is enabled. If the stencil test fails, no change is made to the pixel's color or depth buffers, and fail specifies what happens to the stencil buffer contents.

Stencil buffer values are treated as unsigned integers. When incremented and decremented, values are clamped to 0 and 2ⁿ 1, where n is the value returned by querying GL_STENCIL_BITS.

The other two arguments to glStencilOp specify stencil buffer actions should subsequent depth buffer tests succeed (zpass) or fail (zfail). (See glDepthFunc.) They are specified using the same six symbolic constants as fail. Note that zfail is ignored when there is no depth buffer, or when the depth buffer is not enabled. In these cases, fail and zpass specify stencil action when the stencil test fails and passes, respectively.

Initially the stencil test is disabled. If there is no stencil buffer, no stencil modification can occur and it is as if the stencil tests always pass, regardless of any call toglStencilOp.

GLES20.glStencilOp( GLES20.GL_REPLACE, GLES20.GL_DECR, GLES20.GL_DECR );

==============================================================================================================================

package com.openglesbook.stenciltest;


import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.FloatBuffer;
import java.nio.ShortBuffer;


import javax.microedition.khronos.egl.EGLConfig;
import javax.microedition.khronos.opengles.GL10;


import com.openglesbook.common.ESShader;


import android.content.Context;
import android.opengl.GLES20;
import android.opengl.GLSurfaceView;




public class StencilTestRenderer implements GLSurfaceView.Renderer
{


    ///
    // Constructor
    //
    public StencilTestRenderer(Context context)
    {
        mVertices = ByteBuffer.allocateDirect(mVerticesData.length * 4)
                .order(ByteOrder.nativeOrder()).asFloatBuffer();
        mVertices.put(mVerticesData).position(0);
        mIndices = ByteBuffer.allocateDirect(mIndicesData.length * 2)
                .order(ByteOrder.nativeOrder()).asShortBuffer();
        mIndices.put(mIndicesData).position(0);
    } 
 
    ///
    // Initialize the shader and program object
    //
    public void onSurfaceCreated(GL10 glUnused, EGLConfig config)
    {
        String vShaderStr =
            "attribute vec4 a_position;   \n" +
            "void main()                  \n" +
            "{                            \n" +
            "   gl_Position = a_position; \n" +
            "}                            \n";
        
        String fShaderStr =
            "precision mediump float;  \n" +
            "uniform vec4  u_color;    \n" +
            "void main()               \n" +
            "{                         \n" +
            "  gl_FragColor = u_color; \n" +
            "}                         \n";
                                             
                
        // Load the shaders and get a linked program object
        mProgramObject = ESShader.loadProgram(vShaderStr, fShaderStr);


        // Get the attribute locations
        mPositionLoc = GLES20.glGetAttribLocation(mProgramObject, "a_position");
        
        // Get the sampler location
        mColorLoc = GLES20.glGetUniformLocation ( mProgramObject, "u_color" );


        // Set the clear color
        GLES20.glClearColor ( 0.0f, 0.0f, 0.0f, 0.0f );
        
        // Set the stencil clear value
        GLES20.glClearStencil ( 0x1 );


        // Set the depth clear value
        GLES20.glClearDepthf( 0.75f );


        // Enable the depth and stencil tests
        GLES20.glEnable( GLES20.GL_DEPTH_TEST );
        GLES20.glEnable( GLES20.GL_STENCIL_TEST );


        GLES20.glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
    }


    // /
    // Draw a triangle using the shader pair created in onSurfaceCreated()
    //
    public void onDrawFrame(GL10 glUnused)
    {
        float[][]  colors = { 
            { 1.0f, 0.0f, 0.0f, 1.0f },
            { 0.0f, 1.0f, 0.0f, 1.0f },
            { 0.0f, 0.0f, 1.0f, 1.0f },
            { 1.0f, 1.0f, 0.0f, 0.0f }
        };


        int[]  numStencilBits = new int[1];
        int[]  stencilValues = { 
           0x7, // Result of test 0
           0x0, // Result of test 1
           0x2, // Result of test 2
           0xff // Result of test 3.  We need to fill this
                //  value in a run-time
        };


        // Set the viewport
        GLES20.glViewport ( 0, 0, mWidth, mHeight );
        
        // Clear the color, depth, and stencil buffers.  At this
        //   point, the stencil buffer will be 0x1 for all pixels
        GLES20.glClear ( GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT | GLES20.GL_STENCIL_BUFFER_BIT );


        // Use the program object
        GLES20.glUseProgram ( mProgramObject );


        // Load the vertex position
        GLES20.glVertexAttribPointer ( mPositionLoc, 3, GLES20.GL_FLOAT, 
                                       false, 0, mVertices );
       
        GLES20.glEnableVertexAttribArray ( mPositionLoc );


        // Test 0:
        //
        // Initialize upper-left region.  In this case, the
        //   stencil-buffer values will be replaced because the
        //   stencil test for the rendered pixels will fail the
        //   stencil test, which is
        //
        //        ref   mask   stencil  mask
        //      ( 0x7 & 0x3 ) < ( 0x1 & 0x3 )
        //
        //   The value in the stencil buffer for these pixels will
        //   be 0x7.
        //
        GLES20.glStencilFunc( GLES20.GL_LESS, 0x7, 0x3 );
        GLES20.glStencilOp( GLES20.GL_REPLACE, GLES20.GL_DECR, GLES20.GL_DECR );
        mIndices.position(0);
        GLES20.glDrawElements( GLES20.GL_TRIANGLES, 6, GLES20.GL_UNSIGNED_SHORT, mIndices);
      
        // Test 1:
        //
        // Initialize the upper-right region.  Here, we'll decrement
        //   the stencil-buffer values where the stencil test passes
        //   but the depth test fails.  The stencil test is
        //
        //        ref  mask    stencil  mask
        //      ( 0x3 & 0x3 ) > ( 0x1 & 0x3 )
        //
        //    but where the geometry fails the depth test.  The
        //    stencil values for these pixels will be 0x0.
        //
        GLES20.glStencilFunc( GLES20.GL_GREATER, 0x3, 0x3 );
        GLES20.glStencilOp( GLES20.GL_KEEP, GLES20.GL_DECR, GLES20.GL_KEEP );
        mIndices.position(6);
        GLES20.glDrawElements( GLES20.GL_TRIANGLES, 6, GLES20.GL_UNSIGNED_SHORT, mIndices );


        // Test 2:
        //
        // Initialize the lower-left region.  Here we'll increment 
        //   (with saturation) the stencil value where both the
        //   stencil and depth tests pass.  The stencil test for
        //   these pixels will be
        //
        //        ref  mask     stencil  mask
        //      ( 0x1 & 0x3 ) == ( 0x1 & 0x3 )
        //
        //   The stencil values for these pixels will be 0x2.
        //
        GLES20.glStencilFunc( GLES20.GL_EQUAL, 0x1, 0x3 );
        GLES20.glStencilOp( GLES20.GL_KEEP, GLES20.GL_INCR, GLES20.GL_INCR );
        mIndices.position(12);
        GLES20.glDrawElements( GLES20.GL_TRIANGLES, 6, GLES20.GL_UNSIGNED_SHORT, mIndices );


        // Test 3:
        //
        // Finally, initialize the lower-right region.  We'll invert
        //   the stencil value where the stencil tests fails.  The
        //   stencil test for these pixels will be
        //
        //        ref   mask    stencil  mask
        //      ( 0x2 & 0x1 ) == ( 0x1 & 0x1 )
        //
        //   The stencil value here will be set to ~((2^s-1) & 0x1),
        //   (with the 0x1 being from the stencil clear value),
        //   where 's' is the number of bits in the stencil buffer
        //
        GLES20.glStencilFunc( GLES20.GL_EQUAL, 0x2, 0x1 );
        GLES20.glStencilOp( GLES20.GL_INVERT, GLES20.GL_KEEP, GLES20.GL_KEEP );
        mIndices.position(18);
        GLES20.glDrawElements( GLES20.GL_TRIANGLES, 6, GLES20.GL_UNSIGNED_SHORT, mIndices );
        
        // Since we don't know at compile time how many stecil bits are present,
        //   we'll query, and update the value correct value in the
        //   stencilValues arrays for the fourth tests.  We'll use this value
        //   later in rendering.
        
        GLES20.glGetIntegerv( GLES20.GL_STENCIL_BITS, numStencilBits, 0 );
        
        stencilValues[3] = ~(((1 << numStencilBits[0]) - 1) & 0x1) & 0xff;


        // Use the stencil buffer for controlling where rendering will
        //   occur.  We diable writing to the stencil buffer so we
        //   can test against them without modifying the values we
        //   generated.
        GLES20.glStencilMask( 0x0 );
        
        for ( int i = 0; i < 4; ++i )
        {
            GLES20.glStencilFunc( GLES20.GL_EQUAL, stencilValues[i], 0xff );
            GLES20.glUniform4f( mColorLoc, colors[i][0], colors[i][1], colors[i][2], colors[i][3]);
            mIndices.position(24);
            GLES20.glDrawElements( GLES20.GL_TRIANGLES, 6, GLES20.GL_UNSIGNED_SHORT, mIndices );
        }
    }


    ///
    // Handle surface changes
    //
    public void onSurfaceChanged(GL10 glUnused, int width, int height)
    {
        mWidth = width;
        mHeight = height;
    }


    
    // Handle to a program object
    private int mProgramObject;
    
    // Attribute locations
    private int mPositionLoc;
    
    // Uniform location
    private int mColorLoc;
    
    // Additional member variables
    private int mWidth;
    private int mHeight;
    private FloatBuffer mVertices;
    private ShortBuffer mIndices;
    
    private final float[] mVerticesData =
    { 
            -0.75f,  0.25f,  0.50f, // Quad #0
            -0.25f,  0.25f,  0.50f,
            -0.25f,  0.75f,  0.50f,
            -0.75f,  0.75f,  0.50f,
             0.25f,  0.25f,  0.90f, // Quad #1
             0.75f,  0.25f,  0.90f,
             0.75f,  0.75f,  0.90f,
             0.25f,  0.75f,  0.90f,
            -0.75f, -0.75f,  0.50f, // Quad #2
            -0.25f, -0.75f,  0.50f,
            -0.25f, -0.25f,  0.50f,
            -0.75f, -0.25f,  0.50f,
             0.25f, -0.75f,  0.50f, // Quad #3
             0.75f, -0.75f,  0.50f,
             0.75f, -0.25f,  0.50f,
             0.25f, -0.25f,  0.50f,
            -1.00f, -1.00f,  0.00f, // Big Quad
             1.00f, -1.00f,  0.00f,
             1.00f,  1.00f,  0.00f,
            -1.00f,  1.00f,  0.00f
    };


    private final short[] mIndicesData =
    { 
            0,  1,  2,  0,  2,  3,  // Quad #0
            4,  5,  6,  4,  6,  7,  // Quad #1
            8,  9, 10,  8, 10, 11,  // Quad #2
            12, 13, 14, 12, 14, 15, // Quad #3
            16, 17, 18, 16, 18, 19  // Big Quad
    };
    
}