Date Archives April 2019

OpenGL Rendering Pipeline

Pipeline Overview

OpenGL-managed memory buffers (vertex buffers) contains below two array types

  • Vertex Array – this later projected into screen space, assembled into triangles, and rasterized into pixel-size fragments, and the fragments finally assigned color values and drawn to the frame buffers.
  • Element Array – contains indices of vertex (the order of the index matters when assembling triangles)

 

Vertex shader

  • GPU begins by reading each selected vertex out of the vertex array.
  • Vertex shader calculates the projected position of the vertex in screen space.
  • It can also generate other varying outputs, such as a color or texture coordinates, for the rasterizer to blend across the surface of the triangles connecting the vertex.

 

Triangle assembly

  • Connects the projected vertices to form triangles (3 ways)
    • Take every three elements as an independent triangle
    • Make a triangle strip, reusing the last two vertices of each triangle as the first two vertices of the next
    • Make a triangle fan, connecting the first element to every subsequent pair of elements

 

Rasterization

  • Takes each triangle, and clips it and discards parts that are outside the screen
  • Breaks the remaining visible parts into pixel-sized fragments
  • If vertex had a color value, rasterizer blend those colors across the pixelated surface

 

Fragment shader

  • Gets varying values of vertices after vertex shader and rasterizer steps
  • Outputs color and depth values that then get drawn into the frame buffer
  • Also includes texture mapping and lighting
  • fragment shader runs independently for every pixel drawn, it can perform the most sophisticated special effects.
  • The most performance-sensitive part of the graphics pipeline.

 

Frame buffer

  • Color buffers
  • Depth buffer and stencil buffer filter fragments before drawn to frame buffer
    • Depth testing: discards fragments from objects that are behind the ones already drawn
    • Stencil testing: uses shapes drawn into the stencil buffer to constrain the drawable part of the framebuffer
  • Final color, depth, and stencil values are drawn into corresponding buffers.

 

 

 

References

AR Platforms and Engines

PLATFORMS

ARCore (Google)

  • Supports Android/IOS
  • For rendering, uses OpenGL

ARKit (Apple)

  • For rendering, uses Metal API (previously used OpenGL)

 

Engine or SDK

Unity

  • Cross-platform game engine
  • Supports C#, Javascript, Boo
  • Integration of unity inside android code is painful

Sceneform SDK

  • Google’s 3D framework (released 2018)
  • Sceneform APIs help to develop ARCore apps (e.g., detecting plane, lights estimation)
  • No need to learn 3D graphics or openGL

Vuforia SDK

  • Uses ARCore/ARKit if the running device supports it. If not, it runs own AR technology and engine.
  • Can be used in Unity