GPU Programming for Video Games (2014)

GPU Programming for Video Games (2014)

Summer 2014


  • Almost over!



The shader programs in the slides and unitypackages below
are intended to
illustrate various aspects of shader programming; they are not necessarily
the most efficient or congruent with Unity’s workflow.

  • 5/13: Session 1 – Introduction
    (PDF slides,
    PDF slides 4-up)

  • 5/15: Session 2 – History of gaming hardware
    (PDF slides,
    PDF slides 4-up)

  • 5/20: Session 3 – 3D Coordinates & Transformations
    (PDF slides,
    PDF slides 4-up)
  • 5/22: Session 4 – 3D to 2D Projection
    (PDF slides,
    PDF slides 4-up)
  • 5/27: Session 5 – Lighting & Rasterization
    (PDF slides,
    PDF slides 4-up)
  • 5/29: Session 6 – Texturing & Blending
    (PDF slides,
    PDF slides 4-up)
  • 6/3: Session 7 – GPUs – Under the Hood
    (PDF slides,
    PDF slides 4-up)
  • 6/5: Session 8 – Materials in Unity
    (PDF slides,
    PDF slides 4-up)
    (this lecture will consist of a live
    with Unity; the slides are more of an outline than something I intend
    to show)

    • Vertex vs. pixel
      shading demo
      : Create a new Unity project and drag the unitypackage
      file onto the Asset folder to import it, and then run the
      “Barrels VvD Lighting Demo” scene. Number keys 0 through 5 change the speed
      of rotation, and the x, y, and z keys toggle on and off rotations around those
  • 6/10: Session 9 – Introduction to Shader Languages, Part 1
    (PDF slides,
    PDF slides 4-up)

    • GPU14
      Introductory Shaders (June 14 revision)
      Run “GPUXXIntroShadersScene” to see two spinning treasure chests
      by Mobile Design Lab
      free on Unity Asset store), along with some other spinning objects. There
      is an “original” version and a “GPUXX” version of each of the materials,
      so you can experiment with comparing different shaders and textures.
      I replaced the treasure chest normal map with a normal
      map based on the Georgia Tech
      logo; you can put the treasure chest normal map back in to see something
      more subtle.
      There is also a non-visible game object
      that is just a host for a script I wrote to create some matrices and
      multiply them in various ways. I used this when I was trying to figure out
      what conventions Unity uses, since the official documentation is sometimes
      and confusing.
  • 6/12: Session 10 – Introduction to Shader Languages, Part 2
  • 6/17: Session 11 – Environment Mapping
    (Updated 6/21: PDF slides,
    PDF slides 4-up)

    • GPU14 Environment Mapping
      : This contains three scenes. One compares the “original Phong”
      (displayed as red) and “Phong-Blinn” (displayed as blue)
      specular models, so pixels where the two models are give similar results
      appear as purple. By adjusting the powers, you can get them to match pretty
      closely. The other two scenes illustrate environment mapping, one with
      per-vertex and one with per-pixel computations. A “crossfade” slider lets
      you blend reflections and refractions. The per-vertex version shows
      clear triangulation artifacts, so I didn’t bother implementing Fresnel
      effects for the per-vertex shader, but I did for the per-pixel shader.
      Depending on which “return” statement in the per-pixel shader is in effect
      (comment out the other one), you can use either the original crossfade slider,
      or play with the Fresnel parameters. (The
      model and animations
      , free
      on Unity Asset store, are by
      Dootsy Development. The
      cube map
      is by Humus.)
  • 6/19: Session 12 – Projective Textures & Shadow Mapping
    (PDF slides,
    PDF slides 4-up)

    • GPU14 Projective Texturing
      : Run “GPUXXProjTexScene” — have fun! (For some reason, the
      Scene view sometimes comes up as pure black when you first load the scene;
      just press “play” and then “stop” — that seems to fix it.) (Base
      textures from Lost Garden.)
  • 6/24: Session 13 – Introduction to Surface Shaders
    (PDF slides,
    PDF slides 4-up)
  • 6/26: Session 14 – Case Study:
    Portalarium’s Multi-Shader (with
    Shroud of the Avatar
    pre-alpha demo)
  • 7/1: Session 15 – Surface Shaders with Vertex Modification
    • GPU14
      Vertex Modification Demo
      Run “GPUXXVertexModScene.” One of the squares has a 100×100
      tessellation; the other, which often demonstrates spatial aliasing, has
      a 10×10 tessilation. A custom material inspector lets you use a checkbox
      to switch between planar and circular waves. The “direction” setting is
      ignored if circular waves are active.
  • 7/8: Session 16 – Postprocessing (demo to be posted)
  • 7/10: Session 17 – G-buffers
    • Run the “GPUDepthNormalsTest” scene in the Deferred Rendering
      unitypackage from Session 18 below.
  • 7/15: Session 18 – Deferred Rendering
  • 7/17: Session 19 – Advanced Post Processing with G-Buffers
    • GPU14
      Advanced Post Processing Demo
      Make sure “process” in the AdvPostProc component attached to the Main
      Camera is checked, then
      experiment with the “return” lines in the GPUXXFancyPostProcTest.shader.

Details and descriptions

This webpage constitutes the class syllabus.

Course description: (2 hours lecture, 3 hours unscheduled lab,
3 credit hours total) 3-D graphics pipelines.
Real-time simulation concerns. Game engine architectures.
GPU architectures. Graphics APIs. Vertex, pixel, and compute
shader programming. Deferred rendering. Performance analysis.

Course objective: This class provides the GPU programming
concepts needed to meet timely demands of the multimedia,
visualization, and gaming industries.
The course also bridges the gap between our
current generic computer
architecture courses and the video game design courses offered by
the College of Computing,
the School of Electrical and
Computer Engineering
, and the
School of Literature, Media, and Culture.
The class covers state-of-the-art general purpose graphical processing unit architectures (GPGPUs) from application and hardware design perspectives. The course considers programming models using examples from the algorithmic needs of modern 3-D games (e.g. geometry processing, shading algorithms, physical modeling, collision detection, and artificial intelligence).

Tentative topics

The topics and
the order in which they are covered is subject to change.

  1. Introduction and historical context
  2. Classic 3-D rendering pipeline: geometry transformation, lighting, texturing
  3. Overview of 3-D APIs (ex: OpenGL and Direct3D)
  4. Simulation loops and game engine components
  5. Object-oriented game engines (ex: Unity, Unreal 4, C4)
  6. GPU architectures and GPU assembly code
  7. Architecture comparison: Playstation 4 vs. Xbox One vs. Wii U vs.
    “gaming PC”
  • Introduction to shading languages (HLSL/Cg), vertex and pixel shaders
  • Per-pixel vs. per-vertex lighting
  • Advanced 3-D shading effects (ex: bump mapping, environment mapping)
  • Postprocessing effects (ex: bloom, motion blur)
  • Compute shaders (ex: decompression, physics simulations)
  • Deferred rendering
  • Screen space techniques (ex: ambient occlusion)
  • Real-time shadow techniques
  • Profiling and performance analysis
  • Adapting to the limitations of hand-held devices
  • Possibly occasional guest speakers from industry
    (previous speakers)Note that the course
    does not cover OpenCL or CUDA, and unlike its
    predecessor, “Multicore and GPU Programming for Video Games,”
    does not cover multicore CPU programming.
    OpenCL, CUDA, and general
    multicore programming are well covered in many other classes in ECE and CoC, whereas the use of GPU architectures
    for their native application of computer graphics
    is not extensively
    covered in many other classes, either here at Georgia Tech or other schools.

    Course history

    This is a heavily revised version of an earlier special topics class called
    “Multicore and GPU Programming for Video Games,”
    was taught
    six times,
    in the Fall semesters from 2007 to 2012. Prof.
    Sean Lee
    and I co-created
    the class in 2007; we co-taught it the first three times, and I ran it on
    my own the last three.

    Game industry trends, from both technological and business
    perspectives, demanded that the class be re-imagined to remain
    relevant in 2014. The dropping
    of “Multicore and” from the title — effectively
    refocusing the entire course on GPUs — responds
    to the trend of GPUs taking up
    more and more of the die on modern
    high-performance game platforms than CPUs,
    along with their increased flexibility
    compared with earlier dedicated graphics hardware.

    From a more practical standpoint,
    “Multicore and GPU Programming for Video Games”
    leaned heavily on Microsoft’s
    Game Studio

    platform, which provided a
    mechanism for students to deploy code
    on retail Xbox 360s, and potentially
    sell games on the
    Live Indie Games (XBLIG)

    service. At the start of 2013, Microsoft
    announced that it was sadly
    development of and support for XNA
    (although the “writing was on the wall” for
    some time before that),
    and XBLIG never
    off” the way,
    for instance, Apple’s iOS “App Store” did.
    Given the amount of material we had developed using XNA, I initially
    looked at
    but scrapped it when I was
    unable to readily
    get it running on my Mac.
    In searching for alternatives,
    Unity engine,
    the C4 Engine,
    Unreal Engine 4
    were found to have numerous appealing features.



    ((ECE2036 or CS1372) and ECE2031) or CS2110 or CS2261 or ECE3090 or ECE3035.
    (Reasoning behind the pre-requisites.)

    Course materials: Primarily lecture notes and literature. We may
    also draw material from textbooks such as:

    • Mathematics for 3D Game Programming and Computer Graphics, Third Edition, by
      Eric Lengyel,
      2011 (note: Lengyel is the creator of the C4 game engine).
    • Unity Shaders and Effects Cookbook, by Kenny Lammers, 2013
    • Game Engine Architecture, by Jason Gregory, 2009

    Grades will be based on several major,
    intensive programming projects, as well as a few
    smaller “warm-up” assignments
    designed to get students comfortable with various toolsets.
    There may also be a few
    short essay-type (around 1/2 page) questions and “pencil and paper” exercises.
    Various assignments may have different weights in the final grade calculation;
    these weights will be noted on each assignment.

    On some assignments, students signed up
    for the graduate section
    will have additional
    requirements beyond those signed up for the undergraduate section.
    They may also be required to perform additional assignments that are
    not assigned
    to the undergraduates.

    There will be no
    traditional paper-and-pencil exams,
    either midterms or finals.
    This is a class about programming;
    my philosophy is that any time you
    would spend taking or studying for such exams
    is better spent in front of a computer actually programming!

    Projects: Students will be undertake
    several projects to gain real programming
    in C# using the
    game engine (and maybe
    C++ within
    Engine 4

    and/or the
    C4 Engine
    ; details TBD),
    with emphasis on HLSL/Cg vertex and pixel
    shader programming and more general GPGPU
    programing using technologies such as DirectCompute
    and OpenGL compute shaders.
    In one of these assignments,
    students will code a basic graphics pipeline in a scripting language
    of their choice without
    the aid of a 3-D API (to ensure that they understand
    what a GPU does for them).

    Some of the assignments will be individual assignments;
    however, a few of them may provide the option
    to work in groups of two (such
    assignments may still be completed
    individually if you prefer to work alone).
    For variety,
    we will ask that you not
    partner with the same person on more than one assignment.
    This is a class on programming, so
    even on team assignments, you
    should feel confident that you could complete an assignment on your own if you needed to.

    A lot of the course will use C#,
    but we will not assume that you’ve seen C# before,
    and hence will present some introductory material on C#.
    (If you’ve seen Java, C++,
    or any other “curly-brace” language, you’ve pretty much seen them all,
    but there are some “gothchas” in moving from one to another.)

    On-line discussions: We will use
    to facilitate class discussions.
    I will try
    to check Piazza at least once a day. You are welcome to post questions about
    related to the course material, and also answer other student’s questions, as
    long as
    you don’t “give away the answer” or post chunks of code that are more than a
    few lines.

    T-square usage: In spite of its awfulness, we will use T-square for
    most homework submission and for posting grades, and occassionally for
    distributing resources for assignments.

    Office hours:
    Shortly before assignments are due, I will post an announcement on Piazza describing when and where I will be sure to be available for questions. This will tend to change slightly from week to week, so look for that announcement. (Also, if you walk by my office and happen to see my VL431 office door open, you are welcome to pop in with questions about the class and/or life in general.)

    Of course
    you are always welcome to e-mail me and we can set up a specific time to meet. Again, put “GPU” in your subject.

    Honor code:
    This course
    will be conducted under the rules and guidelines of the Georgia Tech Honor
    infractions will be reported to the Dean of Students.
    The “ground rules” for each assignment,
    which may vary from assignment to assignment, will be given in each
    assignment description. Please ask if any aspects of the given “ground
    rules” seem unclear.

    Backfile policy:
    Use of homework
    solutions and from previous versions of this class is forbidden.
    The material is highly complex,
    so it is extremely difficult to come up with 100%
    new project descriptions on each offering.
    Please be fair to students who may not have access to the same old materials.
    Detection of the use of backfiles will result in significant wrath.

    Major emergencies:
    If you have some sort of major life emergency – serious illness or injury,
    death in the family, house burns down or is flooded, etc. – that seriously
    impedes your progress in the class, please let me know as soon as possible
    so we can work something out.
    You will find professors can be quite reasonable if you keep us in the loop.
    Please don’t disappear with no warning half way through,
    making me think that you dropped the class, and then reappear out of
    nowhere the week before finals asking what you can do to make things up.
    (Yes, this has happened quite a bit, in both undergrad and grad classes.)

    On things that beep and blink:
    Please silence all cell phones and pagers
    before entering class. If you forget to do so and receive
    a call, please shut the noisemaking device down as quickly as possible,
    and return the person’s call after class. (Of course, there
    are reasonable
    exceptions, i.e. if your wife is in the 9th month of her pregnancy and may
    give birth at any moment or your
    kid isn’t feeling well but he or she
    went to school anyway and their school nurse
    may need to call you, leave your
    phone on vibrate, and answer it as quickly as possible and
    immediately step out
    of the room to handle the call.)

    In general,
    please do not instant message, websurf, Facebook (can I use it as a verb?),
    play games, etc. during class
    It can be quite distracting.
    Unless I say otherwise, the preferred position for laptops during
    class is in your backpack

    The Twitter exception:
    Lanterman says something particularly brilliant and clever during lecture,
    you are allowed to
    use your phone to Tweet it or post
    it to Facebook.

    Assorted relevant and fun links