GPU Programming for Video Games (2016)

GPU Programming for Video Games (2016)
<!–
–>

Summer 2016

News

  • HW 6 posted.

Assignments

Schedule

The shader programs in the slides and unitypackages below
are intended to
illustrate various aspects of shader programming; they are not necessarily
the most efficient or congruent with Unity’s workflow.

–>
<!–

  • Videos: Ian
    Bogost on The Colbert Report    ;
    Prince Caspian
    pre-viz
  • 5/15: Session 2 – History of gaming hardware
    (PDF slides,
    PDF slides 4-up)

    –>

  • 5/24: Session 3 – 3D Coordinates & Transformations
    (PDF slides,
    PDF slides 4-up)
  • 5/26: Session 4 – 3D to 2D Projection
    (PDF slides,
    PDF slides 4-up)
  • 5/31: Session 5 – Classic Lighting & Rasterization
    (PDF slides,
    PDF slides 4-up)
  • 6/2: Session 6 – Texturing & Blending
    (PDF slides,
    PDF slides 4-up)
  • 6/7: Session 7 – GPUs – Under the Hood
    (PDF slides,
    PDF slides 4-up)
  • 6/9: Session 8 – Materials in Unity
    (PDF slides,
    PDF slides 4-up)
    (this lecture will mostly consist of a live
    demonstration of Unity’s workflow; the slides are for a side discussion)

    • Vertex vs. pixel
      shading demo      : Create a new Unity project and drag the unitypackage
      file onto the Asset folder to import it, double click on the “Barrels VvP
      Lighting Demo” in the Assets folder (it will look like the main Unity
      application icon) to load that paticular scene, and then click
      the “Run” arrow button to run it.
      Number keys 0 through 5 change the speed
      of rotation, and the x, y, and z keys toggle on and off rotations around those
      axes.
  • 6/14: Session 9 – Introduction to Shader Languages, Part 1
    (PDF slides,
    PDF slides 4-up)

    • GPU16
      Introductory Shaders      :
      Run “GPUXXIntroShadersScene” to see two spinning treasure chests
      (model
      by Mobile Design Lab      ,
      free on Unity Asset store), along with some other spinning objects. There
      is an “legacy” version and a “GPUXX” version of each of the materials,
      so you can experiment with comparing different shaders and textures.
      I replaced the treasure chest normal map with a normal
      map based on the Georgia Tech
      logo; you can put the treasure chest normal map back in to see something
      more subtle.
      There is also a non-visible game object
      that is just a host for a script I wrote to create some matrices and
      multiply them in various ways. I used this when I was trying to figure out
      what conventions Unity uses, since the official documentation is sometimes
      contradictory
      and confusing.
  • 6/16: Session 10 – Introduction to Shader Languages, Part 2
  • 6/21: Session 11 – Environment Mapping
    (PDF slides,
    PDF slides 4-up)

    • GPU16 Environment Mapping
      Demo      : This contains three scenes. One compares the “original Phong”
      (displayed as red) and “Phong-Blinn” (displayed as blue)
      specular models, so pixels where the two models give similar results
      appear as purple. By adjusting the powers, you can get them to match pretty
      closely. The other two scenes illustrate environment mapping, one with
      per-vertex and one with per-pixel computations. A “crossfade” slider lets
      you blend reflections and refractions. The per-vertex version shows
      clear triangulation artifacts, so I didn’t bother implementing Fresnel
      effects for the per-vertex shader, but I did for the per-pixel shader.
      Depending on which “return” statement in the per-pixel shader is in effect
      (comment out the other one), you can use either the original crossfade slider,
      or play with the Fresnel parameters. (The
      horse
      model and animations      , free
      on the Unity Asset store, are by
      Dootsy Development. The
      basilica
      cube map       is by Humus.)
  • 6/23: Session 12 – Projective Textures & Shadow Mapping
    (PDF slides,
    PDF slides 4-up)

  • 6/28: Session 13 – Introduction to Surface Shaders, Part 1
    (PDF slides,
    PDF slides 4-up)

    • GPU16 Surface Shaders
      Demo      : Run the “Custom comparison scene.”
      This demo was created basically to double-check that one can
      re-create a few of the built-in shaders using “custom” lighting models.
  • 6/30: Session 14 – Introduction to Surface Shaders, Part 2
  • 7/7: Session 15 – Surface Shaders with Vertex Modification
    (PDF slides,
    PDF slides 4-up)

    • GPU16
      Vertex Modification Demo      :
      Run “GPUXXVertexModScene.” One of the squares has a 100×100
      tessellation; the other, which often demonstrates spatial aliasing, has
      a 10×10 tessilation. A custom material inspector lets you use a checkbox
      to switch between planar and circular waves. The “direction” setting is
      ignored if circular waves are active.
  • 7/12: Session 16 – Physically Based Rendering, Part 1
    (PDF slides,
    PDF slides 4-up)

  • 7/14: Session 17 – Physically Based Rendering, Part 2
  • 7/26: Session 18 – Postprocessing & Deferred Rendering
    (PDF slides,
    PDF slides 4-up)

    <!–

  • 6/26: Session 14 – Case Study:
    Portalarium’s Multi-Shader (with
    Shroud of the Avatar
    pre-alpha demo)
  • 7/8: Session 16 – Postprocessing (demo to be posted)
  • 7/10: Session 17 – G-buffers
    • Run the “GPUDepthNormalsTest” scene in the Deferred Rendering
      unitypackage from Session 18 below.
  • 7/17: Session 19 – Advanced Post Processing with G-Buffers
    • GPU14
      Advanced Post Processing Demo      :
      Make sure “process” in the AdvPostProc component attached to the Main
      Camera is checked, then
      experiment with the “return” lines in the GPUXXFancyPostProcTest.shader.

    –>

    Details and descriptions

    This webpage constitutes the class syllabus.

    Course description: (2 hours lecture, 3 hours unscheduled lab,
    3 credit hours total) 3-D graphics pipelines. Real-time simulation
    concerns. Game engine architectures.
    GPU architectures. Graphics APIs. Vertex and pixel shader programming.
    Post-processing effects. Deferred rendering.

    Course objective: This class provides the GPU programming
    concepts needed to meet timely demands of the multimedia,
    visualization, and gaming industries.
    The course also bridges the gap between the
    current generic computer
    architecture courses and the video game design courses offered by
    the College of Computing,
    the School of Electrical and
    Computer Engineering    , and the
    School of Literature, Media, and Culture.
    The class covers state-of-the-art general purpose graphical processing unit architectures (GPGPUs) from application and hardware design perspectives. The course considers programming models using examples from the algorithmic needs of modern 3-D games.

    Tentative topics

    The topics and
    the order in which they are covered are subject to change.

    1. Introduction and historical context
    2. Classic 3-D rendering pipeline: geometry transformation,
      lighting, texturing

     

  • Overview of 3-D APIs
  • Simulation loops and game engine components
  • Object-oriented game engines
  • GPU architectures and GPU assembly code
  • Introduction to shading languages (HLSL/Cg), vertex and pixel shaders
  • Per-pixel vs. per-vertex lighting
  • Advanced 3-D shading effects (ex: bump mapping, environment mapping)
  • Physically-based rendering (Cook-Torrance BRDFs, linear vs. gamma space
    lighting)
  • Deferred rendering
  • Screen space techniques (ex: ambient occlusion)
  • Real-time shadow techniques
  • Profiling and performance analysis
  • Adapting to the limitations of hand-held devices
  • Possibly occasional guest speakers from industry
    (previous speakers)Note that the course
    does not cover OpenCL or CUDA, and unlike its
    predecessor, “Multicore and GPU Programming for Video Games,”
    does not cover multicore CPU programming.
    OpenCL, CUDA, and general
    multicore programming are well covered in many other classes in ECE and CoC,
    whereas the use of GPU architectures
    for their native application of computer graphics
    is not extensively
    covered in many other classes, either here at Georgia Tech or other schools.

    Administrivia

    Instructor: Aaron Lanterman, office Van Leer 431,
    e-mail lanterma@ece.gatech.edu. E-mail is the best way to reach me;
    please put GPU in the subject line so I can find your e-mails easily.

    Prereqs:
    ECE2035 or ECE2036 or CS2110 or CS2261 or ECE3035 or ECE3090
    (Reasoning behind the pre-requisites.)

    Course materials: Primarily lecture notes and literature. We may
    also draw material from textbooks such as:

    • Real-Time Rendering, Third Edition, by
      Tomas Akenine-Moller, Eric Hanes, and Naty Hoffman.
    • Mathematics for 3D Game Programming and Computer Graphics, Third Edition, by
      Eric Lengyel,
      2011 (note: Lengyel is the creator of the C4 game engine).
    • Unity Shaders and Effects Cookbook, by Kenny Lammers, 2013
    • Game Engine Architecture, by Jason Gregory, 2009

    Assessment:
    Grades will be based on several major,
    intensive programming projects, as well as a few
    smaller “warm-up” assignments
    designed to get students comfortable with various toolsets.
    There may also be a few
    short essay-type (around 1/2 page) questions and “pencil and paper” exercises.
    Various assignments may have different weights in the final grade calculation;
    these weights will be noted on each assignment.

    There will be no
    traditional paper-and-pencil exams,
    either midterms or finals.
    This is a class about programming;
    my philosophy is that any time you
    would spend taking or studying for such exams
    is better spent in front of a computer actually programming!

    Projects: Students will be undertake
    several projects to gain real programming
    experience in HLSL/Cg vertex and pixel shader programming,
    using
    Unity
    as a convenient game engine for testing shader code.
    In one of these assignments,
    students will code a basic graphics pipeline in a scripting language
    of their choice without
    the aid of a 3-D API (to ensure that they understand
    what a GPU does for them).

    <!–
    Some of the assignments will be individual assignments;
    however, a few of them may provide the option
    to work in groups of two (such
    assignments may still be completed
    individually if you prefer to work alone).     For variety,
    we will ask that you not
    partner with the same person on more than one assignment.
    This is a class on programming, so
    even on team assignments, you
    should feel confident that you could complete an assignment on your own if you needed to.
    –>

    A lot of the course will use C#,
    but we will not assume that you’ve seen C# before.
    (If you’ve seen Java, C++,
    or any other “curly-brace” language, you’ve pretty much seen them all,
    but there are some “gothchas” in moving from one to another that we will
    warn you about.)

    On-line discussions: We will use
    Piazza
    to facilitate class discussions.
    I will try
    to check Piazza at least once a day. You are welcome to post questions about
    anything
    related to the course material, and also answer other student’s questions, as
    long as
    you don’t “give away the answer” or post chunks of code that are more than a
    few lines.

    T-square usage: In spite of its awfulness, we will use T-square for
    most homework submission and for posting grades, and occassionally for
    distributing resources for assignments. We will also use it for posting
    announcements.

    Office hours:
    Shortly before assignments are due, I will post an announcement describing when and where I will be sure to be available for questions. This will tend to change slightly from week to week, so look for that announcement. (Also, if you walk by my office and happen to see my VL431 office door open, you are welcome to pop in with questions about the class and/or life in general.)

    Of course
    you are always welcome to e-mail me and we can set up a specific time to meet. Again, put “GPU” in your subject.

    Honor code:
    This course
    will be conducted under the rules and guidelines of the Georgia Tech Honor
    Code;
    infractions will be reported to the Dean of Students.
    The “ground rules” for each assignment,
    which may vary from assignment to assignment, will be given in each
    assignment description. Please ask if any aspects of the given “ground
    rules” seem unclear.

    Backfile policy:
    Use of homework
    solutions and from previous versions of this class is forbidden.
    The material is highly complex,
    so it is extremely difficult to come up with 100%
    new project descriptions on each offering.
    Please be fair to students who may not have access to the same old materials.
    Detection of the use of backfiles will result in significant wrath.

    Major emergencies:
    If you have some sort of major life emergency – serious illness or injury,
    death in the family, house burns down or is flooded, etc. – that seriously
    impedes your progress in the class, please let me know as soon as possible
    so we can work something out.
    You will find professors can be quite reasonable if you keep us in the loop.
    Please don’t disappear with no warning half way through,
    making me think that you dropped the class, and then reappear out of
    nowhere the week before finals asking what you can do to make things up.
    (Yes, this has happened quite a bit, in both undergrad and grad classes.)

    On things that beep and blink:
    Please silence all cell phones and pagers
    before entering class. If you forget to do so and receive
    a call, please shut the noisemaking device down as quickly as possible,
    and return the person’s call after class. (Of course, there
    are reasonable
    exceptions, i.e. if your
    kid isn’t feeling well but he or she
    went to school anyway and their school nurse
    may need to call you, leave your
    phone on vibrate, and answer it as quickly as possible and
    immediately step out
    of the room to handle the call.)

    In general,
    please do not instant message, websurf, Facebook (can I use it as a verb?),
    e-mail,
    play games, etc. during class    .
    It can be quite distracting.
    Unless I say otherwise, the preferred position for laptops during
    class is in your backpack    .

    The Twitter exception:
    If
    Prof.
    Lanterman says something particularly brilliant and clever during lecture,
    you are allowed to
    use your phone to Tweet it or post
    it to Facebook.

    Assorted relevant and fun links