SIGGRAPH COURSE NOTES - Course 16

OpenKODE: An Open Mobile Media Development Environment

Monday, July 31, 2006
Half Day, 8:30 am - 12:15 pm
Level: Intermediate

Description
The Khronos Group has aggressively defined APIs and the supporting technology necessary to create an open, cross-platform, royalty-free development environment called OpenKODE. OpenKODE supports applications targeting mobile media devices. It includes APIs for 3D graphics, scalable vector graphics, video, and audio, as well as an open digital asset schema designed for interactive applications. This course provides an overview of the Khronos Group's efforts and how the technologies work together to create a cohesive environment for application development and deployment. This course features experts to provide technical overviews of each constituent technology: OpenGL ES 1.1, EGL, OpenGL ES Safety Critical, OpenGL ES 2.0, OpenVG, OpenMAX, OpenSL ES, and COLLADA. The session concludes with a developer's perspective to the OpenKODE environment. Attendees will come away from this course with an understanding of the OpenKODE environment and each of its constituent technologies. They will be armed with information on how to obtain the latest specifications, tutorials, examples, and demos and they will be ready to begin developing applications for the exploding mobile media market.

Final Presentation Materials
The material for these course notes was prepared well in advance of SIGGRAPH 2006. Look for the final presentation materails to be made available on the Khronos web site (www.khronos.org) shortly after SIGGRAPH. They will also be made available on the 3Dshaders web site (3dshaders.com).

Prerequisites
Programming experience in C or C++ and some graphics programming experience. Working knowledge of OpenGL and the mathematics of computer graphics is helpful. Experience with video or audio processing is helpful.

Intended Audience
Software developers who are targeting the mobile media market. Desktop or workstation application developers who need an introduction to developing for mobile media. Anyone who wants to learn about this exploding market.

Organizer
Randi Rost
Intel Corporation

Lecturers
Lars Bishop
NVIDIA Corporation

Chris Hall
Seaweed Systems

Robert Simpson
ATI Research

Dan Rice
Sun Microsystems, Inc.

Brian Murray
Freescale

Mark Barnes
Sony Computer Entertainment America

Timo Suoranta
Futuremark

Syllabus
0:15 Overview of the Khronos Group (Rost)
1:00 OpenGL ES v1.1, v2.0 and EGL (Bishop)
0:30 OpenGL ES 2.0 Programmable Pipeline (Simpson)
0:15 OpenGL ES Safety Critical (Hall)
0:15 Break
0:15 OpenVG (Rice)
0:15 OpenMAX (Murray)
0:15 COLLADA (Barnes)
0:30 Developer Perspectives (Suoranta)

Overview of the Khronos Group

Randi Rost -- Intel Corporation

The Khronos Group is a large and extremely successful standards organization whose mission is to define the APIs and technology required to support the authoring and playback of dynamic media on a wide variety of platforms and devices. Over the past several years, this organization has focused primarily on sorting out the issues to support developers in the wild and wooly mobile media market.

This presentation describes the organization of the Khronos Group, the current set of working groups, how the technologies fit together to create a cohesive platform for development and deployment, and how your organization can use these technologies and become involved in the effort.

Presentation Slides
 
Khronos Overview (.PDF, 4.58 MB)
OpenKODE Structure (.PDF, 245 KB)

OpenGL ES 1.1, 2.0 and EGL

Lars Bishop -- NVIDIA

With the growing popularity of 3D content on handheld and embedded platforms and the recent availability of powerful 3D hardware on these devices, developers required a flexible and powerful 3D rendering API that was designed for the unique nature of handheld platforms.  Because no existing API was directly applicable, the Khronos Group created OpenGL ES.  OpenGL ES provided an immediately recognizable architecture (i.e. based on OpenGL), pared down (and in some dimensions extended) to match the needs of both the hardware platforms and content developers.

This presentation will first discuss OpenGL ES version 1.1, contrasting it with “desktop” OpenGL 1.X.  This includes not only discussing the differences in the APIs themselves, but also some of the reasoning behind these OpenGL / OpenGL ES differences.  By discussing the reasoning behind these differences, the presentation seeks to explain some of the unique challenges inherent in current handheld platforms (as compared to desktop systems). The subsequent version of OpenGL ES (2.0) eliminates the fixed functionality of OpenGL ES 1.1 and replaces it with programmability controlled through the OpenGL Shading Language. A surface definition API called EGL will also be briefly discussed. EGL provides mechansims for creating and utilizing drawing surfaces for all of the relevant Khronos APIs..
Presentation Slides
 
GL ES and EGL (.PDF, 529 KB)
Specification Documents
 
OpenGL ES 1.1 Specification (.PDF, 392 KB)
OpenGL ES 1.1 Extension Pack Specification (.PDF, 392 KB)
OpenGL ES 2.0 Specfication (.PDF, 382 KB)
EGL Reference Pages (HTML)
Header Files
 
gl.h for GL ES 1.1 (.h file, 37 KB)
gl.h for GL ES 2.0 (.h file, 36 KB)
egl.h (.h file, 9 KB)

OpenGL ES 2.0 Programmable Pipeline

Robert Simpson -- ATI Research

Programmable graphics technology is coming to handheld device much more quickly than it came to desktop devices. With this technology comes the need for an API to expose the programmable functionality contained in the device. OpenGL ES and the OpenGL Shading Language were the logical starting points, but both required some changes to meet the needs of the mobile media market. Rather than support fixed function and programmability (as done in standard OpenGL 2.0), OpenGL ES has decided to do away with fixed functionality entirely, and do everything using programmability. This talk describes the key decisions made going from OpenGL ES 1.1 to OpenGL ES 2.0, as well as the key decisions made going from the OpenGL Shading Language in OpenGL 2.0 to the ES version of the language

Presentation Slides
 
GL ES Programmable (.PDF, 714 KB)
Specification Documents
 
OpenGL ES Language Specification ( PDF, 534 KB)

OpenGL ES Safety Critical

Chris Hall -- Seaweed Software

In certain market segments, government organizations have established certification requirements  for both hardware and software components. These certification requirements force limits on software complexity and contain regulations that dictate each phase of the software lifecycle. To support interactive 3D graphics in these safety critical markets, a reduced footprint version of OpenGL has been defined: OpenGL ES-SC.  This presentation covers the need for a safety critical OpenGL profile, discusses the decisions made to create this profile, and discusses the functionality it contains from a developer’s point of view.

Presentation Slides
 
GL ES Safety Critical (.PDF, 131 KB)
Specification Documents
 
OpenGL ES SC 1.0 Specification (.PDF, 392 KB)
OpenGL ES SC Philosophy (.PDF, 51 KB)
Header Files
 
es_sc_gl.h for GL ES SC 1.0 ( .h file,26 KB)

OpenVG

Dan Rice -- Sun

Many applications are based on rendering 2D vector graphics primitives such as Bezier curves. Yet, for desktop systems, 2D vector graphics have traditionally been rendered by the CPU rather than by the graphics accelerator. This approach is not effective on low-powered CPUs for mobile devices. To enable hardware accelerated vector graphics, the Khronos Group has defined an API call OpenVG. This API supports application development for portable mapping and GPS applications, e-book readers, and other applications that rely heavily on scalable vector graphics. OpenVG can be used to implement higher level vector graphics formats such as Flash, SVG, PDF, PostScript, vector fonts, and the like. This presentation provides a look at the API, the rendering pipeline, and the primitives that are supported in OpenVG.

Presentation Slides
 
OpenVG (.PDF, 2.02 MB)
Specification Documents
 
OpenVG 1.0 (.PDF, 1.52 MB)
Header Files
 
openvg.h ( .h file, 26 KB)
vgu.h ( .h file, 4 KB)

OpenMAX

Brian Murray --Freescale

This talk will cover the three layers of the OpenMAX API, which enable streamed multimedia application and codec portability across OS and hardware platforms.  An overview of the Application Layer, the Integration Layer, and the Development Layer APIs within the standard will be covered as well as examples of usage and interoperation.  Audience members should leave with a clear understanding of the OpenMAX APIs and how they relate to enabling multimedia capabilities on mobile devices.

Presentation Slides
 
OpenMAX (.PDF, 456 KB)
Specification Documents
 
OpenMAX DL 1.0 (.PDF,1.35 MB)
OpenMAX IL 1.0 (.PDF, 1.69 MB)

COLLADA

Mark Barnes --Sony Computer Entertainment America

COLLADA is a digital asset exchange schema that helps developers be more productive. It enables 3D authoring tools to work together to raise the power of the tool chains. It also defines a packaging format for content delivery, including shaders, animation, and physics. COLLADA enables leading 3D authoring tools to work effectively together to create OpenGL ES applications and assets. This presentation describes COLLADA: what it is, how it works, and who is supporting it.

Presentation Slides
 
COLLADA (.PDF, 719 KB)
Specification Documents
 
COLLADA Schema 1.4 (.XSD,376 KB)
 

Developer Perspectives

Timo Suoranta --Futuremark

OpenKODE sounds great on paper, but can real live developers actually use it? Will it solve portability problems? Will it make a developer’s life easier or more complicated? This course concludes with a discussion of portability and performance issues by a developer that is actively using the technologies that have been described previously. Conclusions are drawn about the current OpenKODE environment as well as the environment that is coming in the not-too-distant future.

Presentation Slides
 
Developer Perspectives (.PDF, 719 KB)