Happy New Year!

May GPU programming be with you in 2014!

My renderer vs Sketchup

I am currently working on a Sketchup plugin that will allow interactive visualization of 3D objects using ray and path tracing rendering techniques. This will be the first Oculus Rift compatible visualizer based on an interactive ray-tracing engine.
For now, the demo is currently only compatible with nVidia graphics cards supporting the CUDA technology.

Download the Windows x64 installer for CPU and GPU (Cuda) tech demos

 

Path Tracing Experiments

Latest Acceleration Structures

Tribute to GTC 2012 based on SoL-R Raytracer

Tech Demo #3 based of the SoL-R rendering engine. Humble tribute to nVidia GTC demo, back in 2012.

Metaballs tech demo based on the SoL-R engine

OpenGL to Ray-Tracing

OpenGL to Ray-Tracing by changing two lines of your code:

#ifdef USE_SOLR
   // Use raytracing engine
   #include <OpenGL/rtgl.h>
   using namespace RayTracer; 
#else
   // Use OpenGL
   #include <GL/gl.h> /* Header File For The OpenGL Library */
   #include <GL/glu.h> /* Header File For The GLU Library */
#endif // USE_SOLR

SoL-R overrides the OpenGL commands and replaces the default rasterization engine with its interactive ray-tracer. It's as simple as that.

SoL-R is under development but you can already like it on Facebook. Contact me for details.

CUDA Pro Tip: Generate Custom Application Profile Timelines with NVTX

A great article on how to analyse and optimize your CUDA code, using the nVidia tools extension (NVTX).

https://developer.nvidia.com/content/cuda-pro-tip-generate-custom-application-profile-timelines-nvtx

First ever fully ray-traced photo visualizer for Oculus

Got a CUDA card? an Oculus Rift? a Kinect? Download the first ever immersive photo visualizer, fully ray-traced. Then drop a few jpg file in the photos subfolder and enjoy a unique experience!

Download link: https://www.dropbox.com/s/3zjqs2deu0a4gee/VRGallery.zip

Cuda, Kinect and Oculus rift, what a team!

I am finally getting to what I have been thinking about for the last few months, or maybe years: A fully ray-traced virtual and immersive environment. In order to achieve that project, the following technologies were able to fulfill my requirements:

• Oculus rift for immersive 3D visualization and head orientation
• Kinect for the position of the actor in the real 3D space
• CUDA for making it possible to achieve real-time ray-tracing on a consumer PC.
• WiiMotes for hands orientation and object manipulation

The application is written in C++ (No more than a few hundred lines of code), and is currently supported on Windows x64 platforms, mainly because of the Kinect device. A partial implementation also is available on Linux.

Download the Windows demo and judge by yourself :-)

Interactive Ray-tracing and Oculus Rift

That's it, I found an Oculus rift in my mailbox 2 days ago and since then, my social life sort of isn't the same anymore ;-) After getting sick after an endless HL2 session, I decided that it was time to initiate more serious stuff. I you read my blog, you already know that I was really, really waiting for that rift, and despite the amazing weather outside, I spent another afternoon integrating that new device with my interactive CUDA raytracer. And here is the result:

Device initialization and  destruction was as simple as that:
void Scene::initializeOVR()
{
   HMDInfo Info;
   bool InfoLoaded;
   OVR::System::Init();
      m_manager = *OVR::DeviceManager::Create();
   m_HMD = *m_manager->EnumerateDevices<HMDDevice>().CreateDevice();
   if (m_HMD)
   {
      InfoLoaded = m_HMD->GetDeviceInfo(&Info);
      m_sensor = *m_HMD->GetSensor();
   }
   else
   {
      m_sensor = *m_manager->EnumerateDevices<SensorDevice>().CreateDevice();
   }
   if(m_sensor)
   {
      m_sensorFusion.AttachToSensor(m_sensor);
   }
}

void Scene::finalizeOVR()
{
m_sensor.Clear();
   m_HMD.Clear();
m_manager.Clear();
System::Destroy();
}

And getting the orientation of the drift is as difficult as that:
if ( m_sensorFusion.IsAttachedToSensor() )
{
  OVR::Quatf orientation = m_sensorFusion.GetOrientation(); 
  m_viewAngles.x = orientation.x;
  m_viewAngles.y = orientation.y;
  m_viewAngles.z = orientation.z;
}

m_gpuKernel->setCamera( m_viewPos, m_viewDir, m_viewAngles );

Splitting the screen for both eyes was already implemented in my engine, originaly for 3D vision, but I have to admin that Oculus is way more convincing.

More to come...

Animation and morphing finally added to ray-tracing engine

I extracted animation models from 3D studio Max using the following script:

sliderTime  = 0

for t = 0 to 198 do (

  local newName = "E:\\Deployments\\bin\\obj\\Animation\\InPlace\\InPlace_" + (t as string) + ".obj"

  exportFile newName #noPrompt selectedOnly:false using:ObjExp

  sliderTime += 1

)

It's pretty basic, but it does the job. The ray-tracing engine now supports a new concept: frames. The idea is to easily switch from on scene to another. Bounding boxes from each frame are pre-processed in order to produce optimal rendering.

Texture mapping in real-time ray-tracing engine teaser

From ray-tracing to smashing galaxies, a memory management issue - Part I

Ray-tracing vs Rasterization 

Rasterization: The term refers to the popular rendering algorithm for displaying three-dimensional shapes on a computer. Rasterization is currently the most popular technique for producing real-time 3D computer graphics. Real-time applications need to respond immediately to user input, and generally need to produce frame rates of at least 24 frames per second to achieve smooth animation. This is used in Games, CAD, medical imaging, etc. Any application that needs interactive rendering of complex 3D scenes.

 

Ray-tracing: Ray-tracing is a technique for generating an image by tracing the path of light through pixels in an image plane and simulating the effects of its encounters with virtual objects. It is meant to produce much better quality images than rasterization but it’s also much slower because it’s much more CPU consuming.

Appetite for high quality visualization

Why do we need real-time and interactive visualization?

I don’t know if you know about that story but there was an experiment done by, I think, the American intelligence. They were asked to write an application that could quickly identify buildings on satellite pictures. So they started working on complex algorithms involving machine learning, neural networks, image processing and anything that can help identifying the shape of a building on a 2D image. But then they realized that, with brain analysis and electrodes, using humans was a much simpler and cheaper way to achieve what they wanted. So they hired people for a few dollars an hours, and as they were showing them the photos, some parts of their brain were activated when they could identify a building. And this was done in a matter of milliseconds. The human brain is just good at that and has a very optimized recognition algorithm. All this to say that our unconscious is very good at picking things up and can understand more when facing realistic images.

Also, our brain understands dynamic scenes better than static ones, and because the quality of what we see directly influences our comprehension of complex events, it has to be as high as possible. 

Where is the challenge?

With Rasterization, every object is independent from each other. It’s very suitable for embarrassingly parallel algorithms and has a linear and predictable processing time,  depending on the number of objects to process.

With Ray-tracing, every object potentially interacts with all of the others. So what you get is an exponential processing time, as the number of objects increases.

Solving this problem requires a specific management of the objects to be rendered, and this is what I am going to show you now.

Six degrees of separation

One day, a friend told me about the theory of “Six degrees of separation”.  Do you know about this theory?

Six degrees of separation is the theory that everyone and everything is six or fewer steps away, by way of introduction, from any other person in the world, so that a chain of "a friend of a friend" statements can be made to connect any two people in a maximum of six steps

I did not take this sentence very seriously at the time but for some reason, it got stuck in my head. And a few years later, as I was encountering performance problems on one of my project, that theory came back to me as an evidence.

If we are 7 billion people on this planet, and if we never are more than 6 steps away from each other, why would it be different in the memory of my computer? If I have 7 billion objects to process in my rendering algorithm, it should not take more than 6 cycles to access the object that I need to work on…

Ok, that’s the theory. How do we do that in practice? And this is of course depending on the parameters that you can pass to this clever equation, as shown on the slide.

Bounding boxes

In order to apply this algorithm, the first thing to do is to group your objects, so that the group can be considered rather than the objects that it contains. In this example, we talk about “Bounding boxes”.

A bounding box is basically a 3D space that contains a number of objects.

Ray-tracing is all about intersections. In the previous slides, I told you that when you launch a ray, you need to find the intersection with the closest object. If you have 10’000 objects, then you need to compute the intersection between the ray and the object itself, for all of the objects,  and then keep the closest intersection. By grouping objects in boxes, you can start with computing intersection between the ray and the boxes, and if there is an intersection with a box, only then you will look into that box and compute the intersections with the objects it contains.

Fine tuning

A box intersection takes 17 cycles, using the robust and efficient algorithm  provided by the University of Utah ( http://people.csail.mit.edu/amy/papers/box-jgt.pdf )

A object intersection is a lot more expensive and can cost several hundred cycles. So then it’s a trade-off to identify between the number of boxes, and the number of objects.

OO vs Algorithm Specific Structure

Object Oriented Design is good, because it uses concepts that we, humans, are comfortable with.  A car object is a set of wheels, tyres, seats, and it as a color, etc. You can define inheritance that prevents from replicating the code.

With OO, you can write readable and maintainable code, which is essential. But OO is not optimal for performance because it fragments memory and forces one to consider an object systematically as a whole.

If you take the example of a person with three attributes: a first name, a second name, and an age. Now imagine that you have a collection of persons and you want to compute the average age of that population. With OO, you will not only take the age attribute down to the processor but you will also take the first and the second name down to the CPU, even if you do not need them in your algorithm. Only because OO forces you do so, by design.

On the other hand, if you have an algorithm specific data structure, you will separate you “person” object in two parts. One for the name and family name,  the other one for the age. And only the data structure containing the age will be taken down to the processor. This will dramatically increase the performance.

In the case of the ray-tracer, we do exactly the same.  A primitive such as a sphere for example, is composed of a number of attributes such as the center, the radius, the material properties, texture coordinates, etc. But when processing intersection on the first pass of the algorithm, only the center and the radius are needed, and this can be represented in the form of a float4.

For the boxes, only the two corners of the box are needed, meaning two float3.

Order in RAM

Now that we have defined our data structures, this needs to be optimized them for computers.

In our case, we have boxes and we need to compute the ray-box intersection for each of them. So let’s make sure that boxes are aligned in memory.

As we go through our boxes, if an intersection is found, we need to get deeper into the box to make sure that there is an intersection with a primitive. Meaning that ordering primitives also is essential. This is what is being done at a second level. And as explained in the previous slide, only me minimal information is taken down to the processor.

Raytracer getting ready for Oculus Rift

The ray-tracing engine is now capable of generating 360° panorama images interactively. Making it ready to interact with Oculus Rift virtual headset using a simple textured sphere in the default DirectX 3D scene.

Interactive Raytracing on Android mobile device

Last week, I got my first Android phone: A Wiko Cink King! It's not a Samsung, it's not an Apple, but it's definitely worth the 200 EUR it cost me. And since I now have an Android device, I can develop my own apps, and my first implementation had to be related to ray-tracing in some way. Let's be honest, interactive ray-tracing on a 200 EUR device... the dual-core 1GHz CPU simply cannot handle it. But there is another solution: Why not use the GTX 480 sitting in my desktop box remotely? And since I already have an HTTP GPU server in stock, writing the mobile app making HTTP REST calls did not take more that a few hours, using the really cool Android Development Kit.


Sample code for the GPU HTTP server: https://github.com/cudaopencl/GPUHTTPServer
Android SDK: http://developer.android.com/sdk/index.html

APK downloadable from github, but since this needs a GPU server, email-me for a demo!

https://github.com/cudaopencl/CudaRuntimes/tree/master/CudaRaytracer

Skybox implemented. Multi-GPU implementation validated on GTX 690

 

 

 

 

Interactive and Raytraced 3D Model Editor

Screenshots:

Interactive Raytracer now has support for multi-GPU :-)

Complex Geometry and Interactive Raytracing

A few screenshots and a video of the interactive CUDA raytracer with added support for OBJ files.

It took me a while to complete the algorithm that extrapolates normals in a triangle. The trick is to compute the area of each triangle defines by the intersection point and the three vertices. Once you get that, just find the weighted mean, and you're done. Here is the code:

float4 v0 = triangle.p0 - intersection;

float4 v1 = triangle.p1 - intersection;

float4 v2 = triangle.p2 - intersection;

float a0 = 0.5f*vectorLength(crossProduct( v1,v2 ));

float a1 = 0.5f*vectorLength(crossProduct( v0,v2 ));

float a2 = 0.5f*vectorLength(crossProduct( v0,v1 ));

normal = (triangle.n0*a0 + triangle.n1*a1 + triangle.n2*a2)/(a0+a1+a2);

As simple as that :-)