Post Anti-Aliasing #2

hi
few months ago i'v read an article on intel research group called: Morphological Antialiasing.
this technique designed for cpu but with few tricks and hacks we can use it on gpu as well.
the algorithm consist of 3 main steps:
1. find discontinuities between pixels
2. identify predefined patterns
3. blend pixels in the neighborhood of the patterns in 2
1. simple edge detection on the image, using depth or color differences should do the work, keep in mind that you should encode edge type in you color channels so you could use it in 2, lets say red is horizontal edges, and g is vertical edges.
2. this is tricky, you basically need to identify few shapes: Z, U, L
see image below (grabbed from original intel article):

Reshetov A. 2009. Morphological Antialiasing. In Proceedings of High Performace Graphics

so based on the article, you only need to identify L shapes, as Z, U can be split into L shapes.
for more deep information please refer to the original article that can be found here:
http://visual-computing.intel-research.net/publications/publications.htm#Y2009

to identify L shapes there are few tricks, a simple one is to just follow the edges you mark in 1 and see if you get a match (few loops for each side: left/right/top/bottom and of course branching), if so you compute blend weights for these and continue to the next edges.
at the end, you end up with blend weights texture so you could blend the pixel to get the final image, you can use a the trick described in gpu pro 2, they encode the final weights in textures and sample it.
btw: if you have ATI HD 6850+ you have built in support for that, so no need to worry, for consoles you may want to worry a little ;)

this technique isn't simple to implement as a first shot, i tried few algorithms and techniques before i got this thing working.
after seeing the demo from gpu pro 2, i'v got to say i was impressed by the speed of their implementation so i put some tricks into mine as well to get the missing cycles :)

optimization tip: when doing edge detection pass, use the stencil buffer to mark these pixels, then in the next step, use the stencil apply your "massive" shapes/blend weights shader only on edges pixels, this way you won't waste power on irrelevant pixels

here is a few screenshots of the main steps and result:

Edges Detection with encoded edge type using image colors

Blend Weights

Without MLAA

With MLAA

Without MLAA

With MLAA

as you can see, this technique have very good result.
extra: few other techniques you should check:
* GPAA - show it at humus
* FXAA - nvidia sdk 11 (looks pretty good)

cya until next time...

Adding Vegetation

hi
one of the thing i always wanted to add is vegetation, tree, weeds, grass etc...
in a nutshell, to support all kind of vegetation you need to have few things:
1. some app to generate the content, plant model, textures etc.
2. engine supporting geometry instancing.
3. extra (depends of scene size), engine supporting model level of detail (LOD)
4. extra (depends of scene size), engine supporting good outdoor culling.
1. this is very critical, having great app for generating plants is a must, if the model and textures isn't quality enough, the best code won't do much... i checked some apps, and i want to tell you that if you have some $, speedtree is the way to go, checked it, love it...
2. if you are going to render plants, you won't render one tree with 1x1 meter of grass, you probably want to spread it all over a 1x1 km terrain, so you are going to render hundreds of the same plant with different properties or such, so you don't want to kill your gpu with 20000 draw calls... unless real time performance isn't an issue for you.
for me performance is critical so i'v implemented instancing support for each plant type.
3. if your scene is large enough you need to consider LOD support, there is not need to render full plant geometry from certain distance, you probably won't notice if its real geometry or simple billboard, but your gpu will, so consider replacing you full model with low model or even quad when distance to eye pos is large enough.
i added auto lod support in the engine and use it also for plants (maybe i will post on it next time)
4. if your scene is large enough and you have massive amount of plants, so you already know that its a good idea to cull your data.

anther thing, because we are using instancing on the same model again and again, its a better idea to break symmetry when rendering the plants.
you can place them with different rotation and apply random motion for each plant (in shader).
here is a screenshot of a test scene:

vegetation, sun shafts and dynamic atmosphere

cya until next time...

Lens Effect / Bokeh - Getting Into Details

hi
last time i talked about lens effect/bokeh, this time i talk about how to do it using FFT with information of how to implement it on the GPU.
i will not get into the theory of FFT, if you want to, check out this:
http://en.wikipedia.org/wiki/Fast_Fourier_transform

what i will explain here is how to do image processing using FFT and how to achieve a lens effects/bokeh filter.
if we have an image and we apply FFT on it we get this image in something called frequency domain.
after applying FFT every pixel in the image have its own frequency in the frequency domain that compose of a real and imaginary parts (a complex number)
after getting the image in the frequency domain, all we have to do is to decide which frequencies we want to "filter" out and then, apply the inverse FFT to get back to our image domain again, without the "filtered pixels"
that's the main idea.

nice thing with FFT is that its separable, what this means is that to apply FFT in 2D, we can simply apply 2 1D FFT, the first one goes on the rows (horizontal) and the second one goes on the columns (vertical) of the result.
another thing to know is that transforming N points can be done with two N/2 transforms, that is a dived and conquer approach, which helps to reuse computations.
so knowing that, we will use a divide and conquer algorithm called butterflies:
more info can be found here:
http://en.wikipedia.org/wiki/Butterfly_diagram
http://www.cmlab.csie.ntu.edu.tw/cml/dsp/training/coding/transform/fft.html

you can see really nice diagrams which helps to understand how it is works and how we can create and encode indices and weights for doing FFT on the GPU using textures and ping/pong method.

so if for example we have N=8 points to transform, we have log2(8) =3 steps of butterflies to perform, beginning with computation on four 2 points DFT, then two 4 points DFT and finally one 8 points DFT.

so, assuming you understand what the butterfly does, the steps to do it on the GPU is a matter of doing those steps:
1. create 2 textures (render targets) used for the ping/pong operations, those needed to be with high precision float point because the pixel on the frequency domain is a complex number with greater range than the pixel in the image spatial domain.
2. encode indices & weights into texture (also high precision)
3. do log2(Width) horizontal butterflies steps (using RT's from 1 and bf from 2)
4. do log2(Height) vertical butterflies steps (using RT's from 1 and bf from 2) on the result from 3
lets call those steps GPFFT
note: the log2 in steps 3,4 - this needs your image to with power of two size.
so how all of this nasty things helps us to do image processing on an image? and where is the bokeh stuff?
so here is the beauty:
suppose we have 2 images, one of them is our kernal, lets call it K, and the other is our source image that we wish to apply the effect on, lets call it I
apply GPFFT on K,I (transform them into frequency domain), multiply them together (complex multiply, remember we are using complex number in this domain), and then apply inverse GPFFT (to inverse transform the result) on the result to get a new I' convolved with K.
and that's it, all it matter is how your K looks like (the size and shape: triangle, square, octagon etc) and whats I.
as for bokeh/lens effect, basically it is working on bright pixels so simply filter the brighter pixels from your image with some threshold into other image (this is your I) and do the process on it.
this method works on low systems, DX9 with MRT support.
we can optimize the algorithm a lot using DX11 compute shader and get rid of the log(N) passes of steps 3,4 and perform only on pass each.
hope this post didn't make you headache' i'm starting to get one ;P
that's it for now
cya next time...

Lens Effect / Bokeh

hi
this time i'm attacking a problem related to lens blur effect or bokeh.
as you know (or don't) real time app such as games mimic lens effects with blur filters that applied on some image (scaled or not, depending or the app)
until now, blur could be done with few methods/filters such as gaussian, poisson disk, bilateral and etc depending on the problem you want to solve.
for example:
1. Gaussian could be used for blurring image that need a smooth look, like lighting and such...
2. Poisson disk blur could be used to smooth shadow mapping as it faster than 1 and basically produces nice result.
3. Bilateral blur could be for smoothing SSAO, because it can take into account few vars (like normal, depth) when doing the blur and reduce bleeding.
ok, those methods used until now and rely on samples count.
more samples means nicer result but takes a lot of power and most of the time it doesn't really worth it...
what we really need is a filter that doesn't rely on the kernal/filter size and mimic real camera lens effects without killing our app and can achieve this:

image taken from wiki under 'bokeh'
Coarse bokeh on a photo shot with an 85mm lens and 70mm aperture, which corresponds to f/1.2

the circle shapes is what we need to achieve! (or any other shape we want).
how we get this in reality: when we use smaller f in our camera, out of focus areas/points blurred into polygonal shape formed by the aperture blades.

somebody did this in real time or what? yes, Futuremark group in their new 3dmark 2011 benchmark (the underwater demo... amazing stuff...) and i think me 8p
lets see screenshots:

original image

increase filter size

and here is the hexagon shape...
note that any shape could be done with no cost!

and a video...

real time video of lens effect / bokeh

with the new gpu's features and dx11 we can do it in real time, right?
well... yes, but we don't really need dx11 or 10... ;p

what we really need the heavy duty guns! Fast Fourier Transform or FFT in short...
cya until next time...

Deferred Iterative Parallax Mapping

hi
the title name is a "bomb" but its nothing too fancy...
just added parallax mapping using iterative scheme, this method uses few iterations of parallax mapping in order to get more accuracy.
for those who doesn't familiar with parallax mapping, here is a short description:
1. you need two texture maps: height, normal map
2. start from current texcoord used to sample your diffuse map (for example) and offset it by the height value (scaled and biased) of the current pixel in the direction you view direction (in texture/tangent space)
3. sample your diffuse map with the new texcoord.
iterative parallax mapping does just like that, but for X iteration, every iteration tries to move one step closer to more accurate solution.
don't get me wrong, you are not going to get better solution than a ray traced methods (depends on you viewing angle, at some angles you can't even tell the difference), but this method cost almost nothing, especially if you can save your height map values in normal map alpha channel.
also num iteration can be changed depending on surface properties and such.
here is screenshots thats shows it in action using deferred method:

without parallax

with parallax (4 iterations)

OIT - Order Independent Transparency #3

hi

recently i needed to do OIT in DX9 HW and as you know (from my previous posts), i don't have the same access i have in DX11 (i can't create linked list per pixel and do the sorting after), so what i can do is simple depth peeling...
few words about depth peeling: depth peeling works by peel geometry with specific depth value, so to create OIT we can peel geometry starting from the nearest depth (to the camera eye) going to the farthest depth, each step we peel few pixels and store them to in separate RT, then we composite those RT's in back to front order to achieve the right transparency order effect.
so how do we implement that?
well, from what we can see we need color RT to store the result, also a depth buffer so we could compare pixels depth against, we also need a "second" depth buffer so we could reject pixels that already peeled/visited.
1. color RT supported by DX9 and upper
2. first depth buffer, also supported but...
3. second depth buffer, not supported but...
so we have issues with 2,3, to implement pixel depth compression we need to have the ability to pass depth buffer into the pixel shader so we could do the compare ourselves so we could reject already peeled/visited pixels, BUT DX9 doesn't even let us bind depth buffer as texture (even if i'm not writing to it - can be done in DX10, but we need DX9), so we need a way to emulate depth buffer that can be bind as texture and act as depth buffer.
to do this we can use float point RT, for the OIT we will use 2 float point RT, the first one used for first pass, second for the second pass, then the first again and so on, this is our ping/pong depth RT's.
the reason we need this kind of ping/pong RT's is to peel the correct layer every pass.
for example:
to peel the first layer we need empty depth buffer, the we render and store it to our color RT
to peel the second layer, we need to use the depth buffer from the first pass, so we could ignore the pixels handled at the first layer
to peel the third layer, we need to use the depth buffer from the second pass, so we could ignore the pixels handled at the second layer (and first layer)
and so on... you can see that we don't need more then 2 depth buffer to manage all those passes.
ok, so now we can peel layers with our depth buffer RT's, but how can do OIT for more than 8 layers? (we can only bind 8/16 texture), well here comes the reverse depth peeling (RDP) technique, instead of peeling in front to back order, peel with back to front order, so every time you peel one layer you can immediately composite it to the back buffer, so only one color buffer needed.
total memory needed could be one of those:
1. 3 RT's, two 32 bit float point and one RGBA
2. 2 RT's, one 32 bit float point for both depth buffer, and one RGBA
NOTE: in deferred rendering you can get rid of the RGBA buffer.
so as you can see, total memory is very low.
after all the talking, here comes a video that shows RDP in action:

simple planes shows the idea, this model doesn't need per pixel sorting but it shows that the algorithm works well.

Teapot model - needs per pixel sorting

NOTE: technique implemented in render monkey, the video shows very simple mesh containing 8 planes so it will be easy to see the effect.
each peeled layer render with small alpha value, so you can see how layers blended when you see through all layers (you get the white color)
some performance info: RM window 1272x856 ATI 5850 - 270+ FPS
thats it for now, cya...

Post Anti-Aliasing

hi
recently i'v added few post effects and i thought i need some AA solution to remove those jaggy edges.
traditional AA solution have its quality, no questions but it have some drawbacks:
1. deferred or semi deferred need special care and will need DX10+.
2. needs more memory
3. eat gpu power
i guess there is more but this is enough for me to consider some other solution, not the same quality but better than none.
so what i did is pretty simple:
1. run edge detection
2. run post AA which samples 4 pixels and use the value from 1 to know if its an edge or not, if its an edge i output the filtered pixel, if not i output none filtered pixel.
the heart of the algorithm is 1, what i did is taking few neighbors pixels and check for a big change in their depths/normals (for normals i check angles, for depth i check the gradients from center pixel)
again, very simple but i think it does the work, very fast and its generic for xbox,ps3,pc...
here is few screenshots that shows you the process

Without Post AA

Edge Detection

With Post AA