Sunday, 28 August 2016

Hello Unity, AIR I still love you.

It's been quite a slow summer workwise, so I've had plenty time on my hands.
I've been toying with an idea to create a "digital pop-up book", that is to say a book that comes complete with a google cardboard headset, and combines a story with Augmented and Virtual Reality.

I created the first version in AIR, and did manage to get everything working. However, performance was not great. I was using four Stage3D Proxies, which worked fine on my own phone, an Oppo 7 Find, but on smaller phones some levels would disappear for no apparent reason, though I suspect there wasn't enough VRAM available. The Oppo has a dedicated graphics chip, and can easily fool you into thinking everything's pukka.

So, time to look at alternatives. I've messed with Unity since the very early days, when I actually talked with David Helgason, one of the founders, and got a free license. The whole thing started in Denmark, and I was a member of the Web3D Consortium, still trying to make solutions with X3D. At this time Flash ruled supreme, so there was no point (for me) to learn Unity. I'd made a javascript bridge between Flux Player (a then open-source X3D engine) and Flash Player, and was having a whale of a time programming in ActionScript and seeing results in 3D. Made a few good solutions on that basis, before Flux player suddenly became a closed technology, and carried a very hefty license fee.

Just when I thought I would have to drop Flash, in order to carry on working with Web3D, Stage3D came along and I've rarely looked back since. But it's time to look forward.

Now, as then, when I open Unity I can't help feeling I've gone back to the Flash MX days.
Attaching scripts to actual graphical objects the way we used to, each MovieClip having an onClipEvent(load) and onClipEvent(enterFrame). I wonder if that was the inspiration for Unity's Start(), Update() template?
In Unity the MovieClip is loosely replaced by the GameObject. That's another thing that put me off Unity. It assumes you're making a game, which generally I'm not.

Unity allows you to program in JavaScript or C#. I was tempted to go with the JavaScript approach, but that seemed like a step backwards after all that AS3 OOP goodness, so I've learned to use C#.
The transition has been completely painless, for apart from slight differences in syntax, everything is quite familiar.

I'm used to sitting with pages of code in FlashDevelop, and only seeing my 3D creations after compiling. I actually wanted a similar workflow in Unity, though now I am getting used to the way Unity works, and the ability to create structure, without necessarily having a "Main" class.
(I found out how to do that too, but don't need my comfort blanket quite as much any more :)
In Unity you do set a lot of variable and properties through the User Interface, but the user interface is created from your own code, so basically if you work the way I do, you are building a custom user interface for Editing and tweeking your application as you go along. This is actually a very cool approach. I just never got that far in my previous brief attempts to master Unity.

Once you discover the power of a full blown 3D engine, it's easy to go mad.
As I work primarily on Mobile and tablet, I have had to tone it down a bit, but it's fantastic to be able to work in a WYSIWYG environment again, the way we used to in the early Flash years.

Visual Studio is a pleasure to code in. Once I got the intellisense working, I've been able to use it to discover all the properties and methods available on different types of objects and I'm now feeling just as at home as in FlashDevelop.

Unlike most Flashers, I'm not quitting AS3 to take up a new technology, just adding to my toolbox.
Now I'm feeling comfortable with C# a whole new world has opened up and I'm excited to see where it takes me.

My first experiments have been in "Mixed Reality", that is a combination of Virtual and Augmented reality using the Vuforia library. Sometimes it's so easy it feels like cheating. While ActionScript has always been a challenging platform to master (and therefore in itself very satisfying when getting something to work!!), the combination of Unity and C# provides a powerhouse of new possibilities, as all the simple stuff is taken care of. I'm not used to anything being "under the hood", and even libraries like Away3D, I've customised and modified for my own ends.

Though I've had to go from Flash Hero to Unity Zero, I'm quickly clawing my way up to where I can call myself a Unity Developer.

Thursday, 20 March 2014

Further Adventures in AGAL

I'll be honest, when I first saw AGAL some years ago, I just knew it was too obtuse and complicated for me. I was sure they'd make it easier in the next release, but 3 years later I'm still waiting.

Time to take the plunge. There's loads of great articles about AGAL and how to set it all up. It requires a ton of boiler plate AS3 code before you even get to the shader programming, but that part is at least just about identical every time so after a couple hours of wading through it line by line, you don't need to think about it any more.

I've made shaders in PixelBender, so the concept isn't entirely new to me, but in pixel bender you write a routine which is executed on each pixel. In AGAL you write a routine that is executed first on every vertex, and then on every pixel (or fragment) between them.

One of the really annoying things is the way you have to declare any variables you're going to use outside of the actual program. So you end up hopping around between shader code and AS3. The next hurdle is the actual names of the variables, as you don't get to name them yourself. vc0 for example is a vertex constant in register 0, but rather than assigning something to vc0 (vc0 = would have been nice!) you would write something like..

context3D.setProgramConstantsFromVector(Context3DProgramType.VERTEX, 4, sway, -1);

Believe it or not, that has assigned a vector ("sway") to vc0.
As they say in the Matrix, "after a while you don't even see the code". I wish!
With all the register indexes and two letter names it is very easy to make mistakes.

I'm not going to attempt to explain the mechanics, as it's been done to death and way better than I could.
I recommend starting with and if you can get hold of this book, all the better.

And this is your Bible:

That will get you up and running, but then what? There's simply tons of articles about shader programming out there, but precious few about AGAL specifically.
For a bit of inspiration I recommend a trip to, but be warned, your browser will probably crash a few times, and your laptop will melt. These are GLSL shaders for WebGL, so while they may give some ideas it's far from obvious how to implement them in AGAL, and once you get past the basics of Stage3D there's not much help around.

An excellent free resource is the GPU Gems series, hosted at Nvidia.
This evening's experiment has been implementing this little ditty, animated grass.
These books are great, as they explain the concepts, rather than the code.

One of the most beautiful sites you'll ever see in Denmark is a stiff sea breeze causing waves in a ripe field of barley. That was what I decided I'd try to emulate.

As I was working with an example that had different Blend modes, I left them in (After clicking on the window use "B" to switch). I also wanted to explore and understand the mip mapping modes, so you can go through all those by pressing SPACE. Oh, and to prove it's not baked, you can use your mouse to effect the grass too.

Frame rate varies wildly. There are exactly 65536 polygons in this example, all being animated independently in groups of 16 (i.e. a single blade of barley).
There are definitely more efficient ways, but it runs at 60fps on my laptop. I can see online its not quite so good. 
What kills performance here is not the shader, but the fact I'm reading in Perlin noise from a BitmapData animated in Flash for every cycle. I'm sure there's a better way, but it's late enough as it is...


Saturday, 15 March 2014

MatCap in AGAL

Carrying on from the previous posts,  as an exercise in AGAL I've tried implementing MatCap with Normal map in pure AGAL, based on an excellent example by Norbz

Source including maps and FlashDevelop project

Thursday, 6 March 2014

MatCap with NormalMapping

What do you get if you cross this:

With this...

No? Well a glass like complex material that doesn't require any lighting!

Continuing on from the previous post, I've now implemented normal maps into the MatCap material.
Took a lot of head scratching, and it could almost certainly make more use of the Away3D framework, but I ended making the MatcapPass a multitexture shader, and implemented my own normal map routine.

source files including demo usage.

The shader code


Tuesday, 4 March 2014

MatCap Materials, and Smooth Soft Shadows for Away3D

Today I'm going to introduce a couple of techniques for improving realism, while reducing the load in real-time 3D scenes using Away3D 4.0.

You might be surprised to learn there are no lights in this scene whatsoever. Neither are the materials "baked" in any way. In fact this nice shiny plastic material can be applied to any object with mapping coordinates, though curvy, organic shapes work best.

It is known as a "MatCap" material, and was developed for Away3D by Przemek. (props!)

This type of material has long been available in Z-brush, among other packages.
The material consists of a single bitmap of a sphere and a shader that reads values from the bitmap and apply them to the object dependent on a faces angle to the view.

Here's the map used, plus a couple more examples.

You'll find more on MatCap, or Material Capture shaders all over the web. Here's a nice intro vid.

Here's the source, including the version of MatCapMaterial available at time of posting.

Apart from the actual material, take a look at the soft subtle shadow under the teapot. To get such a soft shadow I would normally render the scene using a raytracing renderer such as Mental Ray and attach it to a plane below the object, as it's just about impossible, and extremely processor intensive to do in real-time.

Currently I'm working with objects that are created in real-time, so rendering the shadow in MAX is simply not an option. To get around this I made a class called SuperSoftShadow (also provided in the source download, together with a FlashDevelop project file)

In the examples above and below the shadow is drawn dynamically when the scene initializes, and from then on uses no processor at all, it is a simple plane with the shadow TextureMaterial attached, and a BlendMode.MULTIPLY to blend it in with the background.

A couple of Gotchas there. The background here is an actual plane in the scene. If you try to use BlendMode.MULTIPLY on a background using the view3D.background, the whole screen turns black(!). Another thing I've discovered is the alpha value appears to be ignored when using blend modes.

What the SuperSoftShadow class does, is given an ObjectContainer3D such as a Loader3D, takes a snapshot using an Ortographic camera placed above, and then uses AS3's built in BlurFilter to soften it as much as you like. The resulting bitmapData is then applied to a texture on a plane, also created dynamically to match the size of the container.

Here's how to use it:

shadow = new SuperSoftShadow(container, this.stage, blur, size, yoffset, shadowColor, distance);

A look at the constructor describes the parameters.

container - The container to apply the dropshadow to. This can be a loader3D.

stage - Reference to stage required for creating stage3D instance.

blur - The blur amount for the shadow. Note that using larger bitmap sizes requires using large blur values.

size - The size of the bitmap to use. This can be any power of 2 dimension (512 x 512, 1024 x 1024, 2048 x 2048...etc.) The bigger the bitmap the better the quality.

yoffset - This is the y position within the container, which the shadow will be attached to. The class attempts to place the shadow directly underneath the container (minY) though this may not be the desired position.

shadowColor - A hexadecimal code for the colour of the shadow. This can be useful for radiosity effects for glowing objects, glass objects, etc.

distance - An optional value to control the cross-section of the object container to use. This controls the near plane of the camera that takes the snapshot. 

Another example using dynamic text...

(if you don't have the patience to watch it turn slowly, try the arrow keys :)
You can enter your own text to see the dynamic shadow in action. You can also adjust the extrusion depth of the text with up and down arrows, but you need to press ENTER to update the shadow, or the text.

The material for the text uses yet another technique, image based lighting, a bit more tricky, but worth the effort :)


Friday, 22 June 2012

New Mouse Events for Away3D 4.0

It's about time I posted something useful. Hopefully this will be of use to budding 3D application developers everywhere :)

I've extended the Away3D Mouse3DManager and MouseEvent3D classes to accept the right click and middle click events recently added to Flash Player (Novemeber 2011, I think)

Remember to update your compiler and target Flash Player 11.2 or later. I had to do a bit of mucking about in FlashDevelop to get it working myself, which may well explain why it's not yet in the official Away3d library. Here's a really crazy, wild demo to show it works...

(Click image to open demo. Click/Scroll on ball  to see events triggered)

And here are the files...

Usage Example:


Thursday, 9 February 2012

Flash is Dead, Long live AIR

Gravestone Generator

If you're on my blog, you probably belong to the group of us who appreciate Flash as a rapid development platform that just works, if you understand what you're doing.

If you're a Flash hater, you will no doubt have been delighted to read Adobe's latest press release, with yet more troll fodder.

So, where does that leave Flash actually? It will no longer be attractive for advertisers, as they want maximum reach, which will leave it to game developers, and serious application developers. Perfect.
Drop the ads you don't like in Flash, download the apps and games you do like as AIR (though you'll never even know what language they're written in, right?)

What about advertising? We all love free content, but how will websites pay the bills?
Here's a message to all the Flash haters, which I admit is a large, or at least very vocal group:

Prepare to become HTML5 haters.

Advertising pays for content creation, advertisers compete for your attention. Everything you hated in Flash will now be created in HTML5, and here's the rub, you won't be able to turn it off any more. You won't be able to disable or remove the plugin (not many flash haters actually do remove the plugin, strangely. Make your own conclusions).

As for flash developers, should we drop flash and become "interactive developers" as some Flash evangelists would have you believe? Considering the new firepower Flash 11 has got, it's never been a better time for actually making flash apps and games. The distribution model is changing, and definitely if you are a banner ad designer you should be jumping ship. For those of us that develop applications, Objective C or Java would make sense, to keep with the times and go mobile. But wait, why not just use AIR?
At this moment AIR with hardware acceleration hasn't been released, and despite all my efforts I haven't found even as much as a target launch date. But it is in closed Beta, and Adobe are showing it off  so why not stick it out, the last 10 years might not have been a complete waist of time after all...