Sound and Motion

A couple of new experiments testing the Maximo system for use in Unity, and synchronising animation and sound together. I feel Unity is missing a sound bar in the animation timeline, which would enable more accurate synching, and be a big help with the animation.
The figure is again from my forthcoming AR book about the universe.

WebGL demo

We Are All Made of Stars...

I've been playing around with this fantastic dataset HYG. It contains 120,000 star positions and magnitudes. I've coupled it with the constellations from NASA and made a Sky map, that isn't a skymap. That is to say, it's made of particles rather than the traditional bitmap image.
Several advantages there. First I stripped down the data to the 10,000 stars visible to the naked eye.
The resulting dataset is about 450Kb, compared to several megs for a comparable image map.
Then I've changed the colour from pitch black to a inky blue, which is more the color you see most nights. Obviously that can be easily selected from the Camera background colour. The stars also shimmer ever so slightly. To add a bit of interest the constellations are interactive and display their name on mouse over.

This is part of a children's book I'm writing about the universe. More on that later...
Try it for yourself :) Mouse Wheel to zoom out and see the "star globe".

Fun with Particles

No matter what platform you use, sooner of later you're going to want to have a mess with particles.
The "fun" involved is trying to get them to do what you want. I've been playing with the Unity Particle system for a couple of days now. A lot can be achieved in the UI manually, but who wants to do things manually? So, getting to grips with the system via C# seemed (at first) the best way to go about it. The system has many ahem, idiosyncrasies that at times had me going mental, but in the end I managed to beat it into submission.

The result? I like to call it home. A little old place called the Milky Way...

http://videometry.net/unity/galaxy/

The whole thing starts with a Big Bang (I am a stickler for details, so this does too).
Basically, you first have to generate all the necessary particles. It would be nice if you could set their size via script, but that would be too easy. Not all the properties you have at your disposal in the UI are available through the API, though there are workarounds for a few of them. I found I had to generate the particles first, then go back and scale them individually, not something you want in your Update function.

Issue 2. After piffling about drawing Force graphs to emit the particles in a nice spiral, I found it was WAY easier to rotate the emitter as it was generating the particles. This was great, only I had to do it in WORLD coordinates. No problem, except after generating them, you can't then rotate them, by rotating the parent object. So, after a few shandies I managed to figure out how to generate in world coordinates and then switch to LOCAL afterwards. All well and good, except for a strange anomaly.
When changing emitter to LOCAL (actually finished emitting at that point), the transformation the parent object already had somehow get's lost. So to summarize, first create your emitters, facing in the direction you want and spew your particles into world space. When finished change to LOCAL, and apply the transformation the parent object had before you switched. Simples!

I could write a short book about the ForceOverLifetime module, but I doubt it would be a best seller. If you ever mess with it, to attempt a circular motion, Good Luck! Just remember that if you apply a force in one direction you need to double the force to make it go in the other, so the usual SIN/COS technique for a circular path simply doesn't apply.

There's 40400 particles here, and I must admit I'm impressed with the performance, even on my 3 year old mobile.

It wasn't purely for fun, this is a to illustrate a page an AR book I'm writing for children to learn the wonders of the universe, from the Big Bang to the Special and General relativity. Hopefully that will be a best seller...

Hello Unity, AIR I still love you.

It's been quite a slow summer workwise, so I've had plenty time on my hands.
I've been toying with an idea to create a "digital pop-up book", that is to say a book that comes complete with a google cardboard headset, and combines a story with Augmented and Virtual Reality.

I created the first version in AIR, and did manage to get everything working. However, performance was not great. I was using four Stage3D Proxies, which worked fine on my own phone, an Oppo 7 Find, but on smaller phones some levels would disappear for no apparent reason, though I suspect there wasn't enough VRAM available. The Oppo has a dedicated graphics chip, and can easily fool you into thinking everything's pukka.

So, time to look at alternatives. I've messed with Unity since the very early days, when I actually talked with David Helgason, one of the founders, and got a free license. The whole thing started in Denmark, and I was a member of the Web3D Consortium, still trying to make solutions with X3D. At this time Flash ruled supreme, so there was no point (for me) to learn Unity. I'd made a javascript bridge between Flux Player (a then open-source X3D engine) and Flash Player, and was having a whale of a time programming in ActionScript and seeing results in 3D. Made a few good solutions on that basis, before Flux player suddenly became a closed technology, and carried a very hefty license fee.

Just when I thought I would have to drop Flash, in order to carry on working with Web3D, Stage3D came along and I've rarely looked back since. But it's time to look forward.

Now, as then, when I open Unity I can't help feeling I've gone back to the Flash MX days.
Attaching scripts to actual graphical objects the way we used to, each MovieClip having an onClipEvent(load) and onClipEvent(enterFrame). I wonder if that was the inspiration for Unity's Start(), Update() template?
In Unity the MovieClip is loosely replaced by the GameObject. That's another thing that put me off Unity. It assumes you're making a game, which generally I'm not.

Unity allows you to program in JavaScript or C#. I was tempted to go with the JavaScript approach, but that seemed like a step backwards after all that AS3 OOP goodness, so I've learned to use C#.
The transition has been completely painless, for apart from slight differences in syntax, everything is quite familiar.

I'm used to sitting with pages of code in FlashDevelop, and only seeing my 3D creations after compiling. I actually wanted a similar workflow in Unity, though now I am getting used to the way Unity works, and the ability to create structure, without necessarily having a "Main" class.
(I found out how to do that too, but don't need my comfort blanket quite as much any more :)
In Unity you do set a lot of variable and properties through the User Interface, but the user interface is created from your own code, so basically if you work the way I do, you are building a custom user interface for Editing and tweeking your application as you go along. This is actually a very cool approach. I just never got that far in my previous brief attempts to master Unity.

Once you discover the power of a full blown 3D engine, it's easy to go mad.
As I work primarily on Mobile and tablet, I have had to tone it down a bit, but it's fantastic to be able to work in a WYSIWYG environment again, the way we used to in the early Flash years.

Visual Studio is a pleasure to code in. Once I got the intellisense working, I've been able to use it to discover all the properties and methods available on different types of objects and I'm now feeling just as at home as in FlashDevelop.

Unlike most Flashers, I'm not quitting AS3 to take up a new technology, just adding to my toolbox.
Now I'm feeling comfortable with C# a whole new world has opened up and I'm excited to see where it takes me.

My first experiments have been in "Mixed Reality", that is a combination of Virtual and Augmented reality using the Vuforia library. Sometimes it's so easy it feels like cheating. While ActionScript has always been a challenging platform to master (and therefore in itself very satisfying when getting something to work!!), the combination of Unity and C# provides a powerhouse of new possibilities, as all the simple stuff is taken care of. I'm not used to anything being "under the hood", and even libraries like Away3D, I've customised and modified for my own ends.

Though I've had to go from Flash Hero to Unity Zero, I'm quickly clawing my way up to where I can call myself a Unity Developer.

Further Adventures in AGAL

I'll be honest, when I first saw AGAL some years ago, I just knew it was too obtuse and complicated for me. I was sure they'd make it easier in the next release, but 3 years later I'm still waiting.

Time to take the plunge. There's loads of great articles about AGAL and how to set it all up. It requires a ton of boiler plate AS3 code before you even get to the shader programming, but that part is at least just about identical every time so after a couple hours of wading through it line by line, you don't need to think about it any more.

I've made shaders in PixelBender, so the concept isn't entirely new to me, but in pixel bender you write a routine which is executed on each pixel. In AGAL you write a routine that is executed first on every vertex, and then on every pixel (or fragment) between them.

One of the really annoying things is the way you have to declare any variables you're going to use outside of the actual program. So you end up hopping around between shader code and AS3. The next hurdle is the actual names of the variables, as you don't get to name them yourself. vc0 for example is a vertex constant in register 0, but rather than assigning something to vc0 (vc0 = would have been nice!) you would write something like..

context3D.setProgramConstantsFromVector(Context3DProgramType.VERTEX, 4, sway, -1);

Believe it or not, that has assigned a vector ("sway") to vc0.
As they say in the Matrix, "after a while you don't even see the code". I wish!
With all the register indexes and two letter names it is very easy to make mistakes.

I'm not going to attempt to explain the mechanics, as it's been done to death and way better than I could.
I recommend starting with http://blog.norbz.net/tutorials/ and if you can get hold of this book, all the better.

And this is your Bible:
http://www.saltgames.com/2011/stage-3d-shader-cheatsheet/

That will get you up and running, but then what? There's simply tons of articles about shader programming out there, but precious few about AGAL specifically.
For a bit of inspiration I recommend a trip to shadertoy.com, but be warned, your browser will probably crash a few times, and your laptop will melt. These are GLSL shaders for WebGL, so while they may give some ideas it's far from obvious how to implement them in AGAL, and once you get past the basics of Stage3D there's not much help around.

An excellent free resource is the GPU Gems series, hosted at Nvidia.
This evening's experiment has been implementing this little ditty, animated grass.
http://http.developer.nvidia.com/GPUGems/gpugems_ch07.html
These books are great, as they explain the concepts, rather than the code.

One of the most beautiful sites you'll ever see in Denmark is a stiff sea breeze causing waves in a ripe field of barley. That was what I decided I'd try to emulate.

 DEMO TIME

As I was working with an example that had different Blend modes, I left them in (After clicking on the window use "B" to switch). I also wanted to explore and understand the mip mapping modes, so you can go through all those by pressing SPACE. Oh, and to prove it's not baked, you can use your mouse to effect the grass too.

Frame rate varies wildly. There are exactly 65536 polygons in this example, all being animated independently in groups of 16 (i.e. a single blade of barley).

There are definitely more efficient ways, but it runs at 60fps on my laptop. I can see online its not quite so good. 

What kills performance here is not the shader, but the fact I'm reading in Perlin noise from a BitmapData animated in Flash for every cycle. I'm sure there's a better way, but it's late enough as it is...

Source

MatCap in AGAL

http://videometry.net/AGAL/matcap.html

Carrying on from the previous posts,  as an exercise in AGAL I've tried implementing MatCap with Normal map in pure AGAL, based on an excellent example by Norbz

Source including maps and FlashDevelop project

MatCap with NormalMapping

What do you get if you cross this:

With this...

No? Well a glass like complex material that doesn't require any lighting!

Live demo

Continuing on from the previous post, I've now implemented normal maps into the MatCap material.
Took a lot of head scratching, and it could almost certainly make more use of the Away3D framework, but I ended making the MatcapPass a multitexture shader, and implemented my own normal map routine.

source files including demo usage.

The shader code

Enjoy!