C++ Code Generation using T4 Templates

Posted on January 24, 2012 by Kent

Introduction

Microsoft Visual Studio comes with a code generation tool which they call T4 Templates. It is somewhat integrated into Visual Studio but I have found the integration to be not that streamlined yet, and if your a C++ user its not there at all.

So in order to show you how to use T4 Templates in your projects I am just going to explain it usage solely via the command line using the TextTransform.exe program.

Find the Tool

The program they use to generate the code is called TextTransform.exe. It should be able to be found in any visual studio installation. If you have Visual Studio 10 installed then it will be in the following location on your computer…

C:\Program Files (x86)\Common Files\microsoft shared\TextTemplating\10.0\TextTransform.exe

Find this file on your computer. Now create a new folder somewhere on your system. Make it an easy location you can get to via a commandline like “c:\templatetest”. Copy the TextTransform.exe to this new folder. You are now ready to start experimenting.

Generating your first code

Open up a command prompt (go to the windows menu and just type cmd.exe).
Next navigate to your new folder (cd c:\templatetest).
Now navigate to the folder on your desktop and create a new file in that folder called “test.tt”.

If you used the right click mouse button menu to create a text file then you might find it actually created a file called test.tt.txt on your computer. If it does then on the folder click on “Organize – Folder And Search Options”. In the window that pops up select the “View” tab find the option “Hide Extensions for known file types”. Uncheck this checkbox and press OK. Go back to your folder and you will see the “text.tt.txt” file. Right click and rename this file to test.tt”.

Open the test.tt file in any text editor of your choosing (You can use Visual Studio if you wish and there is even a syntax highlighting plugin that you can install to make it all pretty, but I will leave this for another post since its not vital to actually using the code generator).

In the file paste the following code (Hint: Double click the code to select all of it)

<#@ template debug="false" hostspecific="false" language="C#" #>
<#@ output extension=".h" #>
<# var members = new string [] {"variableA", "variableB", "variableC"}; #>
class MyClass
{
public:
	MyClass();
private:
<#
  foreach (string memberName in members)
  { #>
	int m_<#= memberName #>;
<# } #>
}

Save the file and go to your command window and type in the following: TextTransform.exe test.tt

This code will generate a file called “test.h” that looks like the following.

class MyClass
{
public:
	MyClass();
private:
	int m_variableA;
	int m_variableB;
	int m_variableC;
}

Great! You have just generated your first file!

So what just happened?

The TextTransform.exe program just took your input file and parsed through it and generated the output file.

<#@ template debug="false" hostspecific="false" language="C#" #>

This line tells the generator that the language used in the template is C#. You can also use VB if you know visual basic instead.

<#@ output extension=".h" #>

This line tells it that your out file is going to have the extension “.h”. So here you can swap it for anything you want (.xml, .cpp, .customformat).

The rest of the code uses C# code that is between special flags to let the generator know there is some C# code that will be used to generate some code.

In this example I just wrote out a simple class with 3 member variables.

How to use a file to list the member variables

If you are going to be generating a bunch of files, say for marshaling and un-marshaling of data for a custom networked protocol, then you want to have the model that describes this in some file, that is then read in and parsed, to generate all the required output files (by using multiple template.tt files that generate a specific output file based on the same data).

A simple approach would be to use a plain text file, so lets do that.

Create another file in your directory called “vars.txt”.

Open it up and paste the following into it

variableA
variableB
variableC

Now modify your test.tt file to be the following

<#@ assembly name="System.Xml.dll" #>
<#@ import namespace="System.Xml" #>
<#@ import namespace="System.IO" #>
<#@ template debug="false" hostspecific="false" language="C#" #>
<#@ output extension=".h" #>
<# var members = File.ReadLines("vars.txt");#>
class MyClass
{
public:
	MyClass();
private:
<#
  foreach (string memberName in members)
  { #>
	int m_<#= memberName #>;
<# } #>
}

Now go to the command prompt and run the TextTemplate again TextTransform.exe test.tt

You will now see the following, which is the same as before except this time all the member variables were read in from a file.

class MyClass
{
public:
	MyClass();
private:
	int m_variableA;
	int m_variableB;
	int m_variableC;
}

So now if we wanted to we could write another template file that would output a .cpp file that initializes all these variables. Lets do that.

Create a new file called testcpp.tt and paste in the following code

<#@ assembly name="System.Xml.dll" #>
<#@ import namespace="System.Xml" #>
<#@ import namespace="System.IO" #>
<#@ template debug="false" hostspecific="false" language="C#" #>
<#@ output extension=".cpp" #>
<# var memberVars = File.ReadLines("vars.txt");#>
#include "test.h"
MyClass::MyClass()
{
	<# foreach (string member in memberVars) { #>
	m_<#=member #> = 0;
	<# } #>
}

Run this on in the command prompt using TextTransform.exe testcpp.tt

And it will generate a file called testcpp.cpp (OK the naming is not that great yet and we need to make the name of the file part of the template, I will leave this for now).

The file will look like the following…

#include "test.h"
MyClass::MyClass()
{
		m_variableA = 0;
		m_variableB = 0;
		m_variableC = 0;
	}

So now we have generated a .h and a .cpp file based off a crude text based model description.

Using an XML file as the model

I much prefer to use XML as my model file format and then generate specific parts of the file based off of specific attributes in the model.

Create a new file called class1.xml and paste in the following text

 <Class>
    <MemberVariable Name="variableA"/>
    <MemberVariable Name="variableB"/>
    <MemberVariable Name="variableC"/>
  </Class>

Now open up the test.tt file and replace it with the following

<#@ assembly name="System.Xml.dll" #>
<#@ import namespace="System.Xml" #>
<#@ import namespace="System.IO" #>
<#@ import namespace="System.Collections.Generic" #>
<#@ template debug="false" hostspecific="false" language="C#" #>
<#@ output extension=".h" #>
class MyClass
{
	public:
		MyClass();
	private:
<#
	XmlDocument doc = new XmlDocument();
    string absolutePath = Path.GetFullPath("class1.xml");
    doc.Load(absolutePath);
    foreach (XmlNode node in doc.SelectNodes("/Class/MemberVariable"))
    {
#>
		int m_<#=node.Attributes["Name"].InnerText #>;
<#    } #>

};

Run the TextTemplate on the test.tt file (you should know how to do this by now) and you will get the following (which should also look familiar).

class MyClass
{
	public:
		MyClass();
	private:
		int m_variableA;
		int m_variableB;
		int m_variableC;

};

How to use a function in a template

You could also arrange the code into a function to clean it up and make it easier to read.

Paste the following into test.tt

<#@ assembly name="System.Xml.dll" #>
<#@ import namespace="System.Xml" #>
<#@ import namespace="System.IO" #>
<#@ import namespace="System.Collections.Generic" #>
<#@ template debug="false" hostspecific="false" language="C#" #>
<#@ output extension=".h" #>
class MyClass
{
	public:
		MyClass();
	private:
	<# foreach (string member in this.GetMemberVariables()) { #>
	int m_<#=member #>;
	<# } #>
};

<#+
    public List<string> GetMemberVariables()
    {
        List<string> result = new List<string>();
        XmlDocument doc = new XmlDocument();
        string absolutePath = Path.GetFullPath("class1.xml");
        doc.Load(absolutePath);
        foreach (XmlNode node in doc.SelectNodes("/Class/MemberVariable"))
        {
            result.Add(node.Attributes["Name"].InnerText);
        }
        return result;
    }
#>

When you run this it will produce the same result as before.

Clean up the function and make it more usable

Lets parse this data a bit better so that we can generate a file from multiple attributes described in the XML file. It would also be nice if we could generate multiple classes as well so lets do that!

Paste this code into test.tt.

<#@ assembly name="System.Xml.dll" #>
<#@ import namespace="System.Xml" #>
<#@ import namespace="System.IO" #>
<#@ import namespace="System.Collections.Generic" #>
<#@ template debug="false" hostspecific="false" language="C#" #>
<#@ output extension=".h" #>
<# foreach (ClassDefinition classDef in this.GetClasses()) { #>
class <#=classDef.name #>
{
	public:
		<#=classDef.name #>();
	private:
	<# foreach (MemberDefinition member in classDef.privateMembers) { #>
	<#=member.type #> m_<#=member.name #>; <# if(member.comment.Length > 0) { #> //<#=member.comment #> <# } #>

	<# } #>
};
<# } #>

<#+
	public class ClassDefinition
	{
		public string name;
        public List<MemberDefinition> privateMembers;

		public ClassDefinition()
		{
			privateMembers = new List<MemberDefinition>();
		}
	}

	public class MemberDefinition
	{
		public string name;
		public string comment;
		public string defaultValue;
		public string type;
	}

    public List<ClassDefinition> GetClasses()
    {
        List<ClassDefinition> classes = new List<ClassDefinition>();
        XmlDocument doc = new XmlDocument();
        string absolutePath = Path.GetFullPath("class2.xml");
        doc.Load(absolutePath);
        foreach (XmlNode node in doc.SelectNodes("/root/Class"))
        {
			ClassDefinition newClass = new ClassDefinition();
			newClass.name = node.Attributes["Name"].InnerText;

	        foreach (XmlNode memberNode in node.SelectNodes("MemberVariable"))
		    {
				MemberDefinition newMember = new MemberDefinition();
				newMember.name = memberNode.Attributes["Name"].InnerText;
				newMember.defaultValue = memberNode.Attributes["Default"].InnerText;
				newMember.comment = memberNode.Attributes["Comment"].InnerText;
				newMember.type = memberNode.Attributes["Type"].InnerText;
				newClass.privateMembers.Add(newMember);
			}
            classes.Add(newClass);
        }
        return classes;
    }
#>

Now create a new class2.xml file and paste in the following data

<root>
  <Class Name="MyClassA">
    <MemberVariable Name="variableA" Default="1" Type="int" Comment="This is some documentation describing the variable"/>
    <MemberVariable Name="variableB" Default="2" Type="double" Comment="This is some documentation describing the variable"/>
    <MemberVariable Name="variableC" Default="3" Type="float" Comment="This is some documentation describing the variable"/>
  </Class>
  <Class Name="MyClassB">
    <MemberVariable Name="variableD" Default="4" Type="int" Comment="This is some documentation describing the variable"/>
    <MemberVariable Name="variableE" Default="5" Type="double" Comment="This is some documentation describing the variable"/>
    <MemberVariable Name="variableF" Default="6" Type="float" Comment="This is some documentation describing the variable"/>
  </Class>
</root>

Run the TextTemplate executable and you will now get the following file

class MyClassA
{
	public:
		MyClassA();
	private:
		int m_variableA;  //This is some documentation describing the variable
		double m_variableB;  //This is some documentation describing the variable
		float m_variableC;  //This is some documentation describing the variable
	};
class MyClassB
{
	public:
		MyClassB();
	private:
		int m_variableD;  //This is some documentation describing the variable
		double m_variableE;  //This is some documentation describing the variable
		float m_variableF;  //This is some documentation describing the variable
	};

Conclusion

As you can see TextTransform.exe (or if you prefer the T4 Template Code Generator) is a very powerful tool. I will leave it up to you to create the cpp output file for this template.

I hope you enjoyed this little tutorial. I may expand on this in future with a more fully featured example to generate some actual usable code in a real life situation, but for now this should at least have opened your eyes to the world of code generation. Enjoy!

Techstars – TV documentry on tech startups

Posted on January 20, 2012 by Kent

Techstars, a multi-city statup incubator, in conjuction with Bloomberg released a TV reality-documentary TV series based on what its like to be in the Techstars bootcamp to start your own technology based business.

Based in New York, Techstars took 579 startups and wittled them down to just 11 hopeful entrepreneurs/startups and brought them all into the same space in downtown New York. There they will try to cram 3 years worth of entrepreneurial experience into just 3 months. Do they have the skill and passion to succeed? Watch and find out.

Episode list…

http://www.bloomberg.com/tv/shows/techstars/

First episode can be found here.

There is also a series of videos called “The Founders | Techstars Boulder” which you can find on youtube from 2009. That link doesn’t show all of them in the right order so for your convenience I have listed them below…

Setting up your Canon 60D for shooting movies

Posted on January 20, 2012 by Kent

After having recently bought a Canon 60D I have been shooting some short movies and experimenting with tracking using Syntheyes. The problem is that the quality of the movies seem to be quite bad and I have found it very difficult to create a good track, especially with outside shots. Blury scenes, colors not that clear, no contrast. So I thought either I need a better lens or maybe there are some settings on the camera that I need to tweak to get better shots. Turns out I need both. After a bit of hunting I found these excellent tutorials on how to setup your Canon 60D to enable you to create better movies. Check them out here.

The second video goes over the settings you will need to change to get better shots with your Canon 60D (although in the Video they are using the 7D, but it is very similar).

The third video suggest using a 30mm lens such as the Sigma 30mm F1.4 EX DC HSM. Here is a link to google search, also on Amazon for Canons and finally on Ken Rockwells site.

The google search reveals 221 reviews with an average of 4.5/5 star rating. Ken actually gives this a bad review and states the following “If you shoot Canon, forget this Sigma and get the superior Canon 28mm f/1.8 EF for less money.” But at least on Amazon.co.uk the Canon was 427.99 pounds while the Sigma was 377.36 pounds. The issue is to do with the Auto Focus and also that you end up with black marks in the corners when used on a full frame cameras, but since the Canon 60D is a cropped 1.6x then this will not be an issue for me. Click here to read his review of the Canon 28mm f/1.8 EF. You do loose 2/3 stop by going to the Canon and since I would like to try some low light conditions I am very interested in the Sigma.

Another review of Sigma 30mm lens with the Canon 28mm F1.8 can be found here. This review states that they are very similar.

Also interesting is this youtube clip of a comparison between the Canon 35mm F1.4 and the Sigma 30mm F1.4.

After all this research I decided to give the Sigma a try. Its ordered from Amazon and on its way. I think its going to be a great lens for filming movies and it may even take over from my kit lens (18-135mm F3.5-5.6).

Experiment with Wt, Cinema4D Melange and WebGL

Posted on January 20, 2012 by Kent

Yesterday I decided to have a play with Wt. Wt (or Witty) is a C++ Web Toolkit which lets you write a webserver entirely in C++ without having to know anything about HTML, Javascript, HTTP or TCP/IP.

I was also interested to see that they have basic support for WebGL in there and decided to give it a test drive. Not being one to make things simple I thought I would also throw Cinema4D’s Melange library into the mix to allow the export of some interesting scenes from Cinema4D and try to view them in a web browser.

Getting started with Wt

Getting up and running with Wt is not simple. First you need to download Wt and unzip it somewhere. Then also get CMake and the latest boost libraries. You can choose to get the prebuilt boost libraries from BoostPro, but if like me you don’t have a lot of bandwidth then you can compile the boost libraries without too much of a problem.

After following through some of the instructions here for unix or here for windows on using CMake, setting some paths to boost, postgresql (if you have it) then you should successfully be able to generate the Visual Studio Solution files from the CMake program.

Now you can load up the Wt.sln file into Visual Studio and finally build Wt.

After building the WebGL project (part of the Wt.Sln) you might want to test it out to make sure your browser can support it. To do this open the Properties for that Project and set the debugging command arguments to “–http-address=0.0.0.0 –http-port=8080 –deploy-path=/hello –docroot=.”. Now you can run that project, open up a browser (I am using Firefox 4) and go to “http://127.0.0.1:8080/hello” and you will see… nothing.

If you see nothing like me then you need to edit the “teapot.C” file and set the path to the teapot.obj file for it to load, in my case it was the following…

int main(int argc, char **argv)
{
 readObj("C:\\Projects\\other\\Wt\\wt-3.1.9\\examples\\webgl\\teapot.obj", data);
 return WRun(argc, argv, &createApplication);
}

Now when you run the project and try that link again in your browser you should see a teapot!

Add in Melange

Get Melange from the plugin cafe at Maxon.Net

Next rip apart the WebGL demo project and throw in some code to load in Cinema4D (C4D) files. You will also need to add the melange include and lib directories to your visual studio project.

The code below loads in polygon objects and seperates them into VertexBufferObjects that only have a maximum of 65000 vertices (a current WebGL limitation). I have also not worried about calculating vertex normals and just use face normals.

So… what is the result of all this messing around… well here it is!

Good ol’ primitives.

In the end there were 5 files. These files contain parts of the Wt example files and so any use of this code must also abide by the same licensing terms as outlined on the Wt website.

c4dobjects.h

#ifndef C4D_OBJECTS_H
#define C4D_OBJECTS_H

#include <vector>
#include <string>
#include <boost/tuple/tuple.hpp>

#include <Wt/WGLWidget>
#include <Wt/WMatrix4x4>

#include "c4d_memory.h"
#include "c4d_file.h"

using namespace _melange_;

class PaintWidget;

typedef std::vector PointList;

class C4DWebObjPart
{
public:
	C4DWebObjPart();
	~C4DWebObjPart();

	void SetupBuffers(PaintWidget *pWidget);
	void PaintGL(PaintWidget *pWidget);
	void AddPoint(const Vector &point);
	void AddNormal(const Vector &normal);

	const PointList &GetPoints() const;
	const PointList &GetNormals() const;

	Wt::WGLWidget::Buffer GetPointBuffer() const;
	Wt::WGLWidget::Buffer GetNormalBuffer() const;

	void SetPointBuffer(Wt::WGLWidget::Buffer buffer);
	void SetNormalBuffer(Wt::WGLWidget::Buffer buffer);

private:
	PointList m_points;
	PointList m_normals;
	Wt::WGLWidget::Buffer m_pointsBuffer;
	Wt::WGLWidget::Buffer m_normalBuffer;
};

typedef std::vector Parts;

class C4DWebObj
{
public:
	C4DWebObj();
	~C4DWebObj();
	void SetupBuffers(PaintWidget *pWidget);
	void PaintGL(PaintWidget *pWidget);
	void SetMatrix(const Wt::WMatrix4x4 &mat);
	void AddPart(C4DWebObjPart *pPart);

	Parts &GetParts();
	const Wt::WMatrix4x4 &GetMatrix() const;

private:
	Parts m_parts;
	Wt::WMatrix4x4 m_modelMatrix;
};

typedef std::vector WebObjectsList;

class C4DWebObjects
{
public:
	C4DWebObjects();
	~C4DWebObjects();
	void SetupBuffers(PaintWidget *pWidget);
	void PaintGL(PaintWidget *pWidget);
	void LoadFile(const char *pFilename);
	void AddObjectsRec(BaseObject *pObj);

	WebObjectsList &GetObjects();

private:
	WebObjectsList m_objects;
};

#endif

c4dobjects.cpp

#include "c4dobjects.h"
#include "paintwidget.h"
#include "default_alien_overloads.h"

using namespace _melange_;

// overload this function
void GetWriterInfo(LONG &id, String &appname )
{
	// register your own pluginid once for your exporter and enter it here under id
	// this id must be used for your own unique ids
	// 	Bool AddUniqueID(LONG appid, const CHAR *const mem, LONG bytes);
	// 	Bool FindUniqueID(LONG appid, const CHAR *&mem, LONG &bytes) const;
	// 	Bool GetUniqueIDIndex(LONG idx, LONG &id, const CHAR *&mem, LONG &bytes) const;
	// 	LONG GetUniqueIDCount() const;
	id = 0;
	appname = "C4D File Viewer";
}

//=======================================================
C4DWebObjPart::C4DWebObjPart()
{
}

C4DWebObjPart::~C4DWebObjPart()
{
}

void C4DWebObjPart::AddPoint(const Vector &point)
{
	m_points.push_back(point.x * 0.01);
	m_points.push_back(point.y * 0.01);
	m_points.push_back(point.z * 0.01);
}

void C4DWebObjPart::AddNormal(const Vector &normal)
{
	m_normals.push_back(normal.x);
	m_normals.push_back(normal.y);
	m_normals.push_back(normal.z);
}

const PointList &C4DWebObjPart::GetPoints() const
{
	return m_points;
}

const PointList &C4DWebObjPart::GetNormals() const
{
        return m_normals;
}

Wt::WGLWidget::Buffer C4DWebObjPart::GetPointBuffer() const
{
	return m_pointsBuffer;
}

Wt::WGLWidget::Buffer C4DWebObjPart::GetNormalBuffer() const
{
	return m_normalBuffer;
}

void C4DWebObjPart::SetPointBuffer(Wt::WGLWidget::Buffer buffer)
{
	m_pointsBuffer = buffer;
}

void C4DWebObjPart::SetNormalBuffer(Wt::WGLWidget::Buffer buffer)
{
	m_normalBuffer = buffer;
}

//=======================================================
C4DWebObj::C4DWebObj()
{
}

C4DWebObj::~C4DWebObj()
{
	for(unsigned int i=0;iGetFirstObject();
		if(pObj)
		{
			AddObjectsRec(pObj);
		}
	}
}

void C4DWebObjects::AddObjectsRec(BaseObject *pObj)
{
	if(pObj)
	{
		C4DWebObj *pWebObject = new C4DWebObj();
		if(pObj->GetType() == Opolygon)
		{
			C4DWebObjPart *pWebObjectPart = new C4DWebObjPart();
			pWebObject->AddPart(pWebObjectPart);

			PolygonObject *pPolyObj = (PolygonObject*)pObj;
			const CPolygon* pPolys = pPolyObj->GetPolygonR();
			const Vector *pPoints = pPolyObj->GetPointR();
			LONG polyCount = pPolyObj->GetPolygonCount();
			LONG pointCount = pPolyObj->GetPointCount();

			Matrix mat = pPolyObj->GetMg(); 

                        pWebObject->SetMatrix(Wt::WMatrix4x4(
                                mat.v1.x,mat.v1.y,mat.v1.z,mat.off.x * 0.01,
				mat.v2.x,mat.v2.y,mat.v2.z,mat.off.y * 0.01,
				mat.v3.x,mat.v3.y,mat.v3.z,mat.off.z * 0.01,
				0,0,0,1));

			LONG a;
			for(a=0;aAddPoint(pointA);
				pWebObjectPart->AddPoint(pointB);
				pWebObjectPart->AddPoint(pointC);

				pWebObjectPart->AddNormal(cross);
				pWebObjectPart->AddNormal(cross);
				pWebObjectPart->AddNormal(cross);

				if(poly.c != poly.d)
				{
					pWebObjectPart->AddPoint(pointA);
					pWebObjectPart->AddPoint(pointC);
					pWebObjectPart->AddPoint(pointD);

					pWebObjectPart->AddNormal(cross);
					pWebObjectPart->AddNormal(cross);
					pWebObjectPart->AddNormal(cross);
				}

				if(pWebObjectPart->GetPoints().size() > 65000)
				{
					pWebObjectPart = new C4DWebObjPart();
					pWebObject->AddPart(pWebObjectPart);
				}
			}
		}
		m_objects.push_back(pWebObject);

		AddObjectsRec(pObj->GetDown());
		AddObjectsRec(pObj->GetNext());
	}
}

paintwidget.h

/*
 * Copyright (C) 2010 Emweb bvba, Heverlee, Belgium.
 *
 * See the LICENSE file for terms of use.
 */

#ifndef PAINTWIDGET_H_
#define PAINTWIDGET_H_

#include <Wt/WGLWidget>
#include "c4dobjects.h"

// You must inherit Wt::WGLWidget to draw a 3D scene
class PaintWidget: public Wt::WGLWidget
{
public:
  PaintWidget(Wt::WContainerWidget *root);

  virtual void initializeGL();  // Specialization of WGLWidgeT::intializeGL()
  virtual void paintGL();  // Specialization of WGLWidgeT::paintGL()
  virtual void resizeGL(int width, int height);  // Specialization of WGLWidgeT::resizeGL()

  // Sets the shader source. Must be set before the widget is first rendered.
  void SetShaders(const std::string &vertexShader, const std::string &fragmentShader);
  void LoadFile(const char *fileName);

private:
  // The shaders, in plain text format
  std::string m_vertexShader;
  std::string m_fragmentShader;

  // Program and related variables
  Program m_shaderProgram;
  AttribLocation m_vertexPositionAttribute;
  AttribLocation m_vertexNormalAttribute;
  UniformLocation m_pMatrixUniform;
  UniformLocation m_cMatrixUniform;
  UniformLocation m_mvMatrixUniform;
  UniformLocation m_nMatrixUniform;

  // A client-side JavaScript matrix variable
  JavaScriptMatrix4x4 m_jsMatrix;

  C4DWebObjects m_objects;
};

#endif

paintwidget.cpp

#include "PaintWidget.h"
#include <Wt/WGLWidget>
#include <Wt/WMatrix4x4>

using namespace Wt;

PaintWidget::PaintWidget(WContainerWidget *root): WGLWidget(root)
{
}

// The initializeGL() captures all JS commands that are to be executed
// before the scene is rendered for the first time. It is executed only
// once. It is re-executed when the WebGL context is restored after it
// was lost.
// In general, it should be used to set up shaders, create VBOs, initialize
// matrices, ...
void PaintWidget::initializeGL()
{
  // In order to know where to look at, calculate the centerpoint of the scene
  double cx, cy, cz;
  cx=cy=cz=0;

  // Transform the world so that we look at the centerpoint of the scene
  WMatrix4x4 worldTransform;
  worldTransform.lookAt(
      cx, cy, cz + 10, // camera position
      cx, cy, cz,      // looking at
      0, 1, 0);        // 'up' vector

  // We want to be able to change the camera position client-side. In
  // order to do so, the world transformation matrix must be stored in
  // a matrix that can be manipulated from JavaScript.
  m_jsMatrix = createJavaScriptMatrix4();
  setJavaScriptMatrix4(m_jsMatrix, worldTransform);

  // This installs a client-side mouse handler that modifies the
  // world transformation matrix. Like WMatrix4x4::lookAt, this works
  // by specifying a center point and an up direction; mouse movements
  // will allow the camera to be moved around the center point.
  setClientSideLookAtHandler(m_jsMatrix, // the name of the JS matrix
      cx, cy, cz,                       // the center point
      0, 1, 0,                          // the up direction
      0.005, 0.005);                    // 'speed' factors
  // Alternative: this installs a client-side mouse handler that allows
  // to 'walk' around: go forward, backward, turn left, turn right, ...
  //setClientSideWalkHandler(jsMatrix_, 0.05, 0.005);

  // First, load a simple shader
  Shader fragmentShader = createShader(FRAGMENT_SHADER);
  shaderSource(fragmentShader, m_fragmentShader);
  compileShader(fragmentShader);

  Shader vertexShader = createShader(VERTEX_SHADER);
  shaderSource(vertexShader, m_vertexShader);
  compileShader(vertexShader);

  m_shaderProgram = createProgram();
  attachShader(m_shaderProgram, vertexShader);
  attachShader(m_shaderProgram, fragmentShader);
  linkProgram(m_shaderProgram);
  useProgram(m_shaderProgram);

  // Extract the references to the attributes from the shader.
  m_vertexNormalAttribute = getAttribLocation(m_shaderProgram, "aVertexNormal");
  m_vertexPositionAttribute = getAttribLocation(m_shaderProgram, "aVertexPosition");
  enableVertexAttribArray(m_vertexPositionAttribute);
  enableVertexAttribArray(m_vertexNormalAttribute);

  // Extract the references the uniforms from the shader
  m_pMatrixUniform  = getUniformLocation(m_shaderProgram, "uPMatrix");
  m_cMatrixUniform  = getUniformLocation(m_shaderProgram, "uCMatrix");
  m_mvMatrixUniform = getUniformLocation(m_shaderProgram, "uMVMatrix");
  m_nMatrixUniform  = getUniformLocation(m_shaderProgram, "uNMatrix");

  // Create a Vertex Buffer Object (VBO) and load all polygon's data
  // (points, normals) into it. In this case we use one VBO that contains
  // all data (6 per point: vx, vy, vz, nx, ny, nz); alternatively you
  // can use multiple VBO's (e.g. one VBO for normals, one for points,
  // one for texture coordinates).
  // Note that if you use indexed buffers, you cannot have indexes
  // larger than 65K, due to the limitations of WebGL.
  WebObjectsList &objects = m_objects.GetObjects();
  for(unsigned int i=0;iGetParts();
	for(unsigned int j=0;jSetPointBuffer(createBuffer());
		bindBuffer(ARRAY_BUFFER, pPart->GetPointBuffer());
		const PointList &points = pPart->GetPoints();
		bufferDatafv(ARRAY_BUFFER, points.begin(), points.end(), STATIC_DRAW);

		pPart->SetNormalBuffer(createBuffer());
		bindBuffer(ARRAY_BUFFER, pPart->GetNormalBuffer());

		const PointList &normals = pPart->GetNormals();
		bufferDatafv(ARRAY_BUFFER, normals.begin(), normals.end(), STATIC_DRAW);
	}
  }

  // Set the clear color to a transparant background
  clearColor(0, 0, 0, 0);

  // Reset Z-buffer, enable Z-buffering
  clearDepth(1);
  enable(DEPTH_TEST);
  depthFunc(LEQUAL);
}

void PaintWidget::resizeGL(int width, int height)
{
  // Set the viewport size.
  viewport(0, 0, width, height);

  // Set projection matrix to some fixed values
  WMatrix4x4 proj;
  proj.perspective(45, ((double)width)/height, 1, 40);
  uniformMatrix4(m_pMatrixUniform, proj);
}

// The paintGL function is executed every time the canvas is to be
// repainted. For example: when the camera location is modified,
// an animated object is changed, ...
void PaintWidget::paintGL()
{
  // Clear color an depth buffers
  clear(COLOR_BUFFER_BIT | DEPTH_BUFFER_BIT);

  // Configure the shader: set the uniforms
  // Uniforms are 'configurable constants' in a shader: they are
  // identical for every point that has to be drawn.
  // Set the camera transformation to the value of a client-side JS matrix
  uniformMatrix4(m_cMatrixUniform, m_jsMatrix);

  WebObjectsList &objs = m_objects.GetObjects();
  for(unsigned int i=0;iGetMatrix();
	  uniformMatrix4(m_mvMatrixUniform, modelMatrix);

	  // The next one is a bit complicated. In desktop OpenGL, a shader
	  // has the gl_NormalMatrix matrix available in the shader language,
	  // a matrix that is used to transform normals to e.g. implement proper
	  // Phong shading (google will help you to find a detailed explanation
	  // of why you need it). It is the transposed inverse of the model view
	  // matrix. Unfortunately, this matrix is not available in WebGL, so if
	  // you want to do phong shading, you must calculate it yourself.
	  // Wt provides methods to calculate the transposed inverse of a matrix,
	  // when client-side JS matrices are involved. Here, we inverse-transpose
	  // the product of the client-side camera matrix and the model matrix.
	  uniformMatrix4(m_nMatrixUniform, (m_jsMatrix * modelMatrix).inverted().transposed());

	Parts &parts = pObj->GetParts();
	for(unsigned int j=0;jGetPointBuffer());

		// Configure the vertex attributes:
		vertexAttribPointer(m_vertexPositionAttribute,
		  3,     // size: Every vertex has an X, Y anc Z component
		  FLOAT, // type: They are floats
		  false, // normalized: Please, do NOT normalize the vertices
		  3*4, // stride: The first byte of the next vertex is located this
		  //         amount of bytes further. The format of the VBO is
		  //         vx, vy, vz, and every element is a
		  //         Float32, hence 4 bytes large
		  0);    // offset: The byte position of the first vertex in the buffer
		//         is 0.

		bindBuffer(ARRAY_BUFFER, pPart->GetNormalBuffer());
		vertexAttribPointer(m_vertexNormalAttribute,
		  3,
		  FLOAT,
		  false,
		  3*4, // stride: see above. We jump from normal to normal now
		  0);  // offset: 

		// Now draw all the triangles.
		drawArrays(TRIANGLES, 0, pPart->GetPoints().size()/3);
	}
  }
}

void PaintWidget::SetShaders(const std::string &vertexShader, const std::string &fragmentShader)
{
  m_vertexShader = vertexShader;
  m_fragmentShader = fragmentShader;
}

void PaintWidget::LoadFile(const char *fileName)
{
	m_objects.LoadFile(fileName);
}

main.cpp

#include <Wt/WApplication>
#include <Wt/WBreak>
#include <Wt/WContainerWidget>
#include <Wt/WImage>
#include <Wt/WPushButton>
#include <Wt/WTabWidget>
#include <Wt/WText>
#include <Wt/WTextArea>
#include <Wt/WFitLayout>
#include <Wt/WBorderLayout>
#include <Wt/WGridLayout>

#include "paintwidget.h"

using namespace Wt;

const char *fragmentShaderSrc =
"#ifdef GL_ES\n"
"precision highp float;\n"
"#endif\n"
"\n"
"varying vec3 vLightWeighting;\n"
"\n"
"void main(void) {\n"
"  vec4 matColor = vec4(0.278, 0.768, 0.353, 1.0);\n"
"  gl_FragColor = vec4(matColor.rgb * vLightWeighting, matColor.a);\n"
"}\n";

const char *vertexShaderSrc =
"attribute vec3 aVertexPosition;\n"
"attribute vec3 aVertexNormal;\n"
"\n"
"uniform mat4 uMVMatrix; // [M]odel[V]iew matrix\n"
"uniform mat4 uCMatrix;  // Client-side manipulated [C]amera matrix\n"
"uniform mat4 uPMatrix;  // Perspective [P]rojection matrix\n"
"uniform mat4 uNMatrix;  // [N]ormal transformation\n"
"// uNMatrix is the transpose of the inverse of uCMatrix * uMVMatrix\n"
"\n"
"varying vec3 vLightWeighting;\n"
"\n"
"void main(void) {\n"
"  // Calculate the position of this vertex\n"
"  gl_Position = uPMatrix * uCMatrix * uMVMatrix * vec4(aVertexPosition, 1.0);\n"
"\n"
"  // Phong shading\n"
"  vec3 transformedNormal = normalize((uNMatrix * vec4(normalize(aVertexNormal), 0)).xyz);\n"
"  vec3 lightingDirection = normalize(vec3(1, 1, 1));\n"
"  float directionalLightWeighting = max(dot(transformedNormal, lightingDirection), 0.0);\n"
"  vec3 uAmbientLightColor = vec3(0.2, 0.2, 0.2);\n"
"  vec3 uDirectionalColor = vec3(0.8, 0.8, 0.8);\n"
"  vLightWeighting = uAmbientLightColor + uDirectionalColor * directionalLightWeighting;\n"
"}\n";

class C4DWeb : public WApplication
{
public:
  C4DWeb(const WEnvironment& env);
  bool Init();
};

C4DWeb::C4DWeb(const WEnvironment& env) : WApplication(env)
{
}

bool C4DWeb::Init()
{
	setTitle("C4D File Viewer");
	WContainerWidget *pGLContainer = new WContainerWidget(root());
	pGLContainer->resize(512, 512);
	pGLContainer->setInline(false);
	PaintWidget *pPaintWidget = new PaintWidget(pGLContainer);
	pPaintWidget->resize(512, 512);
	pPaintWidget->SetShaders(vertexShaderSrc, fragmentShaderSrc);
	pPaintWidget->LoadFile("C:\\temp\\test.c4d");

	return true;
}

WApplication *createApplication(const WEnvironment& env)
{
	C4DWeb *pData = new C4DWeb(env);
	if(pData)
	{
		if(pData->Init()) return pData;
	}
	delete pData;
	return NULL;
}

int main(int argc, char **argv)
{
	return WRun(argc, argv, &createApplication);
}

Importing Quake files and previewing levels using Irrlicht

Posted on January 20, 2012 by Kent

Note: This article was written in 2009 and some things may have changed.

Introduction

This tutorial will show you how to use the Irrlicht graphics engine to load quake levels into Cinema4D (including the mesh, materials and textures) and then export that back out again to the Irrlicht graphics engine to preview how your game level will look. You will learn about creating polygon objects, materials and textures in Cinema4D and how to access the data again for exporting.
Note: I have just realized that my code has been messed up a little bit by posting to the website. Some of the loops might not look correct due to the removal of less than and greater than brackets. I will fix this up at a later date, for now just refer to the code in the attachment at the end of this article.

Getting Setup

Before we start you will need to setup your development environment. For this tutorial we are using Cinema4D 11.5, Irrlicht 1.6 and Microsoft Visual Studio C++ 2008 Express Edition. And in this tutorial we are running under Vista and Windows 7, but other operating systems should still work. Its best if you follow through the steps in the Getting started with the Cinema4D SDK and Visual Studio C++ 2008 Express Edition tutorial to get yourself setup. The important thing to remember is that you must copy across the DebugWin32.vsprops file in order to compile the plugin, read the tutorial to find out what to do.

Download the attachment at the end of this article and extract it to your Cinema4D 11.5\plugins directory. There should now be a irrlichtviewer folder directly within the plugins directory which contains the projects files. If instead you have a irrlichtviewer folder within a irrlichtviewer folder then you will need to copy the files down one level. This sometimes happens depending on how you do your unzipping.
You will now have to copy the Irrlicht.dll file down a couple of levels next to where Cinema4D is installed. On my machine it is located here… “C:\Program Files\MAXON\CINEMA 4D R11.5″
Note: If you don’t want to compile just yet and only want to run the project then you can just start Cinema4D now skip to the Running the Project section below.
Right click on Visual Studio and choose “Run as Administrator” to run it with the ability to make changes to files within the “Programs Files” folder, which is most likely where your copy of Cinema4D got installed.
When you have installed Irrlicht onto your machine you will need to update the project paths to make sure it can find the right files.

Right Click on the”irrlichtviewer” project and choose “properties”.
Open up the “Configuration Properties->C/C++-> General” section.
At the top you will see the “Additional Include Directories”. It will contain the text “C:\Projects\3rdParty\irrlicht\include”.
Change this to the include path of your copy of Irrlicht and click Apply.
Next go to the “Configuration Properties->Linker->General” section.
Find the text in “Additional Library Directories”. It will currently read “C:\Projects\3rdParty\irrlicht\lib\Win32-visualstudio”.
Change this to point to the same location of win32-visualstudio as in your lirrlicht install.
When done press OK on the dialog to save your changes.

Right you should now be all setup. From the Build menu choose “Build Solution”.
If this doesn’t build then you may have missed something in the tutorial on setting up with Visual Studio. Most likely if you are first time user you might not have seen the part on how to fixup the “_api” project. Check that tutorial now and then try building your solution again.

Running the Project

Now that you have the project compiling you can run the project. Select Debug->Start Debugging from Visual Studio. You will now be prompted for an executable file. Browse to the location of your Cinema4D.exe that you installed. Mine is installed here… “C:\Program Files\MAXON\CINEMA 4D R11.5″. Make sure to select “Cinema4d.exe” and NOT the 64bit version.
Cinema4D should now be running. If you got an error about Irrlicht.dll then make sure you have copied it to the same location as your Cinema4D.exe
From the Plugins menu choose “IrrlichtViewer->LoadQuake”. Move the viewport around to get a better view and you should now see something like the following image

You will notice in this image that you don’t see any lightmaps in the opengl viewport. OpenGL in Cinema4d does not yet do any mixing of textures. To see what the end result with the lightmaps blended will look like choose Render->RenderView and you will see something like the following…

To see how this is setup we first look at the objects tags…

Each Polygon Object that is loaded in from Irrlicht has 2 Texture tags, 2 UV tags and 1 Normals Tag. These are loaded in in a particular order, that being the first Diffuse Color texture tag, then the UVs that should be used for that texture tag are next to it. After that we have the Lightmap texture tag followed by its UVs and finally the Normals Tag. If you click once on the Lightmap texture tag you will then see the following in the Attribute panel in Cinema4D.

The important thing to notice here is that the Mix Textures tag has been turned on. This will mix this texture with the previous one (that being the Diffuse Color). Now double click on the Lightmap texture tag and you will see the following in the Attributes panel…

Here you will see the Mix Mode for this texture. It is currently set to Normal. Other options are Add, Subtract and Multiply. I found that Normal gives the closest result to the what is rendered in Irrlicht (which we will come to next).

View the Level in Irrlicht

Now that we have the level loaded into Cinema4D you could happily start changing the textures, painting things, adding or removing geometry. But eventually you will want to see what its going to look like inside of Irrlicht. To do first select everything in the cinema4D viewport by pressing Cntrl-A. With everything selected go to the Plugins menu and choose “IrrlichtViewer->Run Game”. A new window will appear with the level loaded into a Irrlicht scene. You can fly around the scene using your mouse and the Arrow keys on your keyboard. When you are finished press the ESC key to close the window. It should look something like the following…

Now thats about all there is to show at the moment. You can also create any other polygonal object and add a material with a diffuse texture and just select that and through to be viewed in Irrlicht as well.
Now we move onto how this was actually all done. There are 2 parts to this, those being the Loader and the RunGame commands. First we will look at the loader.

The LoadQuake Command

Firstly there is the command itself, which is not that much. This just has an execute method and of course it is registered in the main.cpp file.

class LoadQuakeCommandData : public CommandData
{
public:
 virtual Bool Execute(BaseDocument* doc);
};
Bool RegisterQuakeLoaderCommandData()
{
 //Note that I should really get another ID from the maxon plugin cafe here rather than go + 1
 return RegisterCommandPlugin(ID_RUNGAME_CMD+1, "LoadQuake", 0, "", "LoadQuake", gNew LoadQuakeCommandData());
}

The Execute method is where all the exciting stuff happens. Firstly this creates an IrrlichtDevice, sets it up loads in the Quake pk3 file from the media directory located in the irrlichtviewers plugin folder.

Bool LoadQuakeCommandData::Execute(BaseDocument* doc)
{
 IrrlichtDevice *device = createDevice( video::EDT_NULL, dimension2d(640, 480), 16, false, false, false);

 if (!device) return FALSE;

 IVideoDriver* driver = device->getVideoDriver();
 ISceneManager* smgr = device->getSceneManager();
 gui::IGUIEnvironment* gui = device->getGUIEnvironment();

 //add our private media directory to the file system

 Filename fileName = GeGetPluginPath() + Filename("media");

 char path[1024];
 fileName.GetString().GetCString(path,1024);
 device->getFileSystem()->addFolderFileArchive(path);

 /*
 To display the Quake 3 map, we first need to load it. Quake 3 maps
 are packed into .pk3 files, which are nothing other than .zip files.
 So we add the .pk3 file to our FileSystem. After it was added,
 we are able to read from the files in that archive as they would
 directly be stored on disk.
 */
 device->getFileSystem()->addZipFileArchive ( "map-20kdm2.pk3" );
 scene::IQ3LevelMesh* mesh = (scene::IQ3LevelMesh*) smgr->getMesh("maps/20kdm2.bsp");

Now for each piece of geometry in the mesh we create a new polygonal object in Cinema4D and create a normals tag for it.

scene::ISceneNode* node = 0;
 if ( mesh )
 {
 scene::IMesh *geometry = mesh->getMesh(quake3::E_Q3_MESH_GEOMETRY);

 u32 geom;
 for(geom=0;geomgetMeshBufferCount();geom++)
 {
 IMeshBuffer* buffer = geometry->getMeshBuffer(geom);

 u32 vertcount = buffer->getVertexCount();
 u32 indexCount = buffer->getIndexCount();

 //Find the polygon count
 u32 polyCount = indexCount / 3;

 //Create the polygon object
 PolygonObject *newPolyObj = (PolygonObject *)BaseObject::Alloc(Opolygon);
 newPolyObj->ResizeObject(vertcount,polyCount);

 Vector *newPoints = newPolyObj->GetPointW();
 CPolygon *newPolys = newPolyObj->GetPolygonW();

 //Create a normals tag
 VariableTag *nrmtag = NULL;
 SWORD *normalData = NULL;
 nrmtag = newPolyObj->MakeVariableTag(Tnormal,polyCount);
 if(nrmtag)
 {
 normalData = (SWORD*)nrmtag->GetDataAddressW();
 }

Now we load in all the materials and textures into Cinema4D. One thing to note here is that all the textures are created in memory. Nothing is ever saved to disk during the entire Loading of the Quake level to Running it in the IrrlichtViewer.

//Create materials and textures
 if(buffer)
 {
 UVWTag *uvwA = NULL;
 UVWTag *uvwB = NULL;

 Vector textureTrans;
 Vector textureRot;
 Vector textureScale;

 video::SMaterial &material = buffer->getMaterial();
 LONG matCount=0;
 for(matCount=MATERIAL_MAX_TEXTURES-1;matCount>=0;matCount--)
 {
 video::ITexture *pTexture = material.getTexture(matCount);

 if(pTexture)
 {
 //Create a new material for each texture since  they have different UVs. Mix the materials together using the "Mix  Texture" parameter on a TextureTag
 Material *newMat = Material::Alloc();
 doc->InsertMaterial(newMat);

 if(matCount==0)
 {
 //The first material will use the UVs from the the first UVTag.
 uvwA = (UVWTag*)newPolyObj->MakeVariableTag(Tuvw,polyCount);
 }
 else if(matCount == 1 && buffer->getVertexType() == EVT_2TCOORDS)
 {
 //The second material will use the UVs from the second UVTag
 uvwB = (UVWTag*)newPolyObj->MakeVariableTag(Tuvw,polyCount);
 }

 dimension2d size = pTexture->getSize();
 const stringc &name = pTexture->getName();

 //Set the data for the texture
 BaseContainer bc;
 bc.SetLong(TEXTURE_FILEFORMAT,FILTER_BMP);
 bc.SetLong(TEXTURE_WIDTH, size.Width);
 bc.SetLong(TEXTURE_HEIGHT, size.Height);
 bc.SetLong(TEXTURE_MODE, MODE_RGB);

 //Create a new material in Cinema4D
 newMat->SetChannelState(CHANNEL_COLOR,TRUE);
 newMat->SetName(name.c_str());

 //Get the color channel for the new material
 BaseChannel* colorChannel = newMat->GetChannel(CHANNEL_COLOR);

 //Assign the texture name to the material so the material can reference the image.
 BaseContainer bla;
 bla.SetString(BASECHANNEL_TEXTURE,name.c_str());
 colorChannel->SetData(bla);

 //See if we have already loaded this texture
 PaintTexture* newTexture = NULL;
 GeListHead *head = PaintTexture::GetPaintTextureHead();
 if(head)
 {
 PaintTexture *headTexture = (PaintTexture *)head->GetFirst();
 while(headTexture)
 {
 if(headTexture->GetFilename() == Filename(name.c_str()))
 {
 newTexture = headTexture;
 break;
 }
 headTexture = (PaintTexture *)headTexture->GetNext();
 }
 }

 //If no texture was previously loaded then create a new one
 if(!newTexture)
 {
 newTexture = PaintTexture::CreateNewTexture(name.c_str(), bc);

 core::matrix4& textMatrix = material.getTextureMatrix(matCount);
 vector3df trans = textMatrix.getTranslation();
 vector3df rot = textMatrix.getRotationDegrees();
 vector3df scale = textMatrix.getScale();

 textureTrans.x = trans.X;
 textureTrans.y = trans.Y;
 textureTrans.z = trans.Z;

 textureRot.x = Rad(Real(rot.X));
 textureRot.y = Rad(Real(rot.Y));
 textureRot.z = Rad(Real(rot.Z));

 textureScale.x = scale.X;
 textureScale.y = scale.Y;
 textureScale.z = scale.Z;

 u32 bytesPerRow = pTexture->getPitch();
 ECOLOR_FORMAT format = pTexture->getColorFormat();

 UCHAR *pixelData = (UCHAR *)pTexture->lock(true);

 IImage* tempImage =  driver->createImageFromData(format,size,pixelData,true,false); //Use  the pointer for the data but don't delete it when the image is dropped

 //Get the first layer for the new image
 PaintLayerBmp *paintBmp  = NULL;
 PaintLayer *layer = newTexture->GetFirstLayer();
 paintBmp = (PaintLayerBmp*)layer;
 paintBmp->SetName(name.c_str());

 //Write the pixels from the Irrlicht image to the C4D PaintLayerBmp;
 u32 x,y;
 for(x=0;x<(u32)size.Width;x++)
 {
 for(y=0;y<(u32)size.Height;y++)
 {
 video::SColor col = tempImage->getPixel(x,y);

 PIX buffer[3];
 buffer[0] = (UCHAR)col.getRed();
 buffer[1] = (UCHAR)col.getGreen();
 buffer[2] = (UCHAR)col.getBlue();

 paintBmp->SetPixelCnt(x,y,1,buffer,3,MODE_RGB,0);
 }
 }

 pTexture->unlock();
 }

This is where the Mix Flag is set for the texture Tag so that the light maps display correctly when the image is rendered.

//Create the texture tag and add it to the polygon object.
 //Tell the texture to MIX the color if it has a lightmap (which is the secound material in our case)
 {
 TextureTag *texTag = (TextureTag *)newPolyObj->MakeTag(Ttexture);
 texTag->SetMaterial(newMat);

 //Have a look in the file ttexture.h for a list of other flags
 //C:\Program Files\MAXON\CINEMA 4D R11.5\resource\res\description\ttexture.h
 BaseContainer texTagData;
 texTagData.SetReal(TEXTURETAG_SIDE,TEXTURETAG_SIDE_FRONTANDBACK);
 texTagData.SetBool(TEXTURETAG_TILE,TRUE);
 texTagData.SetVector(TEXTURETAG_POSITION,textureTrans);
 texTagData.SetVector(TEXTURETAG_SIZE,textureScale);
 texTagData.SetVector(TEXTURETAG_ROTATION,textureRot);
 texTagData.SetLong(TEXTURETAG_PROJECTION, TEXTURETAG_PROJECTION_UVW);

 if(matCount==1)
 {
 texTagData.SetBool(TEXTURETAG_MIX,TRUE);
 }
 texTag->SetData(texTagData);
 }

 //Tell the material to update everything
 newMat->Message(MSG_UPDATE);
 newMat->Update(TRUE, TRUE);
 }
 }

After that we load in all the Vertices for the geometry and copy then over to the Cinema4D polygonal Object

//Copy across the vertex data to the polygon object.
 if(buffer->getVertexType()==EVT_STANDARD)
 {
 S3DVertex *vertices = (S3DVertex*)buffer->getVertices();
 u32 vertIndex;
 for(vertIndex=0;vertIndex
 {
 const S3DVertex &theVert = vertices[vertIndex];
 newPoints[vertIndex].x = theVert.Pos.X;
 newPoints[vertIndex].y = theVert.Pos.Y;
 newPoints[vertIndex].z = theVert.Pos.Z;
 }
 }
 else if(buffer->getVertexType()==EVT_2TCOORDS)
 {
 S3DVertex2TCoords *vertices = (S3DVertex2TCoords*)buffer->getVertices();
 u32 vertIndex;
 for(vertIndex=0;vertIndex
 {
 const S3DVertex2TCoords &theVert = vertices[vertIndex];
 newPoints[vertIndex].x = theVert.Pos.X;
 newPoints[vertIndex].y = theVert.Pos.Y;
 newPoints[vertIndex].z = theVert.Pos.Z;
 }
 }
 else if(buffer->getVertexType()==EVT_TANGENTS)
 {
 S3DVertexTangents *vertices = (S3DVertexTangents*)buffer->getVertices();
 u32 vertIndex;
 for(vertIndex=0;vertIndex
 {
 const S3DVertexTangents &theVert = vertices[vertIndex];
 newPoints[vertIndex].x = theVert.Pos.X;
 newPoints[vertIndex].y = theVert.Pos.Y;
 newPoints[vertIndex].z = theVert.Pos.Z;
 }
}

And lastly we create the UVs and Normals for the polygonal objects

//Set the normals and UVs for the polygon object
 switch(buffer->getIndexType())
 {
 case video::EIT_16BIT:
 {
 u16 *indices = (u16 *)buffer->getIndices();
 u32 index;
 u32 polyCount = 0;
 for(index=0;index
 {
 newPolys[polyCount].a = indices[index];
 newPolys[polyCount].b = indices[index+1];
 newPolys[polyCount].d = newPolys[polyCount].c = indices[index+2];

 //If its the standard type then we need to cast to S3DVertex
 if(buffer->getVertexType()== EVT_STANDARD && uvwA)
 {
 S3DVertex *vertices = (S3DVertex*)buffer->getVertices();

 UVWStruct uv;

 const S3DVertex &theVertA = vertices[indices[index]];
 const S3DVertex &theVertB = vertices[indices[index+1]];
 const S3DVertex &theVertC = vertices[indices[index+2]];

 //Set the UVs for the first UV set (the diffuse color)
 uv.a.x = theVertA.TCoords.X;
 uv.a.y = theVertA.TCoords.Y;

 uv.b.x = theVertB.TCoords.X;
 uv.b.y = theVertB.TCoords.Y;

 uv.c.x = theVertC.TCoords.X;
 uv.c.y = theVertC.TCoords.Y;

 uvwA->SetSlow(polyCount,uv);

 //Set the normals
 if(normalData)
 {
 StoreNormal(normalData+12*polyCount+0,Vector(theVertA.Normal.X,theVertA.Normal.Y,theVertA.Normal.Z));
 StoreNormal(normalData+12*polyCount+3,Vector(theVertB.Normal.X,theVertB.Normal.Y,theVertB.Normal.Z));
 StoreNormal(normalData+12*polyCount+6,Vector(theVertC.Normal.X,theVertC.Normal.Y,theVertC.Normal.Z));
 StoreNormal(normalData+12*polyCount+9,Vector(theVertC.Normal.X,theVertC.Normal.Y,theVertC.Normal.Z));
 }
 }

 //If its the standard type then we need to cast to S3DVertex
 else if(buffer->getVertexType()==EVT_2TCOORDS)
 {

 UVWStruct uv;
 S3DVertex2TCoords *vertices = (S3DVertex2TCoords*)buffer->getVertices();
 const S3DVertex2TCoords &theVertA = vertices[indices[index]];
 const S3DVertex2TCoords &theVertB = vertices[indices[index+1]];
 const S3DVertex2TCoords &theVertC = vertices[indices[index+2]];

 //UV A (used by the Diffuse Color texture)
 if(uvwA)
 {
 uv.a.x = theVertA.TCoords.X;
 uv.a.y = theVertA.TCoords.Y;

 uv.b.x = theVertB.TCoords.X;
 uv.b.y = theVertB.TCoords.Y;

 uv.c.x = theVertC.TCoords.X;
 uv.c.y = theVertC.TCoords.Y;

 uvwA->SetSlow(polyCount,uv);
 }

 //UV B (used byt the Light map texture)
 if(uvwB)
 {
 uv.a.x = theVertA.TCoords2.X;
 uv.a.y = theVertA.TCoords2.Y;

 uv.b.x = theVertB.TCoords2.X;
 uv.b.y = theVertB.TCoords2.Y;

 uv.c.x = theVertC.TCoords2.X;
 uv.c.y = theVertC.TCoords2.Y;

 uvwB->SetSlow(polyCount,uv);
 }

 //Set the normals
 if(normalData)
 {
 StoreNormal(normalData+12*polyCount+0,Vector(theVertA.Normal.X,theVertA.Normal.Y,theVertA.Normal.Z));
 StoreNormal(normalData+12*polyCount+3,Vector(theVertB.Normal.X,theVertB.Normal.Y,theVertB.Normal.Z));
 StoreNormal(normalData+12*polyCount+6,Vector(theVertC.Normal.X,theVertC.Normal.Y,theVertC.Normal.Z));
 StoreNormal(normalData+12*polyCount+9,Vector(theVertC.Normal.X,theVertC.Normal.Y,theVertC.Normal.Z));
 }
 }

 polyCount++;
 }
 }
 break;
 case EIT_32BIT:
 {
 u32 *indices = (u32*)buffer->getIndices();
 }
 break;
 default:
 break;
 }   
 doc->InsertObject(newPolyObj,0,0);
 newPolyObj->Message(MSG_UPDATE);
 }
 }
 }

 device->drop();

 EventAdd();

 return TRUE;
}

Next we will look at the Irrlicht Viewer code..

Irrlicht Viewer Code

Now that we have all the data loaded the next thing we want to do is open up a Irrlicht Viewer and view the scene as it will appear in any game we might make.
Firstly there is the RunGameCommand. This is a bit different from the other load command because we want to run the game in its own thread. So to do this we will use Cinema4D thread. In this case we have created a new class called GameThread that is amember of the RunGameCommand.

//===========================================================================
// This is the command to export all the data to Irrlicht and run it in its own thread.
//===========================================================================
class RunGameCommandData : public CommandData
{
public:
 virtual Bool Execute(BaseDocument* doc);

private:
 GameThread gameThread;
};

Bool RunGameCommandData::Execute(BaseDocument* doc)
{
 gameThread.theDocToView->SetLink(doc);
 return gameThread.Start(TRUE);
}

Bool RegisterRunGameCommandData()
{
 return RegisterCommandPlugin(ID_RUNGAME_CMD, "Run Game", 0, "", "Run Game", gNew RunGameCommandData());
}

The GameThread class is derived from C4DThread and also IEventReciever. C4DThread has a main method which starts the thread. The IEventReceiver is part of Irrlicht and has an OnEvent virtual method that we can use to detect the ESC key so that we can quit out of the game.

//===========================================================================
// To run the Irrlicht scene we run it in our own thread.
// This also has an IEventReciever so that the user can press the Escape key to exit the scene
//===========================================================================
class GameThread : public C4DThread, public IEventReceiver
{
public:
 GameThread();
 virtual ~GameThread();

 virtual void Main(void);
 virtual const CHAR* GetThreadName(void);

private:
 virtual bool OnEvent(const SEvent& event);

 void Init();
 void DoRecursion(BaseObject *op);
 void AddObject(BaseObject *obj);
 void AddLight(BaseObject *obj);
 void Run();

public:
 AutoAlloc theDocToView;

private:
 void ViewObject(BaseObject *obj);
 IrrlichtDevice *device;
};

The main Method loops over all the selected object in the cinema4d scene, using a special recursive method, and adds all the polygonal objects to an Irrlicht scene.

// This method finds all selected objects in the scene and exports then to Irrlicht
void GameThread::Main(void)
{
 Init();

 BaseDocument *doc = (BaseDocument *)theDocToView->ForceGetLink();

 AutoAlloc selection;
 doc->GetActiveObjects(selection,false);

 LONG a;
 Bool found = FALSE;
 for(a=0;aGetCount();a++)
 {
 C4DAtom* atom = selection->GetIndex(a);
 if(atom->IsInstanceOf(Obase))
 {
 DoRecursion((BaseObject*)atom);
 }
 else if(atom->IsInstanceOf(Olight))
 {
 BaseObject *obj = (BaseObject *)atom;
 AddLight(obj);
 }
 }

 Run(); //Finally run the scene
}

void GameThread::DoRecursion(BaseObject *op)
{
 BaseObject *tp = op->GetDeformCache();
 if (tp)
 {
 DoRecursion(tp);
 }
 else
 {
 tp = op->GetCache(NULL);
 if (tp)
 {
 DoRecursion(tp);
 }
 else
 {
 if (!op->GetBit(BIT_CONTROLOBJECT))
 {
 if (op->IsInstanceOf(Opolygon))
 {
 AddObject(op);
 }
 }
 }
 }
 for (tp = op->GetDown(); tp; tp=tp->GetNext())
 {
 DoRecursion(tp);
 }
}

AddObject is where all the magic happens for this command. First we get the polygon object from Cinema4D, check how many UV Tags it has and then create a mesh in Irrlicht

//Add a polygonal object to the scene including multiple UV sets, materials and textures.
void GameThread::AddObject(BaseObject *obj)
{
 if(!obj) return;

 LONG a;

 //Find the polyon object and the uv sets if they exist
 BaseDocument *doc = GetActiveDocument();
 PolygonObject *pPoly = ToPoly(obj);
 UVWTag *uvwA = (UVWTag *)pPoly->GetTag(Tuvw,0);
 UVWTag *uvwB = (UVWTag *)pPoly->GetTag(Tuvw,1);

 //Create a mesh in Irrlicht
 ISceneManager* smgr = device->getSceneManager();
 IVideoDriver* driver = device->getVideoDriver();   
 scene::SMesh *mesh = new scene::SMesh();
 mesh->setMaterialFlag(EMF_BACK_FACE_CULLING,FALSE);

If we have two UV sets then we need to create a scene::SMeshBufferLightMap object to add to the irrlicht scene. To begin with we then count the number of triangles in the scene. Since Cinema4D has quads as well as triangles we check to see if it shares indices, if it does that means its only a 3 sided triangle. Later on if we found any quads we cut them up and add then as triangles to the scene.

//If there are 2 uv sets then we should have 2 materials and 2  textures. The first being the diffuse color and the second the light map
 if(uvwA && uvwB)
 {
 scene::SMeshBufferLightMap *sbuffer = new scene::SMeshBufferLightMap();

 const Vector *pPoints = pPoly->GetPointR();

 //Count the number of triangles in the scene. Cinema4D also  supports quads so we need to cut these into triangles when we export to  Irrlicht
 LONG count = 0;
 const CPolygon *pPolys = pPoly->GetPolygonR();
 for(a=0;aGetPolygonCount();++a)
 {
 const CPolygon &poly = pPolys[a];

 if(poly.c == poly.d)
 {
 count += 3;
 }
 else
 {
 count += 4;
 }
 }    

 sbuffer->Indices.reallocate(count);
 sbuffer->Vertices.reallocate(count);

 //Get the normal tag if it exists. If it doesn't exist we will create normals ourselves
 VariableTag *nrmtag = (VariableTag*)pPoly->GetTag(Tnormal);

 SWORD *normalData = NULL;
 if(nrmtag)
 {
 normalData = (SWORD*)nrmtag->GetDataAddressW();
 }

 Vector normalA, normalB, normalC, normalD;

Now we add all the polygons to the scene, making sure to cut up any quads into triangles as we do so. We will also set the normals uvs for the polygons in the scene.

//Add all the triangles to the scene
 u16 vertCount = 0;
 for(a=0;aGetPolygonCount();++a)
 {
 const CPolygon &poly = pPolys[a];

 UVWStruct uvsA;
 if(uvwA)
 {
 uvsA = uvwA->GetSlow(a);
 }
 UVWStruct uvsB;
 if(uvwB)
 {
 uvsB = uvwB->GetSlow(a);
 }

 if(normalData)
 {
 GetNormal(normalData+12*a+0,normalA);
 GetNormal(normalData+12*a+3,normalB);
 GetNormal(normalData+12*a+6,normalC);
 GetNormal(normalData+12*a+9,normalD);
 }

 sbuffer->Indices.push_back(vertCount++);
 sbuffer->Indices.push_back(vertCount++);
 sbuffer->Indices.push_back(vertCount++);

 //If no normals then create some
 if(!normalData)
 {
 Vector normal = (pPoints[poly.b] - pPoints[poly.a]) %(pPoints[poly.c] - pPoints[poly.a]);
 normal.Normalize();               
 normalA = normalB = normalC = normal;
 }

 sbuffer->Vertices.push_back(video::S3DVertex2TCoords(pPoints[poly.a].x,  pPoints[poly.a].y,  pPoints[poly.a].z,normalA.x,normalA.y,normalA.z,video::SColor(255,255,255,255),uvsA.a.x,uvsA.a.y,uvsB.a.x,uvsB.a.y));
 sbuffer->Vertices.push_back(video::S3DVertex2TCoords(pPoints[poly.b].x,  pPoints[poly.b].y,  pPoints[poly.b].z,normalB.x,normalB.y,normalB.z,video::SColor(255,255,255,255),uvsA.b.x,uvsA.b.y,uvsB.b.x,uvsB.b.y));
 sbuffer->Vertices.push_back(video::S3DVertex2TCoords(pPoints[poly.c].x,  pPoints[poly.c].y,  pPoints[poly.c].z,normalC.x,normalC.y,normalC.z,video::SColor(255,255,255,255),uvsA.c.x,uvsA.c.y,uvsB.c.x,uvsB.c.y));

 //This is a quad so we need to export a second triangle
 if(poly.c != poly.d)
 {
 //If no normals then create some
 if(!normalData)
 {
 Vector normal = (pPoints[poly.c] - pPoints[poly.a]) %(pPoints[poly.d] - pPoints[poly.a]);
 normal.Normalize();                   
 normalA = normalC = normalD = normal;
 }

 sbuffer->Indices.push_back(vertCount++);
 sbuffer->Indices.push_back(vertCount++);
 sbuffer->Indices.push_back(vertCount++);

 sbuffer->Vertices.push_back(video::S3DVertex2TCoords(pPoints[poly.a].x,  pPoints[poly.a].y,  pPoints[poly.a].z,normalA.x,normalA.y,normalA.z,video::SColor(255,255,255,255),uvsA.a.x,uvsA.a.y,uvsB.a.x,uvsB.a.y));
 sbuffer->Vertices.push_back(video::S3DVertex2TCoords(pPoints[poly.c].x,  pPoints[poly.c].y,  pPoints[poly.c].z,normalC.x,normalC.y,normalC.z,video::SColor(255,255,255,255),uvsA.c.x,uvsA.c.y,uvsB.c.x,uvsB.c.y));
 sbuffer->Vertices.push_back(video::S3DVertex2TCoords(pPoints[poly.d].x,  pPoints[poly.d].y,  pPoints[poly.d].z,normalD.x,normalD.y,normalD.z,video::SColor(255,255,255,255),uvsA.d.x,uvsA.d.y,uvsB.d.x,uvsB.d.y));
 }
 }   
 sbuffer->recalculateBoundingBox();
 mesh->addMeshBuffer(sbuffer);
 sbuffer->drop();
 }

If there was only 1 UV tag on the polygon object then we need to do things slightly differently, but the code is almost exactly the same as what was given above.

else if(uvwA && !uvwB) //No second UV Set. Virtually the same code as above.
 {
 scene::SMeshBuffer *sbuffer = new scene::SMeshBuffer();
 const Vector *pPoints = pPoly->GetPointR();

 LONG count = 0;
 const CPolygon *pPolys = pPoly->GetPolygonR();
 for(a=0;aGetPolygonCount();++a)
 {
 const CPolygon &poly = pPolys[a];

 if(poly.c == poly.d)
 {
 count += 3;
 }
 else
 {
 count += 4;
 }
 }    

 sbuffer->Indices.reallocate(count);
 sbuffer->Vertices.reallocate(count);

 VariableTag *nrmtag = (VariableTag*)pPoly->GetTag(Tnormal);

 SWORD *normalData = NULL;
 if(nrmtag)
 {
 normalData = (SWORD*)nrmtag->GetDataAddressW();
 }

 Vector normalA, normalB, normalC, normalD;

 u16 vertCount = 0;
 for(a=0;aGetPolygonCount();++a)
 {
 const CPolygon &poly = pPolys[a];

 UVWStruct uvsA;
 if(uvwA)
 {
 uvsA = uvwA->GetSlow(a);
 }
 UVWStruct uvsB;
 if(uvwB)
 {
 uvsB = uvwB->GetSlow(a);
 }

 if(normalData)
 {
 GetNormal(normalData+12*a+0,normalA);
 GetNormal(normalData+12*a+3,normalB);
 GetNormal(normalData+12*a+6,normalC);
 GetNormal(normalData+12*a+9,normalD);
 }

 sbuffer->Indices.push_back(vertCount++);
 sbuffer->Indices.push_back(vertCount++);
 sbuffer->Indices.push_back(vertCount++);

 if(!normalData)
 {
 Vector normal = (pPoints[poly.b] - pPoints[poly.a]) %(pPoints[poly.c] - pPoints[poly.a]);
 normal.Normalize();               
 normalA = normalB = normalC = normal;
 }

 sbuffer->Vertices.push_back(video::S3DVertex(pPoints[poly.a].x,  pPoints[poly.a].y,  pPoints[poly.a].z,normalA.x,normalA.y,normalA.z,video::SColor(255,255,255,255),uvsA.a.x,uvsA.a.y));
 sbuffer->Vertices.push_back(video::S3DVertex(pPoints[poly.b].x,  pPoints[poly.b].y,  pPoints[poly.b].z,normalB.x,normalB.y,normalB.z,video::SColor(255,255,255,255),uvsA.b.x,uvsA.b.y));
 sbuffer->Vertices.push_back(video::S3DVertex(pPoints[poly.c].x,  pPoints[poly.c].y,  pPoints[poly.c].z,normalC.x,normalC.y,normalC.z,video::SColor(255,255,255,255),uvsA.c.x,uvsA.c.y));

 if(poly.c != poly.d)
 {
 if(!normalData)
 {
 Vector normal = (pPoints[poly.c] - pPoints[poly.a]) %(pPoints[poly.d] - pPoints[poly.a]);
 normal.Normalize();                   
 normalA = normalC = normalD = normal;
 }

 sbuffer->Indices.push_back(vertCount++);
 sbuffer->Indices.push_back(vertCount++);
 sbuffer->Indices.push_back(vertCount++);

 sbuffer->Vertices.push_back(video::S3DVertex(pPoints[poly.a].x,  pPoints[poly.a].y,  pPoints[poly.a].z,normalA.x,normalA.y,normalA.z,video::SColor(255,255,255,255),uvsA.a.x,uvsA.a.y));
 sbuffer->Vertices.push_back(video::S3DVertex(pPoints[poly.c].x,  pPoints[poly.c].y,  pPoints[poly.c].z,normalC.x,normalC.y,normalC.z,video::SColor(255,255,255,255),uvsA.c.x,uvsA.c.y));
 sbuffer->Vertices.push_back(video::S3DVertex(pPoints[poly.d].x,  pPoints[poly.d].y,  pPoints[poly.d].z,normalD.x,normalD.y,normalD.z,video::SColor(255,255,255,255),uvsA.d.x,uvsA.d.y));
 }
 }   
 sbuffer->recalculateBoundingBox();
 mesh->addMeshBuffer(sbuffer);
 sbuffer->drop();
 }

Now we setup the mesh as a scenenode and add it to scene manager in Irrlicht

mesh->recalculateBoundingBox();

 //Set the position of the mesh in the scene
 IMeshSceneNode* node = smgr->addMeshSceneNode( mesh );

 Vector pos = obj->GetPos() * obj->GetUpMg(); //Location in world coordinates
 node->setPosition(vector3df(pos.x,pos.y,pos.z));

 Vector rotation = obj->GetRot() * obj->GetUpMg();
 node->setRotation(vector3df(Deg(rotation.x),Deg(rotation.y),Deg(rotation.z)));

 Vector scale = obj->GetScale() * obj->GetUpMg();
 node->setScale(vector3df(scale.x,scale.y,scale.z));

 //Set its material type if it has a light map or not
 if(uvwA && !uvwB)
 {
 node->setMaterialType(video::EMT_SOLID);
 }
 else if(uvwA && uvwB)
 {
 node->setMaterialType(video::EMT_LIGHTMAP_M4);
 }

Then finally we copy across all the textures. Note that I am not checking if a texture has already been used in the Irrlicht scene so this is currently wasting more memory than is required.

if (node)
 {
 Filename docpath = GetActiveDocument()->GetDocumentPath();

 LONG textureCount = 0;
 TextureTag  *tex=(TextureTag*)obj->GetTag(Ttexture,textureCount);
 while(tex)
 {
 BaseMaterial *mat = tex->GetMaterial();
 if(mat)
 {
 BaseChannel* diffuseColor = mat->GetChannel(CHANNEL_COLOR);
 if(diffuseColor)
 {
 PaintTexture *theTexture = PaintTexture::GetPaintTextureOfBaseChannel(doc,diffuseColor);
 if(theTexture)
 {
 PaintLayer *layer = theTexture->GetFirstLayer();
 PaintLayerBmp *paintBmp = (PaintLayerBmp*)layer;

 video::IImage *image =  driver->createImage(video::ECF_R8G8B8,core::dimension2di(paintBmp->GetBw(),paintBmp->GetBh()));

 LONG x,y;
 for(x=0;xGetBw();x++)
 {
 for(y=0;yGetBh();y++)
 {
 PIX buffer[3];
 paintBmp->GetPixelCnt(x,y,1,buffer,MODE_RGB,0);
 image->setPixel(x,y,video::SColor(255,buffer[0],buffer[1],buffer[2]));
 }
 }

 video::ITexture *newTex = driver->addTexture(mat->GetName().GetCStringCopy(),image);
 node->setMaterialTexture(textureCount,newTex);
 }
 }
 }
 tex=(TextureTag*)obj->GetTag(Ttexture,++textureCount);
 }
 }
}

There is also a method to Initialize the Irrlicht scene

//Initialize Irrlicht
void GameThread::Init()
{
 device = createDevice( video::EDT_OPENGL, dimension2d(640, 480), 16, false, false, false, this);

 if (!device) return;

 device->setWindowCaption(L"Cinema4D Irrlicht Demo");

 IVideoDriver* driver = device->getVideoDriver();
 ISceneManager* smgr = device->getSceneManager();

 //Turn off all Clipping
 for(int a=0; a < 6; a++)
 {
 driver->enableClipPlane(a,false);
 }

 scene::ICameraSceneNode *cameraNode = smgr->addCameraSceneNodeFPS();
 cameraNode->setTarget(vector3df(0,0,0));
 cameraNode->setPosition(vector3df(200,200,200));

 smgr->setAmbientLight(video::SColorf(1.0,1.0,1.0));

 device->getCursorControl()->setVisible(false);

}

And finally running the game and the OnEvent callback

//Run the game
void GameThread::Run()
{
 if(!device) return;

 IVideoDriver* driver = device->getVideoDriver();
 ISceneManager* smgr = device->getSceneManager();
 IGUIEnvironment* guienv = device->getGUIEnvironment();

 while(device->run())
 {
 if (device->isWindowActive())
 {
 driver->beginScene(true, true, SColor(255,100,101,140));

 smgr->drawAll();
 guienv->drawAll();

 driver->endScene();
 }
 else
 {
 device->yield();
 }

 device->sleep(1);
 }

 device->drop();

}
 //Check for the escape key to exit the "game"
bool GameThread::OnEvent(const SEvent& event)
{
 if (event.EventType == EET_KEY_INPUT_EVENT && event.KeyInput.PressedDown == false)
 {
 if (event.KeyInput.Key == irr::KEY_ESCAPE)
 {
 if (device)
 {
 device->closeDevice();
 return true;
 }                       
 }
 }
 return false;
}
const CHAR* GameThread::GetThreadName(void)
{
 return "GameThread";
}

There are a couple of other methods that are used to Get and Set the normals for the geometry. These are very specific to Cinema4D.

//Some special code to set and get normals from Cinema4D.
#define normDiv 1.0/32000.0

inline static void GetNormal(SWORD *ptr, Vector &v)
{
 v.x = SWORD(ptr[0]*normDiv);
 v.y = SWORD(ptr[1]*normDiv);
 v.z = SWORD(ptr[2]*normDiv);
}

inline static void StoreNormal(SWORD *ptr, Vector v)
{
 ptr[0] = SWORD(v.x*32000.0);
 ptr[1] = SWORD(v.y*32000.0);
 ptr[2] = SWORD(v.z*32000.0);
}

And thats all there is to it!

Conclusion

In this tutorial you learned how to load in a Quake Level using Irrlicht to load in the geometry and then transfer it through to Cinema4D. You also learned how to take the geometry from Cinema4D and export that out again to be viewed in the Irrlicht Viewer. This does not only work with Quake levels though, you can use this to export anything from Cinema4D to be viewed in Irrlicht. And with a few slight changes you could even save out the Irrlicht scene to disk and use that in a stand alone game if you so wished.
Where to from here?
You could build upon this system and add some of your own Tags to Cinema4D. For example you could add a Tag that would tell this exporter that the polygon object is actually the starting location for the player character, then when the game is run the player starts in a more reasonable place. You could then simply insert one of the standard Quake Characters and run around the level.

You could add more tags for things like Monster locations, triggers, treasures etc… and add them to this exporter to populate your little game with some extra data.

You could setup your level in Cinema4D, preview the game and then export to some custom binary file and use it in a stand alone game engine.

A C++ library for communicating with Amazon S3

Posted on January 20, 2012 by Kent

Introduction

Amazon web services (AWS)has many SDKs available for interacting with its services.

Java
Mobile

Android
iOS

But the one that they don’t support and have no SDK for is C++.
Because of the lack of an official SDK from Amazon many people have written their own C++ code for communicating with the REST api. I decided to have a quick look to see what was out there and compile a version that could be used commercially.
There is a lack (I couldn’t find any) of free to use source code examples, including project files, for use with Windows (my main development environment). I believe this is due to the complexity of having to compile and install so many libraries in order to get started (I believe OSX has some of these libraries installed by default).
So the platform that I am targeting initially is Win32 using Visual Studio 2005 and Visual Studio 2008 Express Edition as the compilers. The results of all this will be a working Visual Studio solution that anyone can download and quickly get up and running using Amazon S3 under a windows development environment.
Note: If you can’t be bothered reading all this thrilling information on compilers and C++ and just want to get straight to work using a working Visual Studio solution that you can run away and compile, then you can skip to the end of this article for a link. But just read the next bit first before you do.

Before you start

Before you rush off in glee and start downloading and compiling code the very first thing you must do is create an Amazon S3 Account. If you don’t have an account then the resulting tool that you are about to compile will not be of any use to you other than to let you marvel at your masterful skills of being able to compile oodles of glorious C++ code.
Once your in then go to the Account section (a link can be found at the top of the AWS Management Console).
Next go the Security Credentials section.
Then under Access Credentials select the link Create a new Access Key.
Make a copy of the “Access Key ID” and store it somewhere.
Lastly press the “Show” link under the Secret Access Key settting, copy this information as well along with your Access Key.
NEVER give anyone your Secret Access Key! Keep it hidden, keep it safe.
OK now onto the good stuff… code.

Current Implementations

There are a number of C++ libraries available on internet.

The one I have decided to have a go at compiling is the one from http://www.arklyffe.com/main/s3/. This is under a BSD licence which means if everything proves to be successfull then you can use the library for commercial purposes as long as you display the licence with the product.

What you need to get started

Most of the above libraries that people have developed require both libcurl and OpenSSL. The one I am using also requires curlpp which is a C++ wrapper for libcurl. So first you will have to download these and get each of them compiling on your system. Some may also use the boost libraries.
If you want to see a small description of how these libraries are related then look here.

Compiling libcURL

Before you can compile libcURL you will first need to compile OpenSSL (well actually this is not strictly true, you can compile without it but for communicating with Amazon S3 we need an implementation of hmac for creating the signatures for the http headers when working with the REST api. This could be done using some other libraries but the s3tools are using OpenSSL).

OpenSSL

First thing to do is download openssl.
OpenSSL requires a few things as well to get going. First you will need perl which you can get from here.
In the OpenSSL directory you will see many INSTALL.XXX text files. The one I am following is INSTALL.W32. I am also using Visual Studio for compiling. Here is a copy of their instructions for compiling OpenSSL under Win32.

OpenSLL Visual C++ Instructions

If you want to compile in the assembly language routines with Visual
C++, then you will need the Netwide Assembler binary,
nasmw.exe or nasm.exe, to be available on your %PATH%.

Firstly you should run Configure with platform VC-WIN32:

> perl Configure VC-WIN32 –prefix=c:\some\openssl\dir

Where the prefix argument specifies where OpenSSL will be installed to.

Next you need to build the Makefiles and optionally the assembly
language files:

- If you are using NASM then run:

> ms\do_nasm

- If you don’t want to use the assembly language files at all then run:

> perl Configure VC-WIN32 no-asm –prefix=c:/some/openssl/dir
> ms\do_ms

If you get errors about things not having numbers assigned then check the
troubleshooting section: you probably won’t be able to compile it as it
stands.

Then from the VC++ environment at a prompt do:

> nmake -f ms\ntdll.mak

If all is well it should compile and you will have some DLLs and
executables in out32dll. If you want to try the tests then do:

> nmake -f ms\ntdll.mak test

To install OpenSSL to the specified location do:

> nmake -f ms\ntdll.mak install

Tweaks:

There are various changes you can make to the Win32 compile
environment. By default the library is not compiled with debugging
symbols. If you use the platform debug-VC-WIN32 instead of VC-WIN32
then debugging symbols will be compiled in.

By default in 1.0.0 OpenSSL will compile builtin ENGINES into the
separate shared librariesy. If you specify the “enable-static-engine”
option on the command line to Configure the shared library build
(ms\ntdll.mak) will compile the engines into libeay32.dll instead.

The default Win32 environment is to leave out any Windows NT specific
features.

If you want to enable the NT specific features of OpenSSL (currently
only the logging BIO) follow the instructions above but call the batch
file do_nt.bat instead of do_ms.bat.

You can also build a static version of the library using the Makefile
ms\nt.mak.

After finished compiling OpenSLL

Now that it is compiled you will find the files you require in the path you set above.
You will have a directory with files in a structure like the following

libeay32.dll
openssl.exe
ssleay32.dll

include

openssl

lots of files here…

engines

lots of files here…

libeay32.lib
ssleay32.lib

openssl.cnf

libcURL

Now that you have OpenSSL compiling the next step is to compile libcURL with support for OpenSSL included.
Download the Win32 – MSVC (with SSL) source from here. Open up the visual studio solution file. Add the following pre processor definitions to the configuration you wish to build

USE_SSLEAY
USE_OPENSSL
BUILDING_LIBCURL

For static libraries also add

CURL_STATICLIB

Add the include directory of your OpenSSL directory to the visual studio project settings.
Add the lib directory of your OpenSSL directory to the visual studio project settings.
Add the two OpenSSL libraries to the project. Those being… libeay32.lib ssleay32.lib
You should now be able to compile the libcURL.
The resulting files that you will need can be found in
libcurl\include
libcurl\lib\DLL-Release\libcurl.dll
libcurl\lib\DLL-Release\libcurl_imp.lib

Compling cURLpp

Download the source code from HERE. Set the include and lib directories to libcurl.
In my version a couple of files were in the wrong place in the project file. Delete the files LifetimeLibrary.cpp and PrivateMembers.cpp and re-add them from the utilspp directory.
Add some pre-processor definitions

CURLPP_STATICLIB
CURL_STATICLIB
BUILDING_CURLPP

Build the static libraries for debug and release.

Compiling the S3Tool code

Unzip the code from here.
Create an empty visual studio console project and add all the following files

aws_s3.cpp
aws_s3.h
aws_s3_misc.cpp
aws_s3_misc.h
multidict.h
s3tools.cpp

Add the libcurl, openssl and curlpp include and lib directories to the project.
Set the preprocessor definitions

CURLPP_STATICLIB
CURL_STATICLIB

You will then have to modifiy the following method
string HTTP_Date()
{
time_t t = time(NULL);
tm gmt; gmtime_s(&gmt,&t);
char bfr[256];
size_t n = strftime(bfr, 256, “%a, %d %b %Y %H:%M:%S GMT”, &gmt);
bfr[n] = ‘\0′;
return bfr;
}
And also remove some code around line 213 that trys to load in user credential files, since it doesn’t compile under windows and isn’t really needed anyway.
Add the libs ws2_32.lib wldap32.lib ssleay32.lib libeay32.lib curlppd.lib libcurld.lib

Conclusion

Now if you can’t be bothered doing all that then you can just download the resulting project that I have put together (see link below).
This has been compiled and tested under Visual Studio 2005 and Visual Studio 2008 Express Edition.
It includes all the source code for libcurl, curlpp and s3tools as well as prebuilt Win32 libraries and dlls for openssl. I have removed alot of the other non-windows files from the libraries in order to strip them down, but you can still find a copy of the licences in the libcurl and curlpp directories (the files are named COPYING).
I have only setup the Win32 build for now but may add a Win64 build at a later date.
The project contains the following configurations and is all compiled with Multithreaded (/MT) flag.

DebugStatic
DebugDynamic
ReleaseStatic
ReleaseDynamic

The static builds only depend on the two dlls from openssl, those being libeay32.dll and ssleay32.dll.
The dynamic builds also require libcurl.dll and curlpp.dll.
When you build the solution you will get the following directory structure.

debugdynamic
debugstatic
releasedynamic
releasestatic

Each of these directories will contain all the required dlls and the s3tool executable.
To use the tool you can either use the command line arguments to feed in a file that contains your key and secret access key (which you need to sign up to AWS to get), or you just edit the main method of s3tool.cpp
Find the following line and enter your details.
keyID = “YOUR_KEY”;
secret = “YOUR_SECRET”;
Finally here is the code for you to download and play with DOWNLOAD HERE.
Special thanks goes to the original developer of the s3tools code that enabled me to gain a better understanding of openssl, libcurl, curlpp and most importantly understanding how to actually create the signature, which was the the whole reason why I started this code hunt in the first place. And of course an extra special thanks to the developers of openssl, libcurl and curlpp that the s3tools code is built upon.

Selecting Your Next Game Engine

Posted on January 20, 2012 by Kent

Note: This article was written in 2009 and some things may be different.
Before you can select your game engine you need to make a few choices. Who is your target audience? Is it PC, PS3, iPhones or anyone with a webbrowser? Then you will need to start looking for technology that you can use to develop for that platform. This may give you a number of different solutions to choose from, after which you then pick one that best suits your needs based on your skill set and requirements.

The game platform(s)

Before looking for the game engine you need to decide what platform (or platforms) you are going to be targeting and how you would like to deliver the game to your user. Firstly lets look at a few delivery mechanisms.

Browser based

You could create your game so that it runs within a browser. This usually has the benefit of being easier to install and allowing the user to access the game from multiple machines. I say “usually” because depending on your engine the user may still have to install some component on their system, that could be Flash, Silverlight, ActiveX controls or some other such thing. And depending on the users browser and security permissions it may be difficult for the user to do.

Flash/Silverlight

Most machines these days trust Flash and have it installed. So for a safe bet you could develop your game so that it can be played in flash. However Silverlight is gaining momentum and I personally prefer Silverlight over Flash because (1) its free to develop for and (2) I already have C# development experience so I don’t have to learn Action Script (and I personally know a lot more developers who also know .Net languages which can also be used to develop in Silverlight). Even though reskilling yourself is good, its also very good to see if you can take advantage of your existing skillset as well (hence my Silverlight over Flash decision).
For those interested you can develop Silverlight applications by downloading the Visual Web Developer 2008 Express Edition.
Since I am not a flash user I can not say what is possible for flash. But for Silverlight you can get access to some storage on the users computer to store extra files and information as a kind of caching if you feel the need. It is called “Isolated Storage”. Flash most likely offers a similar system as well.
Both Silverlight and Flash web apps now also have the ability to let the user run them directly from the users computer without being in a webpage. They also allow you to let the user work offline, although for an MMO this has somewhat limited uses. But this does allow for some more flexibility to how you deploy your application.
Now some of you may be wondering why I am bring up Flash and Silverlight at all in this discussion. One reason is that you may very well decide to forgo the entire 3D route and stick to a nice tiled based 2D engine. But the main reason is that you might like to offer some kind of interactive experience on your website and doing this with Flash or Silverlight communicating to your Game server could offer some interesting ideas (a real time map of where everyone is in the world, running statistics).

ActiveX/Other

Some engines these days offer Web based players (Unity, Esperient, Virtools). These allow you to use the engine with all its 3D abilities running it from within the users Web browser. In reality the application is installed on the users computer and simply runs in a window within the Web browser. They have varying abilities and usually sandbox the software so that the engine can not access the users files direction via the engines API.
You must also check which platform the web player supports. For example some may only support Internet Explorer under windows, if your target audience is Mac users or even Firefox users then you are out of luck. One product that has good support for multiple platforms and browsers is Unity3D.

AJAX

Just to throw in one other way. You could even write your entire MMO in javascript and do asychronous calls to your game server (AJAX). Just do a google search for AJAX MMO, you might be surprised what you find.

3D in a Browser

There are already a few products out on the market today that let you publish 3D games directly to the webbrowser. These will run directly in the browser by using technology such as ActiveX controls. Here are a few companies that currently offer this kind of technology…
Unity3D (Its free for personal and commercial use!)
Esperient Creator
Torque3D
Remember also that 3D in the web browser is just around the corner with HTML5 and WebGL. And with the ability of projects such as Project Darkstar offering a Javascript based client to communicate with a Project Darkstar Server we could see more 3D based MMO games being delivered straight into your browser using 3D graphics.

Mobile devices (iPhone, iPodTouch, Android, Cell Phones, Zune).

There may also be a future in using Mobile phones to play MMO games (well maybe just an MO since it might not be able to scale to something Massive on a phone just yet). It is certainly possible to use RakNet (a game networking engine) with an iPhone and a custom back end server to create an MMO. You could also look at what Google are doing with the Android platform. You can make games for Zune using XNA Studio by microsoft so why not a MOG?
To further investigate the iPhone path I can suggest taking a look at sio2 interactive, its an open source 3D game engine for iPhone and iPodTouch. Perhaps you could hook this up with RakNet or Project Darkstar to make a 3D MMO. If you have some money you could also just use something like Unity3D and publish to the iPhone ( as well as other platforms). You could also use Photon and Neutron to have a complete networking solution developing with iPhone, Android, Unity, Win32, .NET, J2ME, BREW and Flash.

Consoles

If your dream is to make an MMO that runs on Playstation 3, XBox360 or the Wii then there are also a few good commercial Game Engines that you could use to start up your company. And here I say “start up your company” because unfortunately if you are going to make a game for these platforms then you will need to be a set up commercial company with a solid game idea that you can pitch to some executives at a Game Publishing house (or try going direct to Microsoft, Sony or Nintendo if you are able to), with the exception of indie games for Xbox360 made with XNA studio (more on this later).
Unreal Engine (XBox360 and PS3)
Unity3D (Wii)
CryEngine3 ( XBox360, PS3)
And then of course you can always build your own Game Engine for these consoles as well (more about this later).

Xbox360 XNA Studio

Now there is one exception to this rule and that is the Xbox360. Microsoft released the XNA Game Studio that you can use to develop games that get published directly to the Xbox360 and you can share them with the Xbox360 community in the Xbox Live Arcade. Note that with XNA you can also publish to the Zune and PC so its not a bad platform for beginning programmers (and you get to develop using C# which is a really nice language).

PC/Mac/Linux based game engines.

There are a lot of game engines out on the market today, both Open Source and Commercial. For a small list of engines, that I was investigating for various reasons, you can check out the Game Engine section in the links page.
For an excellent resource to find an engine that suits your needs you could go to http://www.devmaster.net/engines/ click on Advanced Search and select what you require.

Build your own using Middleware

Now you could write your own 3D engine, physics engine, scripting engine… engine engine engine… In my opinion, don’t waste your time (unless you are extremely passionate about learning the finer details), but seriously, don’t do it!
OR you could use Middleware (free or commercial). This is the approach I am taking.

The Game Engine

For my game engine I have decided to not use any of the available game engines on the market. Instead I am going to select individual middleware and put together my own game engine.

Why did I make this decision?

Cost

By basing the engine on free middleware it not only saves me money but will also save you, the reader, money as well.

Bloat

The current game engines can do alot! Sometimes they do so much that it can be overwhelming to use. By using smaller parts it will keep the overall design of the game and the game engine alot simpler.

Portability

By selecting code that is cross platform it allows us the opportunity to develop a fully cross platform game.

Swappability

Each part of the game engine can be swapped out for a different part if required. Different sound system? Different physics? Different Rendering engine? Scripting?

Source Code

You can upgrade any of the middleware components yourself if you need to since you will have the source code.

Learning

Since this article is about how to develop a game engine for an online game I am going to show you what middleware you could choose and how to integrate it into your own game engine.

Artist and Designer Friendly

Finally I want the engine to be very easy to use by Artists and Designers. It should be very simple to create new graphics and get them into the game. Because of this I am going to write all the tools so that they work within Cinema4D. Artists can design levels, create creatures, do animations and visualize that in the game with the click of a button. This is the only part of the engine pipeline that actually costs money, but if you are going to make a game and need to create content then you should have at least one content creation application at your disposal. Most, if not all, of the game engines on the market today do not have good modeling and animation toolsets. And why should they? They don’t have to, they are a game engine not a modeling application.

How to make your own Game Engine using Middleware

This section will focus on what a game engine actually is and what parts are required to make a game engine. One thing to keep in mind when searching for all these engines is to think about cross-platform ability, if you wish your game to be cross platform that is. If your perfectly happy to just release for windows machines then who I am to judge.

What is required in a game engine?

3D graphics engine

The 3D graphics engine would be built on something such as OpenGL, DirectX or some custom software renderer. It would be able to load in a scene file with geometry, textures and animations and display these to the screen. It will also have good scene management, animation abilities, cameras and lights. If you want to make your graphics look as shiny and realistic as possible then you may also decide to use hardware shaders in your engine. You might even look into using normal maps and displacement maps to give a higher quality look to some low poly models. I am not going to go into alot of detail here, there is simply too many things that could be discussed regarding 3d graphics engines. However I will go over this in a lot more detail in later articles.
One thing to consider when choosing a graphics engine is your own level of programming ability. So far the most easiest graphics engine I have come across today is the Irrlicht graphics engine. Other engines may have a lot more features, but the added complexity of the code makes them hard to learn for beginners. A few other engines which are free to use for commercial products are Ogre3D, Crystal Space and Wild Magic. Wild Magic is interesting because you can buy the book, by David H. Eberly, that explains the entire engine and its design.
Another thing to consider when making your MMO is how much effort do you want to put into the graphics. If you are thinking of something along the lines of World of Warcraft then Irrlicht may be fine for your needs. But if you want to make something that has the graphics like that of the new MMO Aion, made using CryEngine that was used on Crysis, then you might have to look for a more commercial graphics engine. But what I will say is that if you are serious about making an MMO, and you are a team of one to none, then just choose a simple graphics engine and concentrate on everything else, trust me you are going to have more than enough work to do without having to worry about getting some lens flare looking pretty when viewed near a normal mapped, high poly creature doing back flips with flaming god like particles flying out of its ear holes. Sticking to a simple graphics engine that is understandable and easy to work with is a good place to start. And I should point out that Irrlicht is more than adequate for your gaming needs.

Animation System

Now even though I mentioned above that the graphics engine can load in and display your animations you may still want to have a dedicated animation system. This system would blend between different animations and maybe also blend keyframe animations in with physics driven simulations. One common example is the mixing of Physics ragdoll simulations with imported motion capture data. You might have a few actions that were captured in a motion capture suit, then sampled down to a reasonable amount of keyframes. These are then imported into your animation system. There could be a run cycle, walk cycle, firing a gun, getting shot etc… Perhaps when your character gets shot the physics system initially takes over the animation of the character flying away then as he is nearing the ground an animation that moves the character into a landing position is slowly blended in with the physics simulation to make the character appear like he has landed on his hands and knees, followed by a standing animation and then back to running at the opponent to kick there ass.
This kind of system could be done with some kind of state machine to move between the animations and also simple linear interpolation between what the physics engine is giving you and what your keyframes should be at a given time. But in saying all this you don’t need to have such a system if you don’t want to. You could just as easily getting away with straight switching of animations, which is what it looks like in a lot of games anyway, the player runs, then stops straight away, no slowing down from a run to walk to stopping. But does it really matter? No not really. The game play is still good, it just looks a little strange. Not as strange however as players popping into different locations when walking around due to some server communication delay, but still a little strange.
Some systems that you could use to get this kind of system running are Havok Animation and software by Natural Motion

Physics Engine

And so now we move onto the Physics Engine. Again, same decision as the animation system, is it really required. And my answer to that is…. maybe. Its up to you. For an MMO I would not expect to create the same physical simulations on all players machines. Instead I would let each animation system do its own thing on each client and the things that are controlled by physics would just be for effects purposes, ie exploding things with debris flying around or a large stack of boxes falling to the ground only to disappear from the scene 10 seconds later. The reason for not keeping all clients in sync with all the physics objects is because it is too much networking overhead. Sending the locations of every physics object in the world to every client just seems like a lot of work that’s not really required. One other possibility is to have a dedicated server doing all the physics calculations and sending the data to every client, but again its a lot of work for not much gain.
Instead what I would do is have every client use their CPU and GPU to full potential and let things explode, fly around and bounce and blob all over the screen for pure graphics thrills. I would not send any data about any of these physics objects the server, or any client (if your doing a P2P game). This gives you more freedom to make fancy stuff and not care about game objects in the scene.
As far as getting a physics engine goes you could write your own. There are a lot of resources on the internet and writing one can be a lot of fun. But since you want to make a game and not play around with some pretty hard maths and optimizations its best to use one of the excellent libraries that are already out there. The most notable are Havok, PhysX, Bullet, ODE and Newton Dynamics. I will add more to the links pages as I find them (and think they are any good).
PhysX was initially a company making physics cards what would sit in your computer along with your graphics card and provide a hardware boost to your physics. This didn’t take off and eventually NVidia bought them out, not for the hardware, but for the SDK they had. It has since been optimized for their GPU’s and now if you use the PhysX engine in your games and happen to be running a game on a computer using something higher than an NVidia 8800 GT then you should see some speed improvements. It is free to use.
Havok is now owned by Intel and they have made it free to use for windows based game developers. You can download a binary only distribution and get started straight away. It comes with excellent integration with 3D applications such as Maya, Max and XSI. If you decide to sell your game for more that $10 you will have to get a distribution license from Havok, but I think this may be free as well. If you want to use it for any other purpose, such as PS3 or Xbox360, then you will have to purchase the full license.
Bullet is an open source, cross platform, physics engine that is free to use. You can also get all the code and add to it if you wish. It has a large community following and is gaining increased momentum as the physics engine of choice for a lot of projects. It is third in popularity according to a recent survey by the game developer magazine.
ODE and Newton Dynamics are two other open source physics engines. It has been reported that ODE has some problems with simulation stability in some cases. The Newton Dynamics engine is also a very popular physics engine and has been used in some simulation based projects as well as products like Esperient Creator.

Scripting engine

A scripting engine is good for writing your game logic in. The reason for its goodness is because you do not have to recompile one line of code, you could even set up your engine so that you could edit the scripts in game while its running. This frees the game designer from having to worry about visual C++, compiling code and other nasty stuff when all they really want to do is get into the game and tweak some of game logic for getting that last level playing super sweet. There are a number of languages you could choose as your scripting language, each has their own pros and cons which I will not go into here. Here is a list of some commonly used scripting languages in games….
Python
Ruby
Java
Javascript
Angelscript
C# or Boo via Mono
Lua
And once you have chosen your scripting language you will need an editor. You could just use notepad of course (or EditPlus or your favourite text editor of choice). Or you could add an IDE to your engine by using something like Scintilla, which has nice syntax highlighting features.
One thing to mention here is that most of the game logic scripts for an MMO are actually run server side and not client side. But even so there is alot of things that can be achieved by using a scripting engine on the client side as well. And either way you will need to make a decision on the scripting language and how and where you are going to edit them anyway.

Networking engine

You need to be able to talk to your game server and send things like player locations, health information and 30 million creatures just spawned and attacked you at the same time information. To do this you will need some sort of way of sending and receiving this information from the server. You could go the hard (or not so hard for some people) way and write your own using sockets and TCP/IP. Or you could decide to use one of the free or commercial packages out there to do all the underlying hard stuff for you. I personally would rather spend as much time making the game than investigating some packets to see why the right information is not in them.
One good, and free until you earn a lot of money, engine is RakNet. This is a cross platform C++ game networking engine. But in addition to its ability to handle all game object replication and easy serialization and construction of game objects it also has a lot of other neat things that are ideal for making online games. These being a Lobby system, voice communication, RPC (Remote procedure calls) and a patching system to download the latest data to the client machines to keep them in sync, very nice. So you could in effect use this one system to handle all the communication to the server for your games and allow voice communication via P2P for players to communicate to each other.
If you were a flash developer you could use something like SmartFoxServer. Its not just for flash however, you can also use it with Java, .Net and iPhones etc…
Perhaps you would like to code your game in Java. If this is the case then take a look at Project Darkstar. This is a fully capable back end server with client code for both Java and C++. You will still need to serialize the data yourself but the main communication mechanism is all in place and the server has full persistence of all game object data. This means that it handles all the saving and loading of data for your world to the database for you so you don’t have to worry about it. You can just write your server side game logic and get your game going.
There is also a commercial Networking engine called NetDog. I do not know what projects it has been used on or its reputation, but it may be worth a look. It is a commercial engine however and they do not disclose how much it costs.
And finally one other system is Photon. Photon is interesting since it can be used in conjunction with Neutron to give a full MMO networking solution including the backend server and game logic scripting using C# and has SDKs for iPhone, Android, Unity, Win32, .NET, J2ME, BREW and Flash.

Sound engine

Your game will want to have sound. Its a must really. Without sound its really only half a game. You will need to be able to load in sounds, change their basic characteristics, such as pitch, speed and tone. Maybe add some effects over the top to distort or add reverb to the sounds ( or you could just have loads of preprocessed sound files of each type you require). You will also need to be able to change the volume of the sounds and mix them together, perhaps pan around for stereo sound, maybe even go for 5.1 surround sound. If you are going to use the Irrlicht graphics engine then you might consider using the Irrklang audio library since it works in with Irrlicht quite well.
Another well known sound library is OpenAL. Where as Irrklang can cost anywhere from 0 Euro for non-commercial to 1950 Euro for a full commercial product, OpenAL is free.

AI Engine

If your game has non player characters (npc’s) running around then you might also want to have an AI Engine to control them. Bearing in mind that the main control for AI characters will most likely be done server side you might still want to have some purely client based characters running around in the game. One such system for path finding is Havok AI. Another, which is still being worked on, is IrrAI.

Resource Management, Memory management, file loading and streaming

This is not really an engine but just something that you will need if you make a large scale game, such as an MMO. You will need to be able to stream in data from the users hardrive while the game is playing and update your graphics, sound, animations etc… on the fly. You will also have to dump unused memory as you go along as well. This is particularly required for textures and geometry as you move through a large open world. You will need to pre load in various parts of the world in case the user is going to go that direction in the game. If you keep your graphics and textures simple then you will have to do this less often, something to keep in mind.

Editors

To make your game you will need to be able to layout the levels, add sounds, setup characters with the right textures, animations, statistics etc… for all this you will need one or more editors. This section will go over some of the editors that you will require to make your games.

What kind of editors will you need

Level Editor

The level editor is how you create your game worlds. It lets you load in geometry and place it in the world. This is where you lay down the terrain, add trees, rocks and buildings. Tell a building it has a certain amount of hit points before it will fall apart after been hit by multiple rockets. You setup sound in your scene, where the sound direction is coming from and how large of an area the sound can be heard in. You need to be able to add Trigger locations, usually in the form of a large wireframe sphere or cube, to the scene to trigger scripts, effects or sounds when a user comes in contact with the trigger. Here you will setup the lighting of the game, perhaps some simple geometry for navmeshes (Navigation geometry used by the AI system to determine where AI characters can walk) as well as physics settings to mark things as solid or movable. You will also place monsters, traps, treasures in your levels and many other things. The list can go on and on.
Your levels might be broken up into zones in your world. Each zone will be streamed in and out of the game engine as the players walk around the world. The Level editor will need to be able to save all its data and export out all the settings in a file format that the game engine can use, most likely in the form of an XML file.
Unless you get a system that contains a level editor already in it, such as the CryEngine3 Sandbox Editor, you are going to have to build one yourself. However another option is to incorporate all your tools into an existing system such as Maya, Max or Cinema4D, more on this later.

Sound Editor

The sound editor needs to be to load in sounds in various file formats, adjust and tweak the sound, make sounds loopable, remove static noise etc… basically any sound editor out there on the market today will do the job. You do not need to write your own sound editor. If the sound editor you have chosen doesn’t output to the required file type for your sound engine in your game engine then you may have to use a separate converter to do the job. This is usually not a problem since it will be handled by your asset pipeline, ie you check in the WAV file to the Digital Asset Management system and the asset pipeline system automatically converts it to an OGG file and saves that back into the Digital Asset Management system to be used by the game when it runs.

Animation Editor

Unless you have a game engine that already has such a system you are going to have to write one of these yourself. You can write it ontop of your graphics engine, but remember to keep it as separate as possible. This system is mainly going to be used to setup characters with animations and setup the finite state machine to tell it how to blend between the different animations for that character. As an example you might load in a mesh for a character. Then select a few different run and walk cycles for the character and tell the game how it would blend between them using some non linear animation system. You can then preview how the animation looks on the character.
You might even give this system the ability to tweak the animations a bit by modifying the keyframes. But this is not really recommended since you will then have to remember this modifications and try to apply them each time you modify the original animation data in Maya, Max or Cinema4D.

Character Editor

This could very well be integrated with the Animation Editor itself. The Character Editor would be where you setup a full character. You select the mesh, maybe change some textures, tell it what abilities it has, what weapons it can carry, what its initial health points are, allowed spells etc… anything you can think of that might be useful for setting up on this character should be able to be done in the Character Editor. This could also include the animations, as described previously. Now this is not to say that this data is the final definite data for the game. The scripting system when the game is running could of course alter any of these values.

Particle Effects Editor

An editor to create particle effects like fire, smoke, explosions, waterfalls, rain and lots lots more. This again is something that you will have to write yourself and is very specific to the graphics engine that you have chosen to use.

Facial Animation Editor

A lot of games these days have in-depth stories and complex character dialog. If you want to have any of this stuff then you could look into creating a facial animation editor. I will not go into this in too much detail other than to point out a few sites like lifemode, Image-Metrics and Quidam.

Content Creation

Needless to say really but you will need a content creation package such as Maya, Max, XSI, Cinema4D, Modo, Blender or one of the many other products. Each are good.

AI Editor

Perhaps your game will have crowd simulations, a vehicle traffic system, bird flying around in some flocking behaviour or maybe each of your characters has their own brain and you need an editor to setup how everything works… if this is the case then you may need some kind of AI Editor.

Other Editors

There are many other types of editors depending on the type of game you are making. For example vehicle editors and tree editors to name just two. Take a look at an engine like CryEngine3 or UnrealTechnology. The number of things you can do with their technology is astounding.

All in one editor

Perhaps you might want to create an all in one editor, something like Crytek’s sandbox editor or the Unreal Engine. This could even be based on your actual game engine and provide a play mechanism within the editor, ie WYSIWYP (What you see is what you play) functionality. This is a nice idea and can easily be designed for. But one thing to avoid is just jumping straight into the game engine and adding all this functionality. It is still best to think about all these as just components that can be used independently. By doing so you get a clear separation between the different parts of your system. This is useful to stop any kind of spaghetti hacking code from making your system unusable.
You could create an editor so that is has each of these “sub-editors” as plugins. Each of the sub-editors would work with their own data and file formats, but could be accessed via a unified UI. These editors could use the game engine to preview the changes, but make sure not to make the game engine rely on any code from the editors, with the exception of perhaps the loading code. An editor does not have to actually do anything other than allow the user to “edit” the data. An editor could be something as simple as Notepad describing an XML file.

Building your own editors

Eventually you will have to build an editor. As I said previously an editor could be something as simple as a text editor for editing an XML document, or it could be a fully integrated 3D level editing and all in one monster super cool application. Whatever the case may be you will need to learn a few things before you can create your level editor. Firstly lets think about what platform your editor is going to be on?

Editor Platform

For a moment here I am going to expand upon a few things a bit further than 3D games. Lets think about ways we could write editors for our games and where they might be used.

Editor in a web browser

If your game is going to be silverlight of flash based then perhaps you might want to open up the opportunity for the community to create content for your games (make levels etc… ) by using some online editor. It certainly is possible to create a level editor in Flash/Silverlight, have that publish content to a back end database and get real time updates in a game running on the web. Just something to think about.

Editor within the game (Little Big Planet)

Perhaps you have heard of, or played, Little Big Planet for PS3. All the levels were created on the PS3 itself, and they have also allowed the community to create their own levels and share them with the world. This is a very interesting concept and something that perhaps you may want to allow for yourself in your game engine. This doesn’t have to be restricted to PS3 however, you could do this on any platform if you wish. Going back to the idea of a flash/silverlight based game, the game itself could very easily be made into a level editor if that’s the kind of interaction that you wanted to provide your users. The thing to note about this concept however is that if you are to do this, then ONLY do this. By forcing yourself to only ever create levels within the game it will make your tools better. If you cheat and say… “hmmm I really just want to get this done, I will write an XML file and import that to create my level just this once” then you will get lazy and continue to do this over and over and your tools will never end up being any good for the end user to use.

Windows / OSX / Linux based

Do you want your level editor to be used by Windows users as well as OSX and Linux users? Perhaps you want Solaris users to be able to edit levels. Keep this in mind in the very beginning. If you want your editors (and in fact your entire game engine) to be cross platform then you must care for this at the beginning of the project. You can very easily get lost in a pile of really nice libraries and code that will only ever be usable on a single operating system. One point to note here is the language that you choose. If you are going to develop your editors (or game engine) in .Net then you should consider looking at the Mono-Project. This will let you run your .Net applications on other operating systems, but you need to check what is supported in Mono before you start adding functionality to your application otherwise it might not run.

Who is your target audience?

When building your editors you should keep in mind who is going to be using them. What level of expertise do you think they will have, are they kids aged 5 years old making a simple play world for them and their friends or are they professional 3D graphics experts who already know how to make games, or anything in between.
When designing your editors you should try to keep things simple, try not to overwhelm the user with a million options on one screen. And perhaps, maybe one day in the not so distant future, you may want to sell your system to a potential game developer. If this is the case then you will have to provide lots of documentation and tutorials on how to use the system, so the easier it is to use the less work you have to do in documenting it all up.

GUI Systems

To make your editor its best if you use an existing framework for doing GUI’s. They can be a real pain. I have used many of them over the years and I haven’t actually found any of them to be perfect for my needs. I am not going to go into much detail about the GUI systems other than to point you at a few and let you experiment for yourself. I will say however that it is, in my opinion, very easy to develop 2D based level editors in .Net using C#. But QT, or wxWidgets experts may say the same thing about their favorite system.
MFC pretty old microsoft technology (windows only)
wxWidgets(cross platform, open source)
QT (cross platform, LGPL licence available)
Winforms or WPF (Windows, or via Mono on linux)

Build as much as you can into Maya, Max, Cinema4D

One practice that is used within the games industry is to build the tools directly within your preferred 3D application. This could be Maya, Max, Cinema4D, Blender etc…
The benefit of doing this is that the artist do not have to be retrained in a new system. They can stay within Cinema4D and create the characters, rig them up, add animation, textures, triggers, hit locations, create full levels with spawn points and navmeshes etc… These are then exported to an intermediate file format (more on this in a later article on Digital Asset Management) and then processed to get the assets into the right game format and away you go. Most applications these days have features for adding extensive GUI’s, ways of marking up objects in the scenes (IE physics, trigger points etc…) and also provide good API’s and scripting systems for exporting that data out again.
The main downside to this approach is that you will have to have a copy of the 3D application to edit your levels properly. But if you are a small dev team then you will already have a 3D app of choice already. And if your a large team then you can probably afford a couple more copies for a dedicated level editor, and possibly have some other non-graphics intensive editors built as simple .NET applications. IE for setting up game object properties like hitpoints, weapons statistics etc…
For my game engine most of the tools will be developed within Cinema4D. It has a very extensive GUI system and allows you to create any kind of GUI functionality that you may need.

The Game Asset Pipeline

One final thing to mention about the level editors is that you should keep in mind the Game Asset Pipeline. This is the flow of the content from the editors to the final game. As an example a typical workflow for a 3D model would be to edit it in your 3D application, export the file to an intermediate file format, save the intermediate file format and its original 3d file in a digital asset management system, have some system that checks out the intermediate file and processes it extracting out different files required by the game engine in optimized file formats for that engine (you might have a separate mesh file format, skeleton file, multiple animation files, textures, lights, camera files etc…). These game asset are then published to the game and viewed.
For my project Cinema4D will also contain functionality to hook up to a Digital Asset Management (DAM) system to store all versions of the levels and any intermediate files created from them. There will be tools built directly within Cinema4D to load in assets, change them and check them back into the DAM. You can also trigger a game preview which will process all the files and load up the game with the latest content.
If you are interested in getting started with Cinema4D plugin development then you can take a look at another article I wrote a month or so ago.

Deploying game content

Some all in one systems have a method of deploying their games, usually a “one click deployment” method. These work quite well for all in one solutions such as Esperient Creator or Unity3D. What they generally do is convert all the assets to the required game format required by the game. These files are then packaged up (maybe like a Zip file as in the Quake pk3 files). The file is then copied over to a location with all the executable files for the game and voila! You have a game ready to ship.

Incremental updating

Now what about doing updates to a game sometime after release? You could ask the client to download patches to the game. Or you could let the game engine itself look for updates. This could happen while the game is running and do some sort of hot loading, or you could make it happen when the game starts up. When it starts it downloads the latest files from the server and then runs. You can in fact do this very easily using RakNet.

P2P delivery

Another interesting way of delivering content could be for clients to tell a game server where they are located and then each player could download the files from the nearest player via a P2P mechanism. This would reduce the load on the asset server by not having to upload potential gigabytes of data to every user when an update is required.

MMO Development Suites

As I fnish up this article I would like to list some other technology that allows you to get up and running with your own MMO without too much effort. The following is by no means an extensive list of current engines that are out there on the web today. This is just a small selection of what I have found to be the most talked about engines out there that let you jump straight into creating your own MMO games with out too much effort.
www.bigworldtech.com
www.heroengine.net
www.kaneva.com
www.multiverse.net
For a good survey of other engines you can checkout this link at devmaster
http://www.devmaster.net/wiki/MMORPG_Creation_Packages

Conclusion

After reading this article you should have a better understanding of what might be required in a Game Engine. You should make sure to explore many of the game engine components out on the internet today and become familiar with at least a few of them. Unity3D has just released a version that is free for Indie developers and Epic Games has also released a free version of the Unreal Engine as well. Other engines such as Crytek’s CryEngine, that ships with the game Crysis, is something you should also consider having a look at.
But if your not into 3D then checkout the many flash based MMO development kits and see what suits your needs there as well. Whatever you do make sure you keep having fun, since as soon as it starts feeling like hardwork you will loose momentum and your game may never end up being finished.