Designing netKar PRO 2.0 - Part 1

I have started working on the new incarnation of my main project. In the original plans nkp2.0 was intended to be an evolution of the current netKar PRO code. The main improvements planned were:

- New graphics engine (support for HDR and other advanced features "displacement mapping?")
- Updated physics engine to support multicore architecture and/or 3rd parties physic engines (ODE, PhysX, Havok).

With time more features made into the proposed 2.0 spec. The most important being the movement of all the "user interface" elements to a web based app.

I started looking into the implementation details and I realized a couple of facts, and the most important was (we'll go through some history here):

netKar PRO core is now almost 7 years old. It started as an "experiment" in Newbury at the end of 2001. The project went through 3 main phases:

netKar (free) 0.9.0 to 0.9.3
This was the original design. 3D was implemented using OpenGL, user interface with C++ Builder 6.0 and the sim exe using Visual C++. The entire 0.9.x branch used a "code driven" approach to modeling.
In this approach different cars were implemented through a DLL containing the actual data and custom code for a particular car.
All the DLL linked to a static lib containing a "base car" class. The cars were derived from this base core.

netKar (free) from 0.9.4 to 0.9.9 (aka Namie)
The big change here was the move to DX9 (fixed function pipeline) for the graphics engine and, of course, the first multiplayer core in 0.9.9.

netKar PRO v1.0
netKar PRO introduced a number of heavy changes to the Namie core:

- 3D shaders support
The 3D engine had to undergo heavy modifications to support this feature and to be able to keep the support for the legacy FFP functionality.

- "Data driven" modeling
In netKar PRO v1.0 I have abandoned the idea of having cars implemeneted as DLL and I designed a "general" car object that was driven by a database file containing the car description.

- "Tool based pipeline"
Development up to Namie was based on a simple approach that I would call "straight to sim". The idea was to have a direct way into the simulator for external data such as 3D models and physical quantities.
And example of this approach was the 3D importing strategy for Namie. The system could import 2 file formats: ASE and netKar native binary format "NKO". Every time a 3D asset was requested, the system checked for the file "assetname.nko" and procedeed to load it if present. If the nko wasn't available, the system will look for "assetname.ase" and load it. After the ASE loading was completed the system would save a "NKO" version of the same asset.
So, the workflow to import say a new track 3D model was:

. delete track.nko file
. copy the new track.ase file in the folder
. launch netKar Namie (and this will create an updated track.nko)

The design for Namie tried to support simplicity and "no pre processing" workflow.

The shader support for netKar PRO made this approach not possible anymore. There was suddently a need for something to sit in between the source data and the simulation software. For 3D this "something" was the "infamous" (among netKar PRO graphics artists :P) "KOF Edit".
This editor was originally designed to allow to import ASE files, make changes to the materials (applying shaders, settings shaders paramters, changing textures etc.) and save NKO files to be used in the sim, and all that by using the same 3D engine used in the sim. So artist were given a possibility to tweak things and see the effect of their changes in real time and expect a 100% fidelity once the file was loaded into the sim.
At the end, the KOF Edit ended up doing much more than that. After acquiring some artist using XSI for modeling I had to develop an .XSI importer for the KOF. I started to find the KOF editor the ideal place to edit car physics stuff like collision boxes, suspension graphics points and so on. It also supported hieararchy changes, renaming functions, skybox IO...

Right now we're using a new version of the editor called KOF Edit 2. The new version is a .NET managed c++ application that links a static library with the graphic engine.
The old KOF Edit design was a nightmare. It was developed as a C++ Builder application for the GUI and a Visual C++ DLL for the 3D window. The 2 were talking using a single function (originally they were using UDP packets :-S). Every new feature required lots of implementation to create the bridge between the 2 worlds.
The new version is somehow better because, at least, it lives in 1 application, although the Managed and Unmanaged worlds are somehow separated. Plus, working with .net c++ WinForms application is an exercise in patience as it takes ages to refresh the window designer (and sometimes strange errors appear).

Another essential part of the "netKar PRO v1.0 system" was the "nKdK Dev Studio". nKdK stands for "netKar development kit". Original plans for netKar PRO were to offer a system to allow the community to create cars and tracks. The Dev Studio was part of this plan.
It was designed to edit the physical data of the car, engine, suspension, aero, graphics connections, drivetrain and so on. But it was not simply a DB editor, it was a set of tools to support physical data editing.

To close this journey into the nkp system I have to talk about the Tyre Editor application, one of the most important part of the entire system. Allowing to edit/check/design tyre curves using different tyre models.

Although netKar PRO v1.0 implemented a long list of new features and "behind the scenes" changes, it was still somehow "hacked" into the existing Namie codebase. This made the solutions sometimes overcomplicated.

With 158.000 lines of C++ for the simulation, 50.000 lines of C++ lines for the netKar PRO.exe user interface, 41.000 lines of C++ for the KOF Edit.. plus several thousands of lines for Tyre Editor and Dev Studio the task of "hacking" more features into such a vast core for a single developer started to look frightening at least.

I started to look around for some alternatives. And 1 of them looked immediatly interesting: .NET + C#.

I had started to play with C# more and more finding it quite pleasant to use. I used it for the first time in a "real" project some months ago when I had to design an editor to visualize 3D scan data for track making.
Having worked with .NET C++ for the KOFEdit2 I knew I didn't want to get into GUI building with C++ again. So I tried a 2 module approach with C# GUI talking to C++ Managed app linked to a static lib for the 3D engine.
It worked great.. it was easy to develop, super fast, a very succesfull app.

Then I had another experience. I had started redesigning a physic engine abstraction layer (codename "Kimi") to be used for netKar PRO 2.0. It was all done as C++ DLL with pure abstract classes. The interface was cool, and I wanted to try it with some graphics into an editor. So I thought.. let's do it again, I will do the GUI in C#, everything else in C++. I went one step further and implemented a .NET wrapper over the pure C++ physic DLL, called KimiX.
Now I could drive all my physic objects straight from C# and I noticed how easy and fast everything was.

So I started to have this voice in my head repeating "how nice would it be if I could have all my code in C#?".

...