Data Oriented Architecture

In computing, data-oriented design is a program optimization approach motivated by efficient usage of the CPU cache, used in video game development.

The approach is to focus on the data layout, separating and sorting fields according to when they are needed, and to think about transformations of data.

Wikipedia

To be honest, when I was coding the architecture of Mus 3D, designing the seventh iteration of the code, I had so many problems that decided to throw away a good part of the code and to start with a new approach. I created a new branch in git and started to experiment.

Later, before going to sleep, I talked with myself:

- "Jorge, essentially, what your are implementing is a post office: the incoming network datagrams from the players are like letters. Let me explain the steps, Jorge:"

- "Ok Jorge, I am all ears", I answered

• You receive all of these letters, as Mus 3D is a game of cards, each game of 4 players is managed by a "croupier".


• You have to give each letter to its correspondent croupier, the croupier reads it and modifies the elements of the game (and possibly the letter), and...


• Maybe this card has to say something important to the other players, so the croupier makes 3 copies more of this letter and sends each copy to each player (reusing the original letter).


• * Each player might be a human player or an Artificial Intelligence.


• If he/she is a human, the letter is transformed to a TCP network datagram and sent to the Internet, towards the device where that human is playing.


• On the other hand, if it is an IA, the croupier locates her and gives her that letter (In Mus 3D, the AIs are females :-) ). She reads the letter and takes a decision, this means that these IA write a letter with the decision taken and gives it to the croupier, and the cycle continues...

This is a very simplified recipe of how Mus 3D, -and other online games systems-, work.

Looking through the OOP prism, each letter had been a not-always-easy marriage between data and logic. Was difficult to manage, adding new methods without changing other places in the code was close to impossible. And, the worst of all, there was an incipient lack of flexibility. Each bit of the code I changed seemed to affect several other places in the basecode.

So, I started to wonder "How would the ancients solve this problem?".

I remembered Ford and his assembly line, and "Eureka! That could be the solution!"

In an assembly line, each car (or motobike, if you prefer motorbikes), is travelling through a line, the cars are quiet, in silence, just waiting to be transformed into a finished product.

They do nothing, they just wait, are transformed and moved. The employees (or robots) are the ones that do all the work, the ones that add new elements, or transform the existing ones. Some of them make verifications, write down words and figures in paper, of perform other actions. Some of them do nothing, just observe.

Each employee (or robot) makes a little transformation, adds a step, and the trip continues to the next employee, until the car is finished.

Following this metaphor, let me explain how the Data-Oriented-Architecture of Mus 3D works. I will give a more complete explanation about this exciting approach in my next posts, I promise (they are still being cooked).

Meanwhile, let me give some hints in advance:

• Each "letter" is just a flexible data structure, like a JSON object. Maybe it can be just a collection of key-value pairs, or an array of numbers (it depends on the problem to be solved).


• Each "employee" is a method that takes as input a pointer to a letter, and returns a pointer to a letter (most of the times, it returns the same pointer of the same letter, other times it returns a NULL pointer).


• Some employees just do nothing: receive a pointer, read some bits, and if is not a letter for them, they just return the same pointer without making any modification. The trip of the letter continues to the next employee.


• Other employees read some information of the letter an do specific things (for example updating other data structures, or making changes in the letter, maybe they create new letters and put them in other lines, who knows). Finally they return a pointer to that letter.


• Some employees read especific information of the letter, perform some determinant actions and delete the letter (or send it to a recycling pool of letters) and return NULL.


• If the returned pointer is NULL, the trip has finished for that letter: It's no letter anymore to give to the next guy. Let's go to the "incoming letters" pool to get a new one.


• In other cases, the letter arrives to the last employee ("The letters recycler"), and, this employee deletes that letter (or throws it to the recycling pool of letters), the trip has finished.

This could be a brief description of the process happening inside a data assembly line.

At first sight, this architecture seems to open a bunch of new exciting posibilities, doesn't it? It opens new points of view, new approaches to think, ...

So, more specific details will be given in my next posts (with pictures!), meanwhile, here I put some videos that I have found about the topic.

See you in the next post!

Jorge Gascon Perez

Back