ECS

Dynec uses concepts from the ECS (Entity-Component-System) paradigm. It is a data-oriented programming approach that consists of three core concepts:

An entity represent different objects.
Different components store data for an entity.
Systems process the components to execute game logic.

Data

An intuitive way to visualize entities and components would be a table, where each row is an entity and each cell is a component of that entity:

Entity #	Location	Hitpoint	Experience
0	(1, 2, 3)	100	5
1	(1, 3, 4)	80	4
⋮	⋮	⋮	⋮

Everything can be an entity! For example, in a shooters game, each player is an entity, each bullet is an entity, and even each inventory slot of the player may be an entity as well.

The components for a bullet are different from those for a player:

Entity #	Location	Velocity	Damage
0	(1, 2.5, 3.5)	(0, 0.5, 0.5)	20
⋮	⋮	⋮	⋮

A system is a function that processes the data. In a typical simulation program, each system is executed once per "cycle" (a.k.a. "ticks") in a main loop. Usually, systems are implemented as loops that execute over all entities of a type:

for each bullet entity {
    location[bullet] += speed[bullet]
}

An ECS framework schedules systems to run on different threads. Therefore, programs written with ECS are almost lock-free.

Archetypes

In traditional ECS, the type of an entity is identified by the components it has. For example, an entity is considered to be "moveable" if it has the "location" and "speed" components. Systems iterate over the "moveable" entities by performing a "join query" that intersects the entities with a "location" and the entities with a "speed". Thus, an entity type is effectively a subtype of any combination of its components, e.g. both "player" and "bullet" are subtypes of "moveable".

Dynec takes a different approach on entity typing. Dynec requires the type of an entity (its "archeytpe") to be known during creation and immutable after creation ("statically archetyped"). A reference to an entity always contains the archetype.

Dynec still supports adding/removing components for an entity, but this is implemented by making the component optional (effectively Option<Comp>) instead of changing its archetype. Adding/removing a component would not affect systems iterating over all entities of its archetype.

To iterate over entities with only a specific component, the suggested approach is to split the components to a separate entity with a new archetype and iterate over entities with that archetype instead. (It is also possible to iterate over entities with a specific component, but it is less efficient than iterate over all entities of the same component, and joining multiple components is not supported)

Archetypes are typically represented as an unconstructable type (an empty enum) referenced as a type parameter in system declarations. Therefore, multiple systems can reuse the same generic function where the archetype is a type parameter, achieving something similar to the "subtyping" approach. Nevertheless, Dynec discourages treating archetypes as subtypes and encourages splitting shared components to an entity. Therefore, it is possible to reuse the same function for multiple systems by leaving the archetype as a type parameter.

An archetype can be declared through the dynec::archetype macro:

#![allow(unused)]
fn main() {
dynec::archetype! {
    /// A building entity can store goods inside.
    pub Building;

    /// Each road entity represents a connection between two buildings.
    pub Road;
}
}

There is nothing magical here; each line just declares an empty enum and implements Archetype for it.

Components

Components store the actual data for an entity. In Dynec, since entities are statically archetyped, a component is only meaningful when specified togethre with an archetype.

There are two kinds of components, namely "simple components" and "isotope components". For simple components, each entity cannot have more than one instance for each component type. Meanwhile, isotope components allow storing multiple instances of the same component type for the same entity.

Simple components

Simple components are components where each entity can have at most one instance of the component. A type can be used as a simple component for entities of archetype A if it implements comp::Simple<A>. Dynec provides a convenience macro to do this:

#![allow(unused)]
fn main() {
#[comp(of = Bullet)]
struct Location([f32; 3]);
}

This declares a simple component called Location that can be used on Bullet entities.

The same type can be reused as components for multiple archetypes. by applying the macro multiple times:

#![allow(unused)]
fn main() {
use dynec::comp;

#[comp(of = Player)]
#[comp(of = Bullet)]
struct Location([f32; 3]);
}

Initializer

Simple components can be equipped with an auto-initializer. If an entity is created without specifying this component, the auto-initializer is called to fill the component.

The auto-initializer can read values of other simple components, either specified by the entity creator or returned by another auto-initializer. Since Dynec does not persist a component unless it is requested by a system or explicitly registered, this means you can pass a temporary component during entity creation, use its value in other component auto-initializers, and this temporary component gets dropped after entity creation completes.

The auto-initializer can be specified in the macro either as a closure:

#![allow(unused)]
fn main() {
use dynec::comp;

#[comp(of = Bullet, init = |velocity: &Velocity| Damage(velocity.norm()))]
struct Damage(f32);
}

or as a function pointer with arity notation (i.e. write the number of parameters for the function after a /):

#![allow(unused)]
fn main() {
use dynec::comp;

fn default_damage(velocity: &Velocity) -> Damage {
    Damage(velocity.norm()) 
}

#[comp(of = Bullet, init = default_damage/1)]
struct Damage(f32);
}

Presence

A component is either Required or Optional.

Optional components may be missing on some entities. Accessing optional components returns Option<C> instead of C.

Required components must either have an auto-initializer or be passed during entity creation. This ensures that accessing the component always succeeds for an initialized entity; optimizations such as chunk iteration are only possible for Required components. Nevertheless, components are always missing for uninitialized entities created during the middle of a tick; more will be explained in later sections.

A Required component must both set PRESENCE = SimplePresence::Required and implement comp::Must<A>. This is automatically done by specifying required in the #[comp] macro:

#![allow(unused)]
fn main() {
use dynec::comp;

#[comp(of = Bullet, required)]
struct Damage(u32);
}

Finalizers

A finalizer component is a component that prevents an entity from getting deleted.

Yes, I know this may be confusing. Contrary to finalizers in Java/C#, a finalizer is a data component instead of a function. They are actually more similar to finalizers in Kubernetes.

When an entity is flagged for deletion, Dynec checks if all finalizer components for that entity have been removed. If there is at least one present finalizer component for the entity, the entity would instead be scheduled to asynchronously delete when all finalizer components have been unset.

This gives systems a chance to execute cleanup logic by reading the component data of the "terminating" entity. For example, a system that despawns deleted bullets from network players may get a chance to handle bullet deletion:

for each `Bullet` entity flagged for deletion:
    if `Despawn` componnent is set
        read component `NetworkId` for the entity
        broadcast despawn packet to all players
        unset the `Despawn` finalizer component

Without the finalizer component, the system would be unable to get the NetworkId for the despawned bullet since the component has been cleaned up.

Deletion-flagged entities are checked every tick. To avoid a growing backlog of entities to delete, finalizer components should be removed as soon as possible after deletion has been flagged.

Best practices

Small component structs

Dynec prevents systems that write to the same component type from executing concurrently to avoid data race. In reality, most systems only need to access a subset of fields, so avoid putting many unrelated fields in the same component type. Instead, prefer small, often single-field structs, unless the multiple fields are naturally related, e.g. positions/RGB values that are always accessed together.

Optional types

Avoid using Option in component types; instead, use optional components to represent unused fields. By default, Dynec uses a compact bit vector to track the existence of components, which only takes 1 bit per component. Meanwhile, Option<T> needs to preserve the alignment of T, so a type like Option<f64> is 128 bits large (1 bit for None, 63 bits for alignment padding, 64 bits for the actual data), which is very wasteful of memory.

Heap-allocated types

Minimize external (heap) memory referenced in entity components. Heap allocation/deallocation is costly, and the memory allocated is randomly located in the memory, which means the CPU needs to keep loading new memory pages into its memory cache layers and greatly worsens performance. Dynec stores component data in (almost) contiguous memory and prefers processing adjacent entities in the same CPU, so keeping all relevant data in the component structure is preferred.

While this is inevitable for some component types like strings, types like Vec can often be avoided:

If each entity has a similar structure of items (i.e. comp[0] for entity 1 has the same logic as comp[0] for entity 2), use isotope components instead.
If the items in the vector are unstructured (i.e. comp[0] for entity 1 has the same logic as comp[1] for entity 1), consider turning each item into an entity and process the entity instead.

Isotope Components

Sometimes we want to store multiple components of the same type on an entity. For example, we want to store the ingredients that make up a bullet. The straightforward approach is to use a Vec<(Element, Ingredient)>/HashMap<Element, Ingredient>, but this is very bad for performance and memory due to many heap allocations. This is where isotope components come handy.

An isotope component works like a component that stores a map of "discriminants" to component values. For example, in the example above, Element can be used as the discriminant that distinguishes between different "weight" components, and an entity has a separate Ingredient for each Element.

Like simple components, isotope components are also archetyped, but they implement comp::Isotope<A> instead, which can also be achieved through the #[comp] macro:

#![allow(unused)]
fn main() {
#[derive(Discrim)]
struct Element(u16);

#[comp(of = Bullet, isotope = Element)]
struct Ingredient(f64);
}

Unlike vector/map simple components, Dynec treats each discriminant as a different component such that it has its own storage and lock mechanism, so systems can execute in parallel to process different discriminants of the same component.

Choosing the discriminant type

Dynec creates a new component storage for every new isotope discriminant. If you use the storage::Vec (the default) storage, the space complexity is the product of the number of entities and the number of possible discriminants. Therefore, the number of possible discriminant values must be kept finite.

An example valid usage is to have each discriminant correspond to one item defined in the game config file, which is a realistically small number that does not grow with the game over time. Ideally, the possible values of discriminant are generated from a 0-based auto-increment, e.g. corresponding to the order of the item in the config file.

Initializer

Similar to simple components, isotope components can also have an auto-initializer. However, new discriminants may be introduced after entity creation, so isotopes cannot be exhaustively initialized during entity creation but initialized when new discriminants are added instead. Therefore, isotope auto-initializers cannot depend on any other values.

Presence

Isotope components can also have a Required presence like simple components. However, since discriminants are dynamically introduced, it is not possible to initialize an entity with all possible discriminants exhaustively. An isotope component can be Required as long as it has an auto-initializer.

Systems

Systems contain the actual code that process components. A system can be created using the #[system] macro:

#![allow(unused)]
fn main() {
use dynec::system;

#[system]
fn hello_world() {
    println!("Hello world!");
}
}

After the #[system] macro is applied, hello_world becomes a unit struct with the associated functions hello_world::call() and hello_world.build(). call calls the original function directly, while build() creates a system descriptor that can be passed to a world builder.

We can package this system into a "bundle":

#![allow(unused)]
fn main() {
use dynec::world;

pub struct MyBundle;

impl world::Bundle for Bundle {
    fn register(&mut self, builder: &mut world::Builder) {
        builder.schedule(hello_world.build());
        // schedule more systems here
    }
}
}

Then users can add the bundle into their world:

#![allow(unused)]
fn main() {
let mut world = dynec::new([
    Box::new(MyBundle),
    // add more bundles here
]);
}

Alternatively, in unit tests, the system_test! macro can be used:

#![allow(unused)]
fn main() {
let mut world = dynec::system_test!(
    hello_world.build();
);
}

Calling world.execute() would execute the world once. Run this in your program main loop:

#![allow(unused)]
fn main() {
event_loop.run(|| {
    world.execute(&dynec::tracer::Noop);
})
}

Ticking

Since dynec is just a platform-agnostic ECS framework, it does not integrate with any GUI or scheduler frameworks to execute the main loop. Usually it is executed at the same rate as the world simulation, screen rendering or turns (for turn-based games), depending on your requirements.

It is advisable to keep latency-sensitive operations out of the main loop, i.e. do not process them directly with the Dynec scheduler so that the world tick rate does not become a necessary latency bottleneck. Dynec systems are designed for ticked simulation, not event handling; event handlers may interact with the ticked world through non-blocking channels.

Parameter, local and global states

Parameter states

A system may request parameters when building:

#![allow(unused)]
fn main() {
#[system]
fn hello_world(
    #[dynec(param)] counter: &mut i32,
) {
    *counter += 1;
    println!("{counter}");
}

builder.schedule(hello_world.build(123));
builder.schedule(hello_world.build(456));

// ...
world.execute(dynec::tracer::Noop); // prints 124 and 457 in unspecified order
world.execute(dynec::tracer::Noop); // prints 125 and 458 in unspecified order
}

The parameter type must be a reference (&T or &mut T) to the actual stored type.

Each #[dynec(param)] parameter in hello_world must be a reference (&T or &mut T), adds a new parameter of type T to the generated build() method in the order they are specified, with the reference part stripped.

Parameter states, along with other states, may be mutated when the system is run. Each system (each instance returned by build()) maintains its own states.

Local states

Unlike parameter states, local states are defined by the system itself and is not specified through the build() function.

#![allow(unused)]
fn main() {
#[system]
fn hello_world(
    #[dynec(local(initial = 0))] counter: &mut i32,
) {
    *counter += 1;
    println!("{counter}");
}

builder.schedule(hello_world.build());
builder.schedule(hello_world.build());

// ...
world.execute(dynec::tracer::Noop); // prints 1, 1 in unspecified order
world.execute(dynec::tracer::Noop); // prints 2, 2 in unspecified order
}

0 is the initial value of counter before the system is run the first time. If parameter states are defined in the function, the initial expression may use such parameters by name as well.

Global states

States can also be shared among multiple systems using the type as the identifier. Such types must implement the Global trait, which can be done through the #[global] macro:

#![allow(unused)]
fn main() {
#[derive(Default)]
#[dynec::global(initial = Self::default())]
struct MyCounter {
    value: i32,
}

#[system]
fn add_counter(
    #[dynec(global)] counter: &mut MyCounter,
) {
    counter.value += 1;
}

#[system]
fn print_counter(
    #[dynec(global)] counter: &MyCounter,
) {
    println!("{counter}");
}
}

If no initial value is specified in #[global], the initial value of a global state must be assigned in Bundle::register.

#![allow(unused)]
fn main() {
impl world::Bundle for Bundle {
    fn register(&mut self, builder: &mut world::Builder) {
        builder.schedule(add_counter.build());
        builder.schedule(print_counter.build());
        builder.global(MyCounter { value: 123 });
    }
}

// ...
world.execute(dynec::tracer::Noop); // prints 123 or 124 based on unspecified order
world.execute(dynec::tracer::Noop); // prints 124 or 125 based on unspecified order
}

The program panics if some used global states do not have an initial but Bundle::register does not initialize them.

Note that &T and &mut T are semantically different for global states. Multiple systems requesting &T for the same T may run in parallel in a multi-threaded runtime, but when a system requesting &mut T is running, all other systems requesting &T or &mut T cannot run until the system is complete (but other unrelated systems can still be scheduled).

Component access

As the name "ECS" implies, the most important feature is to manipulate the "E" and "C" from the "S".

Accessing simple components

Simple components can be accessed through ReadSimple or WriteSimple. First we declare the components we need, similar to in the previous chapters:

#![allow(unused)]
fn main() {
use dynec::{comp, system};

dynec::archetype!(Bullet);

#[comp(of = Bullet, required)]
struct Position(Vector3<f32>);
#[comp(of = Bullet, required, initial = Velocity(Vector3::zero()))]
struct Velocity(Vector3<f32>);
}

We want to update position based on the value of the velocity. Therefore we request reading velocity and writing position:

#![allow(unused)]
fn main() {
#[system]
fn motion(
    mut position_acc: system::WriteSimple<Bullet, Position>,
    velocity_acc: system::ReadSimple<Bullet, Velocity>,
) {
    // work with position_acc and velocity_acc
}
}

We will go through how to work with the data later.

When a system that requests WriteSimple<A, C> is running for some A and C, all other systems that request ReadSimple<A, C> or WriteSimple<A, C> cannot run until the system is complete. Therefore, if you only need to read the data, use ReadSimple instead of WriteSimple even though the latter provides all abilities that the former can provide.

Accessing isotope components

Isotope components are slightly more complex. A system may request access to some ("partial access") or all ("full access") discriminants for an isotope component.

Full access allows the system to read/write any discriminants for the isotope type, and lazily initializes new discriminants if they were not encountered before. Therefore, when a system using WriteIsotopeFull is running, all other systems that access the same component in any way (read/write and full/partial) cannot run until the system is complete; when a system using ReadIsotopeFull is running, all other systems that use WriteIsotopeFull or WriteIsotopePartial on the same component cannot run until the system is complete.

The usage syntax of full accessors is similar to simple accessors:

#![allow(unused)]
fn main() {
#[system]
fn add(
    weights: ReadIsotopeFull<Bullet, IngredientWeight>,
    mut volumes: WriteIsotopeFull<Bullet, IngredientVolume>,
) {
    // ...
}
}

Partial access only requests specific discriminants for the isotope type. The requested discriminants are specified through an attribute:

#![allow(unused)]
fn main() {
#[system]
fn add(
    #[dynec(param)] &element: &Element,
    #[dynec(isotope(discrim = [element]))]
    weights: ReadIsotopePartial<Bullet, IngredientWeight, [Element; 1]>,
    #[dynec(isotope(discrim = [element]))]
    mut volumes: WriteIsotopePartial<Bullet, IngredientVolume, [Element; 1]>,
) {
    // ...
}
}

The discrim attribute option lets us specify which discriminants to access. The expression can reference the initial values of parameter states. However, mutating parameter states will not change the discriminants requested by the isotope. The third type parameter to ReadIsotopePartial/WriteIsotopePartial is the type of the expression passed to discrim.

Since a partial accessor can only interact with specific discriminants, multiple systems using WriteIsotopePartial on the same component type can run concurrently if they request a disjoint set of discriminants.

Iterating over entities

The recommended way to process all entities with accessors is to use the EntityIterator API. EntityIterator contains the list of initialized entities stored in an efficient lookup format, useful for performing bulk operations over all entities.

An EntityIterator can be joined with multiple accessors to execute code on each entity efficiently:

#![allow(unused)]
fn main() {
#[system]
fn move_entities(
    entities: system::EntityIterator<Bullet>,
    position_acc: system::WriteSimple<Bullet, Position>,
    velocity_acc: system::WriteSimple<Bullet, Velocity>,
) {
    for (_entity, (position, velocity)) in entities.entities_with_chunked((
        &mut position_acc,
        &velocity_acc,
    )) {
        *position += velocity;
    }
}
}

entities_with_chunked also supports isotope accessors, but they must be split for a specific discriminant first by calling split on the accessor (split_mut for mutable accessors):

#![allow(unused)]
fn main() {
#[system]
fn move_entities(
    #[dynec(param)] &element: &Element,
    entities: system::EntityIterator<Bullet>,
    velocity_acc: system::WriteSimple<Bullet, Velocity>,
    #[dynec(isotope(discrim = [element]))]
    weights_acc: system::ReadIsotopePartial<Bullet, IngredientWeight, [Element; 1]>,
) {
    let [weights_acc] = weights_acc.split([element]);
    entities
        .entities_with_chunked((
            &mut velocity_acc,
            &weights_acc,
        ))
        .for_each(|(_entity, (velocity, weight))| {
            *velocity /= weight;
        }
    }
}
}

Note: entities_with_chunked returns an iterator, so you may use it with a normal for loop as well. However, benchmarks show that for_each has performs significantly better than for loops due to vectorization.

You may also use par_entities_with_chunked instead to execute the loop on multiple threads. par_entities_with_chunked returns a rayon ParallelIterator, which has a very similar API to the native Iterator.

Partitions

The execution order of systems is actually undefined. Although the scheduler avoids executing systems requesting conflicting resources from running concurrently, it is undefined which system executes first. For example, consider the following code:

#![allow(unused)]
fn main() {
#[global(initial = Self::default())]
struct Clock {
    ticks: u32,
}

#[system]
fn update_clock(#[dynec(global)] clock: &mut Clock) {
    clock.ticks += 1;
}

#[system]
fn render_status(
    #[dynec(global)] status: &mut StatusBar,
    #[dynec(global)] clock: &Clock,
) {
    status.set_text(format!("Current time: {}", clock.ticks));
}

#[system]
fn render_progress(
    #[dynec(global)] progress: &mut ProgressBar,
    #[dynec(global)] clock: &Clock,
) {
    progress.set_progress(clock.ticks);
}
}

It is actually possible that render_status is executed before update_clock but render_progress is executed afterwards, in which case the progress bar and the status bar have different values in the end.

To avoid this problem, we introduce partitions, which can ensure some systems are executed before some others.

A partition is any thread-safe value that implements Debug + Eq + Hash. Any two equivalent values (with the same type) are considered to be the same partition. So for the example above, we can create a ClockUpdatedPartition type:

#![allow(unused)]
fn main() {
#[derive(Debug, PartialEq, Eq, Hash)]
struct ClockUpdatedPartition;

#[system(before(ClockUpdatedPartition))]
fn update_clock(#[dynec(global)] clock: &mut Clock) {
    clock.ticks += 1;
}

#[system(after(ClockUpdatedPartition))]
fn render_status(
    #[dynec(global)] status: &mut StatusBar,
    #[dynec(global)] clock: &Clock,
) {
    status.set_text(format!("Current time: {}", clock.ticks));
}

#[system(after(ClockUpdatedPartition))]
fn render_progress(
    #[dynec(global)] progress: &mut ProgressBar,
    #[dynec(global)] clock: &Clock,
) {
    progress.set_progress(clock.ticks);
}

// Other systems not related to `ClockUpdatedPartition`.
#[system]
fn other_systems() {}
}

Thus, update_clock is already executed when ClockUpdatedPartition is complete. Then render_status and render_progress are only executed after the partition is complete, so they render the value of the updated clock. This is illustrated by the following diagram:

---
displayMode: compact
---
gantt
    dateFormat s
    axisFormat %S
    tickInterval 1s
    section Worker 1
        other_systems   :0, 0.5s
        render_status   :1, 1s
    section Worker 2
        update_clock    :0, 1s
        render_progress :1, 0.5s
    ClockUpdatedPartition :milestone, 1, 1

In this example, Worker 1 cannot start executing render_status even though it is idle, until all dependencies for ClockUpdatedPartition have completed.

If the scheduler detected a cyclic dependency, e.g. if update_clock declares ClockUpdatedPartition it panics with an error like this:

Scheduled systems have a cyclic dependency: thread-safe system #0 (main::update_clock) -> partition #0 (main::ClockUpdatedPartition) -> thread-safe system #0 (main::update_clock)

dynec

ECS

Data

Logic

Archetypes

Components

Simple components

Initializer

Presence

Finalizers

Best practices

Small component structs

Optional types

Heap-allocated types

Isotope Components

Choosing the discriminant type

Initializer

Presence

Systems

Ticking

Parameter, local and global states

Parameter states

Local states

Global states

Component access

Accessing simple components

Accessing isotope components

Iterating over entities

Partitions

Online entity updates