Collision filtering is used in the first place for collision detection with colliders. They are though used also in other places where groups of elements have to be checked for interacting with each other or not. The main use is though for collision detection.

A collision filter composes of a category and filter layer mask. A layer mask is a 64-bit bit flag value. Each bit can be individually set or cleared. 64 bits should be enough for the most complex filtering problems. Layer masks match each other if they share one or more set bits. Layer masks with no bits set do not match other layer masks even if they have no bits set either.

The category usually denotes what kind of type a collider belongs to. Typically this has only one bit set but more complex configurations are possible. For example it is possible to have disjoint groups of bits. A category could then have one bit set from each category.

The filter denotes which bits in the category of another collider have to be set for a collision to be possible. The filter typically has many bits set and states which categories can collide.

Category and filtering are compare between two colliders to determine if a collision is possible. The category layer mask of the first collider is matched against the filter layer mask of the second collider and vice versa. If both comparisons result yield a match a collision is possible otherwise not. This system allows for all kinds of complex scenarios. The image below shows an example scenario to visualize the working of filtering.

The example shows three colliders A, B and C. Each has one bit set in his category layer mask while their filter layer mask contains all accepted bits. In this example collider A is a trigger, collider B is an actor and collider C a special agent actor. We want regular actors to collide with triggers and other actors but special agent actors to collide only with other actors. Collider A has the trigger bit set. As an example filtering does not contain the trigger bit. This makes triggers not collide with other triggers. It has the actor bit set so it does collider with other actors. Collider B has the actor bit set in the category. Since it has actor and trigger bit set it can collide with other actors and with triggers. Now collider C is a special actor which is not supposed to interact with triggers because for example it is a ghost or some high tech secret agent with cool suit preventing him to trigger anything. It has the actor bit set in the category and as such would be hit by the trigger collider. It has though only the actor bit set in the filter so while it can still collider with actors (they might not see you but still bump into you) it can not collide with triggers. This fulfills the set out requirements.

The example below sets up the situation in the image.

// collider A, a trigger
var LayerMask category = LayerMask.new()
category.setBit( 6 )         // set bit 6 the trigger bit

var LayerMask filter = LayerMask.new()
filter.setBit( 2 )           // set bit 2
filter.setBit( 4 )           // set bit 4 the actor bit
filter.setBit( 5 )           // set bit 5

colliderA.setCollisionFilter( CollisionFilter.new( category, filter ) )      // set collision filter

// collider B, a regular actor
category = LayerMask.new()
category.setBit( 4 )         // set bit 4 the actor bit

filter = LayerMask.new()
filter.setBit( 3 )           // set bit 3
filter.setBit( 4 )           // set bit 4 the actor bit
filter.setBit( 6 )           // set bit 6 the trigger bit

colliderB.setCollisionFilter( CollisionFilter.new( category, filter ) )      // set collision filter

// collider C, a special agent actor
category = LayerMask.new()
category.setBit( 4 )         // set bit 4 the actor bit

filter = LayerMask.new()
filter.setBit( 0 )           // set bit 0
filter.setBit( 3 )           // set bit 3
filter.setBit( 4 )           // set bit 4 the actor bit
filter.setBit( 5 )           // set bit 5

colliderC.setCollisionFilter( CollisionFilter.new( category, filter ) )      // set collision filter