Generally game audio engines use a rather simple system for rendering audio: 1. position an audio source in space 2. apply effects on the audio clip depending on the area the object is in 3. shift volume / phase / pan depending on the angle and distance to the listener
But in real life audio propogates in waves through every medium and is reflected too. For example: If you yell into a wall, you will you much louder than if you yell to the horizon (crude example, I know).
So my idea is that you build an audio engine that works rather like the raytracing in graphics: Each pass (100 times a second --> 441 samples)
- You have an audio source sending Dry (meaning uneffected) sound in a fixed amount of directions, evenly distributed on the angle sphere.
- Every ray propogates for a limited amount of time through space.
- If one hits a wall a new (smaller) set of rays is created and propogates with less of an amplitude (depending on properties set to the wall)
- When this process is done and all rays reached their max length, each ray is treated as a linear function and the minimal distance between the ray and each listeners channel (for example L/R) is calculated.
- Phase shift and amplitude of each ray is now calculated in the following was
The amplitude is anti-proportional and the phase shift is proportional to the distance the ray travelled as well as the distance between the ray and the listener while this distance becomes less effective the further the ray has travelled.
I'm sorry if that wasn't clear enough - maybe this pseudocode may help you understand what I mean.
// Code example in C# style
// C++ would be more performant but this is easier to get
// Base classes for clarity
class AudioRay
{
Vector3 Source;
Vector3 Direction;
float DistanceOffset;
float MaxLength;
float Volume = 1;
public AudioRay(Vector3 src, Vector3 dir, float off, float maxLen, float vol);
public struct Intersection
{
Vector3 Point;
Vector3 InAngel;
Vector3 OutAngle;
float MaterialMuffling;
}
public Intersection GetIntersection()
{
// Todo - just concept - I don't really know how to calculate that
}
}
static class RayCaster
{
static Vector3 GetNearestPoint(Vector3 targetPoint, Vector3 source, Vector3 direction)
{
// Nearest point on a line to a point...
}
static List<AudiorRay> Cast(AudioRay ray, float maxLength)
{
List<AudiorRay> rays = new List<AudiorRay>();
AudioRay.Intersection intersection = ray.GetIntersection();
if(intersection!= null) // Only cast new rays when the old one intersected with a wall
{
float len = ray.DistanceOffset + (intersection.Point - ray.Source).Length;
if(len < maxLength) // ... and if they are below the max length
{
List<AudioRay> newRays = new List<AudioRay>();
// Add the new rays --> This is only the central one but the others will be evenly distributed
newRays.Add(new AudioRay(intersection.Point, interse.OutAngle, len, maxLenght, vol * intersection.MaterialMuffling));
// Cast the new rays but
newRays.ForEach(t => rays.AddRange(Cast(t)));
}
}
}
}
// Main class
class AudioSource
{
Vector3 Position = new Vector3(0, 0, 0);
Vector3 ListenerPos = new Vector3(10, 20, 30);
int StartRayCount = 162;
int BounceRayCount = 11;
float BounceRayAngle = 25;
float MaxRayLength = 1000;
// The dry wave parameter is not the buffer but the whole file so we can adress
// every sample all over the file.
// I suspect a 512 sample out buffer (~12 ms)
// wavePositionOffset = position in the track
public float[] Pass(float[] dryWave, int wavePositionOffset)
{
List<AudioRay> rays = new List<AudioRay>();
int samples = StartRayCount;
float offset = 2.0f / (float)samples;
float increment = Math.Pi * ( 3.0f - Math.Sqrt(5.0f));
// Create source rays
for(int i = 0; i < samples; i++)
{
float y = ((i * offset) - 1f) + (offset / 2f);
float r = Math.Sqrt(1f - Math.Pow(y, 2f));
float phi = (i % samples) * increment;
float x = Math.Cos(phi) * r
float z = Math.Sin(phi) * r
AudioRay ray = new AudioRay();
ray.Source = Position;
ray.Direction = new Vector3(x, y, z).Normalized;
ray.DistanceOffset = 0f;
ray.MaxLength = MaxRayLength;
rays.Add(ray);
rays.AddRange(RayCaster.Cast(ray, MaxRayLength));
}
// Create the out buffer
float[] oB = new float[512]();
float distanceVolumeFalloff = -0.01; // Complete silence after 100 m
float phaseShift = 20; // 20 samples / m shift
// calculate all rays phases and volumes
foreach(AudioRay ray in rays)
{
for(int i = 0; i < 512; i++)
{
Vector3 nearestPoint = GetNearestPoint(ListenerPos, ray.Position, ray.Direction);
float dist = ray.DistanceOffset + (nearestPoint - ray.Position);
// vol can't go below 0
// Respect the material properties stored in RayVolume
float vol = Math.Max(1 - dist * distanceVolumeFalloff, 0f) * ray.Volume;
int sampleOffset = (int)(dist * phaseShift);
int pos = wavePositionOffset-sampleOffset
if(pos > 0 && pos < dryWave.Length)
oB[i] += dryWave[pos] * vol;
}
}
return oB;
}
}
My question here is: Why don't engines support this audio model yet? I don't think there would be too much CPU overhead since reverberation is essentially the same and I "implemented" performance values (for example StartRayCount and MaxRayLenght). The only thing that might be (performance wise) hard to do is the raycasting. But your GPU does this millions of times and were talking maybe several hundreds say 100 times a second. This is still way less than what a 3D video scene does.
Pixels / second: 1920 x * 1080 y * 60 FPS = 124,416,000 only Full HD - Quadrupel for 4K
Rays / second: say about 200 to 1000? * 100 rays/second = 200,000 to 1,000,000
Before anyone cries out loud: I did my research and I've read this.
