atomr-accel-patterns
cargoUniversal GPU actor blueprints for atomr-accel-cuda: DynamicBatchingServer, InferenceCascade, ModelReplicaPool, FairShareScheduler, ModelHotSwapServer, SpeculativeDecoder, MoeRouter.
Audits
No audits for this package yet.
Universal GPU actor blueprints for atomr-accel-cuda: DynamicBatchingServer, InferenceCascade, ModelReplicaPool, FairShareScheduler, ModelHotSwapServer, SpeculativeDecoder, MoeRouter.