Accelerating Data-Driven Simulations For Deformable Bodies And Fluids