If you go with the GPU round, there are a lot of NVIDIA cards to choose from (Kepler, Maxwell, Pascal, Volta, Turing).
If you go with a cloud provider, Amazon Web Services and Google Cloud Platform are the heavyweights, while startups such as Paperspace and Lambda Labs are also viable options.
If you work solo or in a startup, you should build or buy a 4x recent-architecture PC for model development. For model training, if you run many experiments, you can either buy shared server machines or use cloud instances.
If you work in a large company, you are more likely to rely on cloud instances for both model development and model training, as they provide proper provisioning and infrastructure to handle failures.