Continuous Training for Production ML in the TensorFlow Extended (TFX) Platform
Abstract
Large organizations rely increasingly on continuous ML
pipelines in order to keep machine-learned models continuously up-to-date with respect to data. In this scenario, disruptions in the pipeline can increase model staleness and
thus degrade the quality of downstream services supported by
these models. In this paper we describe the operation of continuous pipelines in the Tensorflow Extended (TFX) platform
that we developed and deployed at Google. We present the
main mechanisms in TFX to support this type of pipelines in
production and the lessons learned from the deployment of
the platform internally at Google.
pipelines in order to keep machine-learned models continuously up-to-date with respect to data. In this scenario, disruptions in the pipeline can increase model staleness and
thus degrade the quality of downstream services supported by
these models. In this paper we describe the operation of continuous pipelines in the Tensorflow Extended (TFX) platform
that we developed and deployed at Google. We present the
main mechanisms in TFX to support this type of pipelines in
production and the lessons learned from the deployment of
the platform internally at Google.
Research Areas
