The task of updating deep learning and AI models can incur significant costs concerning computational resources and energy usage, particularly when these models need to adapt to new tasks or changes in data. Researchers have created a groundbreaking approach that predicts these costs, providing users with the information they need to decide the optimal times for AI model updates, thereby enhancing the sustainability of AI.
The task of updating deep learning and AI models can incur significant costs concerning computational resources and energy usage, particularly when these models need to adapt to new tasks or changes in data. Researchers have created a groundbreaking approach that predicts these costs, providing users with the information they need to decide the optimal times for AI model updates, thereby enhancing the sustainability of AI.
“Previous research has been conducted to enhance the efficiency of deep learning model training,” explains Jung-Eun Kim, the lead author of the related study and an assistant professor of computer science at North Carolina State University. “Nevertheless, throughout a model’s lifetime, it may require multiple updates. As our findings demonstrate, updating an existing model is often much more cost-effective than starting anew.”
“To tackle sustainability challenges associated with deep learning AI, we need to examine computational and energy expenditures over the entire life cycle of a model, which includes the impacts of updates. Without a way to predict these costs beforehand, effective planning for sustainability efforts becomes impossible. Thus, our research holds particular significance.”
Training a deep learning model requires substantial computational effort, and users prefer to avoid updates for as long as possible. However, two scenarios can necessitate these updates. Firstly, the AI’s task may need to change. For instance, a model originally designed to classify digits and traffic signs might now need to also identify vehicles and pedestrians. This scenario is referred to as a task shift.
Secondly, the input data may evolve over time. This could mean integrating a new type of data or modifying how existing data is formatted. In either case, an update to the AI is needed to handle these changes, termed a distribution shift.
“No matter what prompts the need for an update, it is incredibly beneficial for AI developers to have a realistic estimate of the computational requirements for these updates,” Kim points out. “This information helps them make educated choices about when to perform the update and how to allocate computational resources accordingly.”
To estimate the upcoming computational and energy costs, the researchers introduced a novel tool named the REpresentation Shift QUantifying Estimator (RESQUE).
In essence, RESQUE enables users to compare the original dataset used to train a deep learning model with the new dataset intended for updating the model. This analysis provides estimates of the computational and energy expenditures involved in the update process.
The resulting costs are displayed as a unified index value, which can then be examined across five different metrics: epochs, parameter changes, gradient norms, carbon emissions, and energy consumption. The epochs, parameter change, and gradient norm metrics serve as indicators of the computational effort required for model retraining.
“Additionally, to clarify the implications of these figures within a broader sustainability framework, we also inform users of the energy required (in kilowatt hours) to retrain the model,” adds Kim. “Moreover, we estimate the carbon emissions (in kilograms) produced to generate that energy.”
The team conducted rigorous experiments utilizing various datasets, numerous distribution shifts, and different task adjustments to confirm the effectiveness of RESQUE.
“Our findings revealed that RESQUE’s predictions closely matched the actual costs associated with updating deep learning models,” Kim states. “Furthermore, we have consistently found that initiating a new model from the ground up requires significantly more computational power and energy than simply retraining an existing one.”
In the near term, RESQUE serves as a beneficial tool for anyone undertaking deep learning model updates.
“RESQUE aids users in budgeting computational resources for updates and forecasting the anticipated duration of the updates,” Kim elaborates.
“On a larger scale, this research provides crucial insights into the costs linked to deep learning models throughout their life cycle, which can facilitate informed decisions regarding their sustainability and practical application. If we aim for AI to remain functional and effective, these models must be both adaptable and sustainable.”
