IPA FE Subject A 2025 Question 1

Additional learning with specific data, so the model can be applied to the target task — fine-tuning takes a model that is already pre-trained and trains it further for one purpose.

A large language model is built in two stages. Pre-training trains a general model on a huge amount of text (for example, predicting the next word), so it learns the language and general knowledge. Fine-tuning then continues the training from those learned parameters with a much smaller set of data chosen for the purpose, such as question-and-answer pairs from one company or documents of one field, so that the model performs that task well. Starting from the pre-trained model needs far less data and computation than training from scratch.

Answer (d)

Why not others:

• •(a) — reinforcement learning, learning from rewards for good outputs, is a different learning method; it can be used in a later tuning stage (RLHF, reinforcement learning from human feedback), but fine-tuning is defined by the additional training on task-specific data
• •(b) — repeating the pre-training data adds nothing new for a target task and tends to overfit; fine-tuning uses different, task-specific data
• •(c) — learning from a large amount of general text data is pre-training itself, the stage before fine-tuning

Key rule: Pre-training learns general language from huge amounts of text; fine-tuning adds a small round of training on specific data to adapt that model to a particular task.

Basic Theory · AI-written, may contain errors